Information extraction, argumentation information extraction, and data processing method for information extraction

Abstract

This specification provides data processing methods for information extraction, argumentative information extraction, and information extraction. The information extraction method includes: acquiring a document to be extracted and topic statements, wherein the document to be extracted contains multiple initial statements; inputting the topic statements and multiple initial statements into the encoder of an information extraction model to encode a statement feature set, wherein the information extraction model is a pre-trained text generation model; inputting the statement feature set into a relation extraction layer to extract the statement relationships between statements, constructing statement relation features, wherein the statement relationships are the logical relationships between statements; and inputting the statement feature set and statement relation features into a decoder to decode the target information in the multiple initial statements. Utilizing statement relationships—the more semantically rich and flexible logical relationships between statements—for information extraction improves the accuracy and generalization ability of information extraction.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to an information extraction method. Background Technology

[0002] With the development of computer technology, document argument mining based on neural network models has greatly improved the efficiency, accuracy, and comprehensiveness of arguments on a specific topic.

[0003] Currently, argument extraction models based on neural networks, given a defined topic text, extensively search a large number of documents for extraction, analyze the text within, and extract a wealth of target information, such as argumentative, sentiment, and grammatical information. This provides comprehensive and accurate arguments in subsequent applications. Neural network models include two categories: text generation models and relation discrimination models. Text generation models generate and fill in table content based on a pre-defined template. However, this table-filling method limits the extraction of target information to the constraints of the template, resulting in insufficient generalization ability. Relation discrimination models extract word-level target information based on the word relationships within the document. However, given the limited semantic information expressed by words, the accuracy of the obtained word-level target information is insufficient. Therefore, a method with high generalization ability and high accuracy is urgently needed. Summary of the Invention

[0004] In view of the above, embodiments of this specification provide an information extraction method. One or more embodiments of this specification also relate to another information extraction method, a method for extracting argumentative information, a data processing method for information extraction, an information extraction apparatus, another information extraction apparatus, an argumentative information extraction apparatus, a data processing apparatus for information extraction, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, an information extraction method is provided, comprising:

[0006] Obtain the document to be extracted and the topic statements, wherein the document to be extracted contains multiple initial statements;

[0007] The topic statement and multiple initial statements are input into the encoder of the information extraction model to encode and obtain a set of statement features. The information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0008] The statement feature set is input into the relation extraction layer to extract the statement relations between statements and construct the statement relation features, where the statement relations are the logical relations between statements;

[0009] Input the statement feature set and statement relationship features into the decoder to decode and obtain the target information in multiple initial statements.

[0010] According to a second aspect of the embodiments of this specification, an information extraction method is provided, applied to a cloud-side device, comprising:

[0011] The receiving end device sends the document to be extracted and the topic statement, wherein the document to be extracted contains multiple initial statements;

[0012] The topic statement and multiple initial statements are input into the encoder of the information extraction model to encode and obtain a set of statement features. The information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0013] The statement feature set is input into the relation extraction layer to extract the statement relations between statements and construct the statement relation features, where the statement relations are the logical relations between statements;

[0014] Input the statement feature set and statement relationship features into the decoder to decode and obtain the target information in multiple initial statements;

[0015] The target information is fed back to the end-side device.

[0016] According to a third aspect of the embodiments of this specification, a method for extracting argument information is provided, comprising:

[0017] Obtain the document to be extracted and the topic statements, wherein the document to be extracted contains multiple initial statements;

[0018] The topic statement and multiple initial statements are input into the encoder of the argument information extraction model to obtain a set of statement features. The argument information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0019] The statement feature set is input into the relation extraction layer to extract the argumentation relationship between each statement and construct the argumentation relationship feature, where the argumentation relationship is the argumentation logic relationship between statements;

[0020] Input the statement feature set and the argument relationship feature into the decoder to decode and obtain the target argument information in multiple initial statements.

[0021] According to a fourth aspect of the embodiments of this specification, a data processing method for information extraction is provided, applied to a cloud-side device, comprising:

[0022] Obtain the sample dataset, which includes multiple sample statements, each labeled with a relationship tag. The sample statements include the sample topic statement and the sample initial statement corresponding to the sample topic statement.

[0023] The sample topic statement and the sample initial statement are input into the encoder of the information extraction model to encode the sample statement feature set. The information extraction model also includes a relation extraction layer and a decoder.

[0024] The sample statement feature set is input into the relationship extraction layer to extract the sample statement relationships between each sample statement and construct the sample statement relationship features, where the sample statement relationship is the logical relationship between sample statements;

[0025] Input the sample statement feature set and sample statement relationship features into the decoder to decode and obtain the sample target text information and sample information type of the sample target text information in multiple sample initial statements;

[0026] Based on the target text information and sample sentences, the first loss value is calculated, and based on the sample information type and the sample sentence relationship label, the second loss value is calculated.

[0027] Based on the first and second loss values, the model parameters of the encoder, relation extraction layer and decoder are adjusted, and the trained information extraction model is obtained when the preset training termination condition is met.

[0028] The model parameters of the information extraction model are sent to the edge device.

[0029] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0030] Memory and processor;

[0031] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the above-mentioned information extraction method, argument information extraction method, or information extraction data processing method.

[0032] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-described information extraction method, argument information extraction method, or information extraction data processing method.

[0033] According to a seventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described information extraction method, argument information extraction method, or information extraction data processing method.

[0034] In this embodiment, a document to be extracted and topic sentences are obtained, wherein the document to be extracted contains multiple initial sentences; the topic sentences and multiple initial sentences are input into the encoder of the information extraction model to encode and obtain a sentence feature set, wherein the information extraction model is a pre-trained text generation model, and the information extraction model also includes a relation extraction layer and a decoder; the sentence feature set is input into the relation extraction layer to extract the sentence relationships between each sentence and construct sentence relationship features, wherein the sentence relationship is the logical relationship between sentences; the sentence feature set and sentence relationship features are input into the decoder to decode and obtain the target information in the multiple initial sentences. This approach fully explores the relationship between the main statement and the initial statements in the document to be extracted, constructing statement relationship features corresponding to the statement features. Based on the target information decoded from the statement relationship features and statement feature set, information extraction is performed using the logical relationship between statements that contains richer semantic information, thus improving the accuracy of information extraction. On the basis of the text generation model, the approach adaptively determines the relationship between statements for information extraction, rather than filling in content based on a pre-set table template. This more flexibly captures the structural logical relationship between the statements in the document to be extracted, improving the generalization ability of the model and the generalization ability of information extraction. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating an information extraction method provided in one embodiment of this specification;

[0036] Figure 2 This is a flowchart of another information extraction method provided in one embodiment of this specification;

[0037] Figure 3 This is a flowchart illustrating a method for extracting argumentative information according to one embodiment of this specification;

[0038] Figure 4 This is a flowchart illustrating a data processing method for information extraction provided in one embodiment of this specification;

[0039] Figure 5 This is a schematic diagram of data processing in an information extraction method provided in one embodiment of this specification;

[0040] Figure 6 This is a front-end interaction diagram of an information extraction method provided in one embodiment of this specification;

[0041] Figure 7 This is a flowchart illustrating the processing procedure of an information extraction method for argumentative essay writing, provided in one embodiment of this specification.

[0042] Figure 8 This is a schematic diagram of the structure of an information extraction device provided in one embodiment of this specification;

[0043] Figure 9 This is a schematic diagram of another information extraction device provided in one embodiment of this specification;

[0044] Figure 10 This is a schematic diagram of the structure of an argument information extraction device provided in one embodiment of this specification;

[0045] Figure 11 This is a schematic diagram of the structure of a data processing device for information extraction provided in one embodiment of this specification;

[0046] Figure 12 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0047] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0048] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0049] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0050] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0051] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0052] Table Filling: Based on a pre-defined table template, the information generated by the text generation model is filled into the corresponding table. For example, if the table template corresponds to an argumentative relationship and includes: thesis statement, evidence investigation, and explanation of evidence, then the corresponding text information is generated and filled in.

[0053] Argument Quadruple: Textual information about argument mining in a document, including four information types: argument, evidence, position, and evidence type.

[0054] Argument information extraction: The process of extracting argument quadruples from a document to be extracted.

[0055] Argument extraction: Extract argument text information corresponding to argument types of statements on a specific topic from a document to be extracted.

[0056] Argument extraction: Extracting argumentative text information of the types of arguments that support the argument from a document to be extracted.

[0057] Position Classification: For a piece of argumentative text information of the type of argument, the text information of that type of argumentative text information is determined to either support or oppose the topic statement based on its logical relationship with the topic statement.

[0058] Argument Type Classification: For a given argument text, determine its type, including but not limited to: research type, expert type, example type, and exposition type. Specifically, research type indicates the argument is based on scientific research; expert type indicates it is based on expert explanation; example type indicates it is based on case study; and exposition type indicates it is based on logical exposition.

[0059] Text generation model: A generative model for text processing. Given the label value y of a sample text, the text generation model models the conditional probability of the feature vector x of the sample text, that is, models the conditional probability p(x|y). It studies what probability distribution each sample text follows.

[0060] Text discriminative model: A discriminative model for text processing. The text discriminative model models the probability of the label value y of the sample text given the feature vector x of the sample text, that is, it models the conditional probability p(y|x). This is generally used for component processing, that is, given a sample text x, it calculates the probability that it belongs to each class.

[0061] Transformer model: A neural network model based on attention mechanism, which calculates and analyzes the features of data through attention.

[0062] BERT (Bidirectional Encoder Representation from Transformers) model: A neural network model with bidirectional attention encoding representation function.

[0063] T5 (Text-To-Text Transfer Transformer) model: a translation model for text-to-text conversion.

[0064] Implicit vector: A type of deep feature vector, which is a feature encoding vector obtained by deeply extracting high-dimensional hidden features from the features.

[0065] Attention mechanisms selectively focus on useful information while ignoring useless information. This mechanism, often referred to as an attention mechanism, manifests in neural network models as the selective encoding and decoding of feature vectors. It includes self-attention, cross-attention, and multi-head attention mechanisms.

[0066] Biaffine feature transformation: a high-order mutual information feature transformation, such as second-order and higher mutual information feature transformations.

[0067] Bilinear feature transformation: a low-order mutual information feature transformation, such as a first-order mutual information feature transformation.

[0068] This specification provides an information extraction method, and also relates to another information extraction method, an argument information extraction method, an information extraction data processing method, an information extraction device, another information extraction device, an argument information extraction device, an information extraction data processing device, a computing device, a computer-readable storage medium, and a computer program, which will be described in detail in the following embodiments.

[0069] See Figure 1 , Figure 1A flowchart of an information extraction method according to an embodiment of this specification is shown, including the following specific steps:

[0070] Step 102: Obtain the document to be extracted and the topic statements, wherein the document to be extracted contains multiple initial statements.

[0071] The embodiments in this specification apply to the client and / or server of applications, web pages, or mini-programs with information extraction functions.

[0072] A topic statement is a textual statement that defines a specific topic. For example, the importance of cultural development and economic development. A topic statement clearly defines the scope of the topic as cultural development and economic development. The topic statement is predetermined by the user, and it enables the extraction of relevant target information from the document to be extracted.

[0073] The document to be extracted is a document containing text information corresponding to the topic statement. Examples include academic papers, news magazine reports, social media messages, and records of opinions. The document to be extracted must be in a text format, such as doc, txt, pdf, caj, or html. For example, corresponding to the topic statement "the importance of cultural development and economic development," we might identify a PDF academic paper titled "A Study on the Laws of Modern Economic Development" and an HTML record of expert opinions titled "Expert XX Discusses the New Stage of Cultural Development." At least one document must be extracted. The document to be extracted can be directly uploaded by the user or retrieved based on the topic statement; this is not limited here. The document to be extracted is stored in a document database, which can be an open-source database or a historical database; this is not limited here.

[0074] The initial statements are text statements in the document to be extracted. In step 102, the statement relationships between the initial statements and the topic statements, as well as the statement relationships between the initial statements themselves, are not determined. For example, the document "Research on the Laws of Modern Economic Development" contains multiple initial statements such as "The growth of modern economic development mainly depends on the combined effect of factors A and B" and "The economic development laws reflected in recent years are...".

[0075] For example, the application receives the topic statement "nutritional value of beef and mutton" input and uploaded by the user through the client of the information extraction application. Based on the topic statement, the application performs a document retrieval to obtain the PDF document "Overview of Meat Nutrition", which contains N initial statements.

[0076] The document to be extracted and the topic statements are obtained. The document to be extracted contains multiple initial statements, which lay the foundation for subsequent statement encoding.

[0077] Step 104: Input the topic statement and multiple initial statements into the encoder of the information extraction model to encode and obtain the statement feature set. The information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0078] The information extraction model is a neural network model with information extraction capabilities. This model is a text generation model, including but not limited to: the Transformer model, the BERT model, and the T5 model. The information extraction model includes an encoder, a relation extraction layer, and a decoder. It achieves information extraction by performing feature encoding, feature transformation, and feature decoding on the text information input to the model. The information extraction model is a text generation model with feature enhancement capabilities.

[0079] The encoder is a feature vector encoder for a text generation model with text feature encoding capabilities. By encoding the features of the input text, it obtains the corresponding feature encoding vector, thereby realizing the extraction of text features, which are then used for subsequent feature transformation and feature decoding.

[0080] The statement feature set is the collection of statement features corresponding to each statement, and each statement feature is a feature encoding vector for each statement. Statement features are implicit vectors, a type of statement-level feature encoding vector, which contains richer semantic information compared to word-level feature encoding vectors.

[0081] The encoder of the information extraction model is used to input the topic statement and multiple initial statements to obtain a set of statement features. Specifically, the encoder of the information extraction model is used to input the topic statement and multiple initial statements to perform implicit vector feature encoding on each statement to obtain a set of statement features with multiple statement features.

[0082] For example, the topic statement S0 "Nutritional value of beef and mutton" and N initial statements Si (i∈[1,N]) from "A Review of Meat Nutrition" are input into the encoder of the T5 model to perform implicit vector feature encoding on the N+1 statements, resulting in a statement feature set Hi (i∈[1,N+1]) of N+1 statement features.

[0083] In the embodiments of this specification, by performing sentence-level feature encoding on the topic statement and multiple initial statements, a set of sentence features containing rich semantic information is obtained, which lays the feature foundation for the subsequently extracted sentence relationships while ensuring the accuracy of the sentence relationships.

[0084] In one optional embodiment of this specification, the encoder includes an embedding layer and an encoding layer;

[0085] Correspondingly, step 104 includes the following specific steps:

[0086] The topic statement and multiple initial statements are input into the embedding layer, and the topic statement and multiple initial statements are concatenated to obtain the statement embedding sequence;

[0087] The embedded sequence of statements is input into the encoding layer, and the resulting set of statement features is obtained through encoding.

[0088] The embedding layer is the feature embedding layer of the text generation model. By concatenating the text, it achieves the sequential connection between sentences, resulting in the corresponding sentence embedding sequence. The encoding layer is the encoding layer that performs deep feature encoding on the sentence sequence. By performing deep feature encoding on the low-dimensional sequence, it obtains a high-dimensional feature encoding vector, which is the implicit vector in the embodiments of this specification.

[0089] The topic statement and multiple initial statements are input into the embedding layer. The topic statement and multiple initial statements are concatenated to obtain a statement embedding sequence. Specifically, the topic statement and multiple initial statements are input into the embedding layer, concatenated, and corresponding statement identifier characters and delimiters are added between each statement to obtain the statement embedding sequence. Further, the topic statement and multiple initial statements are concatenated and feature embedding is performed. Here, the delimiters represent the division between statements, including start and end delimiters. Statement identifier characters are the identifier characters corresponding to each statement, used to uniformly locate and identify each statement, and used to identify the feature vector of the statement relationship features in step 106. For example, if the statement identifier character corresponding to the first statement is "#0" and the statement identifier character corresponding to the second statement is "#1", then the statement relationship feature between the first and second statements in the statement relationship features is represented as "#0#1:1". Feature embedding involves low-dimensional feature encoding of each character in the statement to obtain a low-dimensional feature vector, i.e., the statement embedding sequence. The statement embedding sequence can be represented as: Among them, I(S) i ) represents the statement embedding sequence of the i-th sentence. <ss>As the starting separator, <se>The terminating delimiter is #i, and the statement identifier is #i. The m-th character in the statement.

[0090] The sentence embedding sequence is input into the encoding layer, and the resulting sentence feature set is encoded. Specifically, the sentence embedding sequence is input into the encoding layer, and implicit vectors are calculated from the embedded sequence to obtain the sentence feature set. Taking the encoder of the T5 model as an example, the formula for calculating the implicit vectors is shown in Formula 1:

[0091] H = T5_Encoder([I(S 0 ), I(S 1 )……I(S n )])

[0092] Where H is the calculated implicit vector, and the length of this implicit vector is L, I(S n ) represents the statement embedding sequence of the nth statement.

[0093] For example, the topic statement S0 "Nutritional value of beef and mutton" and N initial statements Si (i∈[1,N]) from "A Review of Meat Nutrition" are input into the embedding layer of the T5 model. The topic statement and multiple initial statements are concatenated, and the corresponding statement identifier character "#i" and separator "" are added between each statement. <ss> ”，" <se>", to obtain the statement embedding sequence I(S) n The embedded sequence of statements is input into the encoding layer. Using Formula 1, the implicit vector of the embedded sequence of statements is calculated and encoded to obtain the statement feature set Hi (i∈[1,N+1]).

[0094] In the embodiments of this specification, the embedding layer is used to concatenate the topic statement and multiple initial statements to reconstruct the statement, thereby improving the accuracy of subsequent encoding. The encoding layer is used to perform statement-level feature encoding on the topic statement and multiple initial statements to obtain a set of statement features containing rich semantic information. This lays the feature foundation for the subsequently extracted statement relationships while ensuring the accuracy of the statement relationships.

[0095] In one optional embodiment of this specification, the encoding layer is an attention feature encoding layer;

[0096] Correspondingly, the statement is embedded into the sequence input encoding layer, and the statement feature set is obtained by encoding, including the following specific steps:

[0097] The sentence embedding sequence is input into the attention feature encoding layer, and attention feature encoding is performed on the sentence embedding sequence to obtain the sentence feature set.

[0098] The attention feature encoding layer is a feature encoding layer with an attention mechanism. This includes, but is not limited to, the encoding layers of the Transformer model, the BERT model, and the T5 model. Specifically, it includes a multi-attention layer and a feed-forward network (FFN). The multi-attention layer performs attention calculations on the input feature encoding vector, and the feed-forward layer performs feature fitting on the input feature encoding vector.

[0099] The sentence embedding sequence is input into the attention feature encoding layer, and attention features are encoded on the sentence embedding sequence to obtain the sentence feature set. Specifically, the sentence embedding sequence is input into the attention feature encoding layer, attention is calculated on the sentence embedding sequence to obtain attention features, and feature fitting is performed on the attention features to obtain the sentence feature set. The attention calculation is shown in Formula 2.

[0100]

[0101] Where Attention is the attention feature, Softmax() is a normalization function, Q is the query vector, K is the key vector, V is the value vector, and d k A preset temperature coefficient is used to reduce the gradient.

[0102] For example, embedding statements into sequence I(S) n Input the attention feature encoding layer of the T5 model, use Formula 2 to calculate the attention of the embedded sequence of sentences, obtain the attention feature Attention, and perform feature fitting on the attention feature to obtain the sentence feature set Hi (i∈[1,N+1]).

[0103] In the embodiments of this specification, a set of sentence features is obtained by encoding, and then an attention feature encoding layer with an attention mechanism is used for encoding, thereby improving the accuracy of the encoded set of sentence features.

[0104] Step 106: Input the statement feature set into the relation extraction layer, extract the statement relations between each statement, and construct the statement relation features, where the statement relations are the logical relations between statements.

[0105] The relation extraction layer is a feature transformation layer that extracts relationships between statements. In this layer, logical relationships between corresponding statements are determined using at least two statement features, thus obtaining the statement relationships. The relation extraction layer implements feature enhancement, that is, it extracts features not present in the statement features themselves, namely, the feature relationships between statement features, which are used to guide feature decoding in step 108, improving the accuracy of the obtained target information.

[0106] Sentence relationships refer to the logical relationships between sentences, including but not limited to argumentative, emotional, and grammatical relationships. For example, if sentence 1 states the argument and sentence 2 provides supporting evidence, the sentence relationship is argumentative. Another example is a statement from a social media post, and sentence 2 expresses an emotional response such as "I agree" or "It's completely unreasonable," the sentence relationship is emotional. Yet another example is a main clause in sentence 1, and sentence 2 is a conditional clause modifying sentence 1, the sentence relationship is grammatical.

[0107] The statement relationship feature is the feature encoding vector corresponding to the statement relationship. For example, for three statements, in terms of sentiment relationship, positive sentiment is 0, neutral sentiment is 1, and negative sentiment is 2. The relationship between the first and second statements is positive sentiment, the relationship between the second and third statements is neutral sentiment, and the relationship between the first and third statements is negative sentiment. The feature encoding vector in diagonal matrix form corresponding to this statement relationship is: The feature encoding vector is determined to be a statement relationship feature.

[0108] The statement feature set is input into the relation extraction layer to extract the statement relationships between statements, thus constructing statement relationship features. Specifically, the statement feature set is input into the relation extraction layer, and based on the feature correlations between the statement features in the set, feature classification is performed to determine the statement relationships between statements, thus constructing statement relationship features. The feature correlations between features are determined by the relation extraction layer based on feature transformations using at least two statement features. Feature classification determines the type of statement relationship between statements, and statement relationship features are constructed based on the type of statement relationship.

[0109] For example, the statement feature set Hi (i∈[1, N+1]) is input into the relation extraction layer. Based on the feature correlation Xij (i∈[1, N+1], j∈[1, N+1]) between each statement feature in the statement feature set: Hi and Hj (i∈[1, N+1], j∈[1, N+1]), feature classification is performed to determine the argumentation relation Yij (i∈[1, N+1], j∈[1, N+1]) between each statement, and the feature encoding vector Y of the argumentation relation is constructed.

[0110] In the embodiments of this specification, the logical relationship between statements, which contains richer semantic information, is extracted. The constructed statement relationship feature can more accurately represent the statement relationship between each statement. At the same time, it avoids the limitation of the predetermined table template and improves the adaptability of determining statement relationship.

[0111] In one optional embodiment of this specification, the relation extraction layer includes a first feature transformation layer, a second feature transformation layer, and a feature classification layer;

[0112] Correspondingly, step 106 includes the following specific steps:

[0113] The first statement feature of the first statement is input into the first feature transformation layer to obtain the first statement transformation feature. The second statement feature of the second statement is input into the second feature transformation layer to obtain the second statement transformation feature. The first statement and the second statement are any two statements from the topic statement and multiple initial statements.

[0114] Input the first statement transformation features and the second statement transformation features into the feature classification layer to obtain the statement relationship between the first statement and the second statement;

[0115] Based on the relationships between statements, statement relationship features are constructed.

[0116] The relation extraction layer extracts the sentence relationships between statements. When sentence relationships are multifaceted, such as grammatical relationships (including noun clauses, relative clauses, adverbial clauses, etc.) and coordinate clauses (causal, contrastive, etc.), the relation extraction layer needs to classify these relationships while confirming them. This involves performing targeted feature transformations on the input statements before classifying the sentence relationships. The relation extraction layer is a bilinear or biaffine feature transformation layer, serving as a feature enhancement module for the information extraction model.

[0117] The feature transformation layer is a feature transformation layer that performs feature transformation on statement features based on statement relationships. It can be a linear transformation or a non-linear transformation, representing a specific type of high-dimensional feature transformation. For example, if the statement relationship is an argumentative relationship, two feature transformation layers are set up: one for argument feature transformation and one for evidence feature transformation. These layers adaptively transform the statement features to extract the argument features and evidence features from the statements. In the embodiments of this specification, the first feature transformation layer is the argument feature transformation layer, and the second feature transformation layer is the evidence feature transformation layer. The first feature transformation layer performs argument-type feature transformation on the input first statement, and the second feature transformation layer performs evidence-type feature transformation on the input second statement. The terms "first" and "second" do not limit the feature transformation layers or the statements themselves.

[0118] Statement transformation features are feature encoding vectors for statements based on statement relationships. They are high-dimensional feature encoding vectors that adapt to statement relationships. They can be understood as transformation features that adapt statement features to statement relationships.

[0119] The feature classification layer is a classification layer that classifies the relationships between statements based on transformed features. For example, if the relationship between statements is an argumentative relationship, the transformed features output by the two feature transformation layers are argument features and evidence features. The feature classification layer determines whether the relationship between the corresponding statements is an argument-evidence relationship, and what type of evidence it is, based on these argument features and evidence features. The feature classification layer can be a bilinear feature classification layer or a biaffine feature classification layer.

[0120] The first statement's first statement features are input into the first feature transformation layer to obtain the first statement transformed features. Similarly, the second statement's second statement features are input into the second feature transformation layer to obtain the second statement transformed features. Specifically, the first statement's first statement features are input into the first feature transformation layer, and based on the statement relationships, an argument-type feature transformation is performed on the input first statement to obtain the first statement transformed features. The second statement's second statement features are input into the second feature transformation layer, and based on the statement relationships, an argument-type feature transformation is performed on the input second statement to obtain the second statement transformed features. When the feature transformation layer is a linear transformation, the calculation formula for the transformed features is shown in Formula 3.

[0121]

[0122]

[0123] Where, x i Let represent the statement transformation features of the i-th sentence, and Linear1() represent the linear feature transformation of the first feature transformation layer. Let x be the statement feature of the i-th sentence. j Let be the statement transformation feature of the j-th sentence, and Linear2() be the linear feature transformation of the second feature transformation layer. The statement feature of sentence j.

[0124] The first and second statement transformation features are input into the feature classification layer to obtain the statement relationship between the first and second statements. Specifically, the first and second statement transformation features are input into the feature classification layer, and relationship features are obtained based on these features. The relationship features are then classified to obtain the statement relationship between the first and second statements. The relationship features reflect the feature correlation between at least two features.

[0125] The formula for calculating the relationship between statements is shown in Formula 4:

[0126]

[0127] Wherein, P(y ij The classification result representing the relationship between the i-th and j-th sentences is represented by probability. Softmax() is a normalization function. U and W i and W j Here, U represents the model parameters for the feature classification layer, and W represents the global weight parameters. i and W j These are local weight parameters.

[0128] Based on the relationships between statements, statement relationship features are constructed. Specifically, each statement is labeled based on its relationship with another statement to construct the statement relationship features.

[0129] For example, the first statement feature H1 of the first statement is input into the first feature transformation layer. Based on the argumentation relationship, the first statement feature is transformed into argument feature x1. The second statement feature H2 of the second statement is input into the second feature transformation layer. Based on the argumentation relationship, the second statement feature is transformed into evidence feature x2. The first statement transformation feature and the second statement transformation feature are input into the classification layer. Based on the first statement transformation feature and the second statement transformation feature, the relationship feature y12 is obtained. The relationship feature is classified to obtain the argumentation relationship Y12 between the first statement and the second statement: topic-argument, argument-evidence, topic-stance, or argument-evidence type. Based on the argumentation relationship between each statement, N+1 statements are labeled to construct the feature encoding vector Y of the argumentation relationship.

[0130] In the embodiments of this specification, a feature transformation layer is used to adaptively transform the statement features, resulting in statement transformation features, which improves the adaptability of the statement features. A feature classification layer is used to determine the statement relationships between statements based on the statement transformation features, improving the accuracy of determining statement relationships. Furthermore, statement relationship features are constructed based on these relationships, further improving the accuracy of the determined statement relationship features, and consequently improving the accuracy of the target information obtained through subsequent decoding.

[0131] In one optional embodiment of this specification, statement relationship features are constructed based on the statement relationships between statements, including the following specific steps:

[0132] Based on the relationships between statements, determine the corresponding type of statement relationship label;

[0133] Each statement is labeled based on its relational tags to construct relational features.

[0134] Statement relationship labels are quantized labels corresponding to the feature encoding vectors of statement relationships, including empty and non-empty labels. For example, for three statements, in terms of sentiment relationship, positive sentiment is labeled 0, neutral sentiment is labeled 1, and negative sentiment is labeled 2. The relationship between the first and second statements is positive sentiment, the relationship between the second and third statements is neutral sentiment, and the relationship between the first and third statements is negative sentiment. By labeling each statement based on the statement relationship labels, a feature encoding vector in diagonal matrix form is obtained.

[0135] It should be noted that the statement relationship feature can be abstractly understood as a diagonal matrix or as a table. For a document to be extracted containing n initial sentences, a table of size n*(n+1) is constructed, where each cell corresponds to a statement relationship label Yij. Statement relationship labels include empty labels and non-empty labels.

[0136] For example, based on the argumentative relationship between N+1 statements, the corresponding argumentative relationship label Yij is determined: 00 (topic-argument), 01 (argument-evidence), 10 (topic-stance), or 11 (argument-evidence type). Each statement is labeled based on the statement relationship label to construct the feature encoding vector Y of the argumentative relationship.

[0137] In the embodiments of this specification, the statement relationship is converted into the corresponding statement relationship feature based on the statement relationship label, thereby constructing a more accurate statement relationship feature for subsequent decoding.

[0138] Step 108: Input the statement feature set and statement relationship features into the decoder to decode and obtain the target information in multiple initial statements.

[0139] The target information is the relevant information in the initial statement corresponding to the topic statement. It can be the initial statement itself or at least one word within it. The target information can be at the word level or the statement level. Target information includes text information and information types within the text. For example, if the documents to be extracted include "Research on the Laws of Modern Economic Development" and "XX Experts Discuss the New Stage of Cultural Construction," the target information corresponding to the topic statement "The Importance of Cultural Construction and Economic Development" is: "Economic development is the primary goal at present": the argument; "The combined effect of factors A and B" - elaborating on the argument; "The economic development laws reflected in recent years are..." - elaborating on the argument.

[0140] The decoder is a feature vector decoder of a text generation model with text feature encoding function. It obtains the corresponding text information and the corresponding feature encoding vector through feature decoding, thereby realizing the extraction of text features, which are then used for subsequent feature transformation and feature decoding.

[0141] The statement feature set and statement relationship features are input into the decoder to decode and obtain the target information in multiple initial statements. Specifically, the statement feature set and statement relationship features are input into the decoder, and the statement relationship features are used as guiding information to decode the statement feature set to obtain the target information in multiple initial statements.

[0142] For example, the statement feature set Hi (i∈[1,N+1]) and the feature encoding vector Y of the argument relationship are input into the decoder of the T5 model. The feature encoding vector of the argument relationship is used as the guiding information to decode the statement feature set and obtain the target information in N initial statements: "Beef has a higher nutritional value than mutton" supports the argument; "Every 100g of beef contains 20g of protein" - survey evidence; "People in country A mainly eat beef, and their physical fitness ranks..." - example evidence.

[0143] In this embodiment, a document to be extracted and topic sentences are obtained, wherein the document to be extracted contains multiple initial sentences; the topic sentences and multiple initial sentences are input into the encoder of the information extraction model to encode and obtain a sentence feature set, wherein the information extraction model is a pre-trained text generation model, and the information extraction model also includes a relation extraction layer and a decoder; the sentence feature set is input into the relation extraction layer to extract the sentence relationships between each sentence and construct sentence relationship features, wherein the sentence relationship is the logical relationship between sentences; the sentence feature set and sentence relationship features are input into the decoder to decode and obtain the target information in the multiple initial sentences. This approach fully explores the relationship between the main statement and the initial statements in the document to be extracted, constructing statement relationship features corresponding to the statement features. Based on the target information decoded from the statement relationship features and statement feature set, information extraction is performed using the logical relationship between statements that contains richer semantic information, thus improving the accuracy of information extraction. On the basis of the text generation model, the approach adaptively determines the relationship between statements for information extraction, rather than filling in content based on a pre-set table template. This more flexibly captures the structural logical relationship between the statements in the document to be extracted, improving the generalization ability of the model and the generalization ability of information extraction.

[0144] In one optional embodiment of this specification, the decoder includes an attention feature decoding layer;

[0145] Correspondingly, step 108 includes the following specific steps:

[0146] The statement feature set and statement relationship features are input into the attention feature decoder. Attention decoding is performed on the statement feature set and statement relationship features to obtain target information from multiple initial statements.

[0147] The attention-based feature decoding layer is a feature decoding layer with an attention mechanism. This includes, but is not limited to, the decoding layers of the Transformer model, the BERT model, and the T5 model. Specifically, it includes a multi-attention layer and a feed-forward network (FFN). The multi-attention layer performs attention calculations on the input feature encoding vector, and the feed-forward layer performs feature fitting on the input feature encoding vector.

[0148] The sentence feature set and sentence relationship features are input into the attention feature decoder. Attention decoding is performed on the sentence feature set and sentence relationship features to obtain target information from multiple initial sentences. Specifically, the sentence feature set and sentence relationship features are input into the attention feature decoding layer. Attention is calculated on the sentence feature set and sentence relationship features to obtain attention features. Feature fitting is performed on the attention features to obtain target information from multiple initial sentences. The attention calculation is shown in Equation 2.

[0149] For example, the statement feature set Hi (i∈[1,N+1]) and the feature encoding vector Y of the argument relationship are input into the attention feature decoding layer. Attention is calculated on the statement feature set and the feature encoding vector of the argument relationship to obtain the attention feature. Feature fitting is performed on the attention feature to obtain the target information in N initial statements: "Beef has a higher nutritional value than mutton" supports the argument; "Every 100g of beef contains 20g of protein" - survey evidence; "People in country A mainly eat beef, and their physical fitness ranks..." - example evidence.

[0150] In the embodiments of this specification, target information from multiple initial statements is decoded, and an attention feature decoding layer with an attention mechanism is used for decoding, thereby improving the accuracy of information extraction.

[0151] In one optional embodiment of this specification, the target information includes target text information and the information type of the target text information;

[0152] Correspondingly, after step 108, the following specific steps are also included:

[0153] Based on the information type of the target text information, the target text information is grouped to obtain at least one target information group.

[0154] The target text information is the text information in the initial statement that corresponds to the topic statement. It can be the initial statement itself or at least one word in the initial statement. That is, the target text information can be at the word level or the statement level.

[0155] The information type of the target text information is the information type corresponding to the target text information. For example, if the statement relationship is an argumentative relationship, and the target text information is the third sentence in the document to be extracted, which is an argumentative statement and is a survey-type argumentative statement, then the argumentative statement and the survey-type argumentative statement are information types.

[0156] The target information group is a set of target text information and its information types according to preset grouping rules. For example, target text information may include statements supporting a type of argument, statements opposing a type of argument, statements providing evidence for an investigation, statements providing examples of evidence, and statements providing explanations of arguments. Grouping by stance yields two groups: {Statements supporting a type of argument; Statement providing evidence 1; Statement providing examples of evidence 1; Statement providing explanations of arguments 1} and {Statements opposing a type of argument; Statement providing evidence 2; Statement providing examples of evidence 2; Statement providing explanations of arguments 2}. Similarly, it can be divided according to the type of evidence presented, resulting in three target information groups: Statement providing evidence for an investigation, Statement providing examples of evidence, and Statement providing explanations of arguments. For argumentative relationships, each group can be divided according to argumentative plurals, such as argumentative quadruples.

[0157] Based on the information type of the target text information, the target text information is grouped to obtain at least one target information group. Specifically, the target text information is grouped according to a preset grouping rule based on the information type of the target text information to obtain at least one target information group.

[0158] It should be noted that in this embodiment of the specification, grouping can be implemented by setting a corresponding output layer in the information extraction model, or by setting a corresponding grouping module outside the model; no limitation is made here. The target information group can be obtained by converting according to a preset information template. Taking the argumentative relationship as an example, the specific implementation formula of the information conversion model is shown in Formula 5:

[0159]

[0160] Where, q k Sentence features that characterize the topic statement; Sentence features that characterize arguments, and information features that characterize stances. The syllogistic features of the supporting argument, t k Information features characterizing argument types, where M and N represent the number of argument statements and the number of supporting arguments, respectively. For argument statements supported by multiple supporting arguments, the separator "|" can be used to separate the different arguments and argument types. For documents containing multiple argument statements, we use the special symbol [SEP] to separate different Γ(q) statements. k ).

[0161] For example, the target information in N initial statements is: "Beef has a higher nutritional value than mutton" - supporting argument; "Every 100g of beef contains 20g of protein" - survey evidence; "People in country A mainly consume beef, and their physical fitness ranks..." - example evidence. Here, "Beef has a higher nutritional value than mutton" is the target text information, and the supporting argument is the corresponding information type; "Every 100g of beef contains 20g of protein" is the target text information, and the survey evidence is the corresponding information type; "People in country A mainly consume beef, and their physical fitness ranks..." is the target text information, and the example evidence is the corresponding information type. According to a preset grouping rule: one group is supported, which is based on the information type of the target text information. The target text information is grouped to obtain at least one target information group.

[0162] In the embodiments of this specification, the target text information is grouped based on its information type, which improves the readability of the output target information and enhances the user experience.

[0163] In one optional embodiment of this specification, the information extraction model is trained through the following steps:

[0164] Obtain the sample dataset, which includes multiple sample statements, each labeled with a relationship tag. The sample statements include the sample topic statement and the sample initial statement corresponding to the sample topic statement.

[0165] The sample topic statement and the sample initial statement are input into the encoder of the information extraction model to encode the sample statement feature set. The information extraction model also includes a relation extraction layer and a decoder.

[0166] The sample statement feature set is input into the relationship extraction layer to extract the sample statement relationships between each sample statement and construct the sample statement relationship features, where the sample statement relationship is the logical relationship between sample statements;

[0167] Input the sample statement feature set and sample statement relationship features into the decoder to decode and obtain the sample target text information and sample information type of the sample target text information in multiple sample initial statements;

[0168] Based on the target text information and sample sentences, the first loss value is calculated, and based on the sample information type and the sample sentence relationship label, the second loss value is calculated.

[0169] Based on the first and second loss values, the model parameters of the encoder, relation extraction layer, and decoder are adjusted, and the trained information extraction model is obtained when the preset training termination condition is met.

[0170] In information extraction models, the encoder and decoder are not highly correlated with sentence relationships, and pre-trained text generation models such as Transformer, BERT, and T5 can generally be used directly. However, the relation extraction layer is directly related to the extraction of sentence relationships and requires targeted pre-training. Therefore, in the embodiments of this specification, the fine-tuning of the encoder and decoder, as well as the pre-training of the relation extraction layer, are simultaneously implemented. Thus, a first loss value is obtained corresponding to the accuracy of the decoded target information, and a second loss value is set corresponding to the accuracy of the extracted sentence relationship features. Using both loss values ​​simultaneously to adjust the model parameters improves the model's adaptability.

[0171] The sample dataset is a collection of sample sentences used for training the information extraction model. It includes multiple sample sentences; since the information extraction model is a text generation model, the sample sentences serve as labeled samples, enabling judgment between the encoder and decoder. Multiple sample sentences are labeled with relational tags, acting as labeled samples for the relation extraction layer. The sample sentences include sample topic sentences and their corresponding initial sample sentences. The sample topic sentences are sample text sentences that define a specific topic, and the initial sample sentences are sample text sentences corresponding to the sample topic sentences. The sample sentences can be manually generated, for example, by a neural network model with text generation capabilities, or obtained from a sample database. This sample database can be a historical database or a public database, such as an open-source sample database. The relational tags for the sample sentences can be manually labeled or automatically labeled using relevant labeling algorithms, for example, by a neural network model with labeling capabilities.

[0172] The sample statement feature set is the set of sample statement features corresponding to each sample statement, and the sample statement features are the feature encoding vectors of each sample statement.

[0173] The sample statement relationship refers to the logical relationship between sample statements, including but not limited to argumentative, emotional, and syntactic relationships. The sample statement relationship feature is the feature encoding vector corresponding to the sample statement relationship.

[0174] The target text information of the sample is the text information in the initial sample statement that corresponds to the sample topic statement. It can be the initial sample statement itself or at least one word in the initial sample statement; that is, the target text information can be at the word level or the statement level. The sample information type of the target text information is the information type corresponding to the target text information. For example, if the statement relationship is an argumentative relationship, and the target text information is an argumentative statement, and this argumentative statement is a survey-type argumentative statement, then the argumentative statement and the survey-type argumentative statement are the sample information types.

[0175] The first loss value characterizes the performance of the encoder and decoder. The second loss value characterizes the performance of the relation extraction layer. The first and second loss values ​​include, but are not limited to: cross-entropy loss, probability distribution loss, autoregressive loss, L1 loss, L2 loss, etc.

[0176] The preset training termination conditions are pre-defined criteria for determining the end of training, including but not limited to: loss threshold, number of iterations, and all sample pairs have completed training.

[0177] The encoder is adjusted based on the first and second loss values. Specifically, a third loss value is calculated based on the first and second loss values, and then the encoder is adjusted using a gradient update method based on the third loss value. Specifically, the third loss value is calculated by weighting the first and second loss values. The encoder is then adjusted based on this third loss value using a gradient update method. Generally, the encoder and decoder are pre-trained with large sample data. However, the relation extraction layer is difficult to pre-train with large sample data; therefore, a smaller weight is assigned to the first loss value, and a larger weight is assigned to the second loss value.

[0178] It should be noted that the formula for calculating the first loss value is shown in Formula 6:

[0179]

[0180] Where logPθ() is the formula for calculating the self-loss of the text generation model, yt is the character probability distribution of the sample sentence in the t-th iteration, and H is the sentence feature corresponding to the target text information of the sample.

[0181] The formula for calculating the second loss value is shown in Formula 7:

[0182]

[0183] Where N represents the set of non-empty labels in the sample statement relation labels, and P represents the set of empty labels in the sample statement relation labels. The sample statement relationship features between the i-th and j-th sentences are... Sample statement relation labels are the sample statement relation features of the i-th and j-th sentences.

[0184] For example, a sample dataset is obtained from an open-source sample database. This dataset contains 10,000 sample statements, each labeled with a sample argumentative relationship. The sample statements include 400 main sample statements and 99,600 corresponding initial sample statements. After batch processing, the main sample statements and initial sample statements are input into the encoder of the T5 model to obtain a sample statement feature set. This feature set is then input into a relation extraction layer to extract the sample argumentative relationships between the sample statements, constructing sample argumentative relationship features. Finally, the sample statement feature set and the sample argumentative relationship features are input into the decoder of the T5 model. The target text information and sample information type of the target text information in the initial sentences of 99,600 samples are decoded. Using formula 6, the first loss value L1 is calculated based on the target text information and sample sentences. Using formula 7, the second loss value L2 is calculated based on the sample information type and sample argument relationship label. The first loss value and the second loss value are weighted and calculated to obtain the third loss value L3 = ω1L1 + ω2L2. Based on the third loss value, the model parameters of the encoder, relation extraction layer and decoder are adjusted using the gradient update method. Under the condition of satisfying the preset loss value threshold, the trained information extraction model is obtained.

[0185] In the embodiments of this specification, the relationship between sample topic statements and initial statements of each sample is fully explored, and sample statement relationship features corresponding to sample statement features are constructed. Based on the sample statement relationship features and the sample statement feature set decoded to obtain sample target text information and sample information type, the logical relationship between sample statements containing richer semantic information is used for model training, which improves the accuracy of the model. On the basis of the text generation model, the sample statement relationship between sample statements is adaptively determined for model training, rather than based on a pre-set table template for content filling. This more flexibly captures the structural logical relationship between each sample statement, improving the generalization ability of the model. Based on the first loss value of the information itself and the second loss value of the information type, the model is adjusted to improve the accuracy of the model.

[0186] In one optional embodiment of this specification, the sample statement relational tags include empty tags and non-empty tags;

[0187] Correspondingly, based on the sample information type and the sample statement relationship label, the second loss value is calculated, including the following specific steps:

[0188] Based on the number of non-empty tags, empty tags are sampled to obtain the target empty tag;

[0189] The second loss value is calculated based on the sample information type, non-empty label, and target empty label.

[0190] Generally, the number of empty labels in the sample statement relation labels is much greater than the number of non-empty labels. In order to meet the sample balance requirements for model training and ensure the training effect of the model, it is necessary to sample the empty labels. The specific sampling method is to sample based on the number of non-empty labels, for example, based on a preset balanced distribution of the number, or based on the same or proportional number of labels.

[0191] Corresponding to Formula 7, where N represents the set of empty labels obtained after sampling.

[0192] In the embodiments of this specification, empty labels are sampled based on the number of non-empty labels to obtain target empty labels, and the sample balance of model training is used to calculate the second loss value, which ensures the training effect of the model and improves the model performance of the information extraction model.

[0193] See Figure 2 , Figure 2 A flowchart of another information extraction method according to an embodiment of this specification is shown. This method is applied to a cloud-side device and includes the following specific steps:

[0194] Step 202: Receive the document to be extracted and the topic statement sent by the receiving end device, wherein the document to be extracted contains multiple initial statements.

[0195] Step 204: Input the topic statement and multiple initial statements into the encoder of the information extraction model to encode the statement feature set. The information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0196] Step 206: Input the statement feature set into the relation extraction layer, extract the statement relations between each statement, and construct the statement relation features, where the statement relations are the logical relations between statements.

[0197] Step 208: Input the statement feature set and statement relationship features into the decoder to decode and obtain the target information in multiple initial statements.

[0198] Step 210: Feed back the target information to the end device.

[0199] Cloud-side devices are network cloud devices that provide information retrieval functions; they are virtual devices. End-side devices are terminal devices located on platforms such as applications, web pages, or mini-programs that also have information retrieval functions; they are physical devices. Cloud-side devices and end-side devices are connected via network transmission channels for data transmission. Cloud-side devices have higher computing power than end-side devices.

[0200] It should be noted that steps 202 to 208 have already been described above. Figure 1 The examples are described in detail and will not be repeated here.

[0201] In this embodiment, a document to be extracted and topic sentences are obtained, wherein the document to be extracted contains multiple initial sentences; the topic sentences and multiple initial sentences are input into the encoder of the information extraction model to encode and obtain a sentence feature set, wherein the information extraction model is a pre-trained text generation model, and the information extraction model also includes a relation extraction layer and a decoder; the sentence feature set is input into the relation extraction layer to extract the sentence relationships between each sentence and construct sentence relationship features, wherein the sentence relationship is the logical relationship between sentences; the sentence feature set and sentence relationship features are input into the decoder to decode and obtain the target information in the multiple initial sentences. This approach fully explores the relationship between the main sentences and the initial sentences in the document to be extracted, constructing sentence relationship features corresponding to the sentence features. Based on the target information decoded from the sentence relationship features and sentence feature set, information extraction is performed using the logical relationship between sentences, which contains richer semantic information. This improves the accuracy of information extraction. On the basis of the text generation model, the approach adaptively determines the relationship between sentences for information extraction, rather than filling in content based on a pre-set table template. This more flexibly captures the structural logical relationship between sentences in the document to be extracted, improving the generalization ability of the model and the generalization ability of information extraction. At the same time, information extraction is implemented on cloud-side devices with higher computing power, improving the efficiency of information extraction and reducing the computing power cost of edge devices.

[0202] See Figure 3 , Figure 3 A flowchart of a method for extracting argument information according to an embodiment of this specification is shown, including the following specific steps:

[0203] Step 302: Obtain the document to be extracted and the topic statements, wherein the document to be extracted contains multiple initial statements.

[0204] Step 304: Input the topic statement and multiple initial statements into the encoder of the argument information extraction model to encode the statement feature set. The argument information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0205] Step 306: Input the statement feature set into the relation extraction layer, extract the argumentative relations between each statement, and construct the argumentative relation features, where the argumentative relation is the argumentative logical relationship between statements.

[0206] Step 308: Input the statement feature set and argument relationship features into the decoder to decode and obtain the target argument information in multiple initial statements.

[0207] The embodiments in this specification apply to the clients and / or servers of applications, web pages, or mini-programs that have argumentation information extraction capabilities. Examples include a paper generation application or web page, a training sample generation application, and an intelligent reply application.

[0208] The document to be extracted is a document containing argumentative text information corresponding to the topic statements.

[0209] The argument information extraction model is a neural network model with argument information extraction capabilities. This model is a text generation model, including but not limited to: Transformer, BERT, and T5 models. The argument information extraction model includes an encoder, a relation extraction layer, and a decoder. Information extraction is achieved by performing feature encoding, feature transformation, and feature decoding on the text information input to the model. The argument information extraction model is a text generation model with feature enhancement capabilities.

[0210] The relation extraction layer is a feature transformation layer that extracts relationships between statements. In this layer, the argumentative relationships between corresponding statements are determined using at least two statement features. The relation extraction layer achieves feature enhancement by extracting features not present in the statement features themselves, namely, the feature relationships between statement features. These features guide feature decoding in step 308, improving the accuracy of the obtained target argumentative information.

[0211] Statement relationships refer to the logical arguments between statements, including but not limited to arguments, evidence, stances, and types of evidence. Argument relationship features are the feature encoding vectors corresponding to these relationships.

[0212] The target argument information is the relevant argument text information corresponding to the topic statement in the initial statement. It can be the initial statement itself or at least one word in the initial statement. The target argument information can be at the word level or the statement level. The target argument information includes argument text information and information types of argument text information.

[0213] This can be understood as the embodiments of this specification being similar to... Figure 1 The main difference in the embodiments of the specification is that the relation extraction layer in this embodiment is set up for the specific statement relation of argumentation, and therefore has the feature enhancement capability to extract argumentation information.

[0214] It should be noted that the embodiments in this specification are different from those in the previous one. Figure 1 The embodiments described in this specification are embodiments based on the same inventive concept. These embodiments are adaptive embodiments in the application scenario of extracting argumentative information. Therefore, steps 302 to 308 are referred to above. Figure 1 The detailed descriptions of steps 102 to 108 in the embodiments will not be repeated here.

[0215] In this embodiment, the document to be extracted and the topic statements are obtained, wherein the document to be extracted contains multiple initial statements; the topic statements and multiple initial statements are input into the encoder of the argument information extraction model to encode and obtain a set of statement features, wherein the argument information extraction model is a pre-trained text generation model, and the argument information extraction model also includes a relation extraction layer and a decoder; the set of statement features is input into the relation extraction layer to extract the argument relationships between the statements and construct argument relationship features, wherein the argument relationship is the argument logic relationship between the statements; the set of statement features and the argument relationship features are input into the decoder to decode and obtain the target argument information in the multiple initial statements. This method fully explores the argumentative relationships between the main statements and the initial statements in the document to be extracted, constructing argumentative relationship features corresponding to the statement features. Based on the argumentative relationship features and the target argumentative information decoded from the set of statement features, the method utilizes the argumentative relationship—a logical relationship between statements containing richer semantic information—to extract argumentative information, improving the accuracy of argumentative information extraction. On the basis of the text generation model, the method adaptively determines the argumentative relationships between statements for argumentative information extraction, rather than filling in content based on a pre-set table template. This more flexibly captures the structural argumentative logical relationships between statements in the document to be extracted, improving the model's generalization ability and enhancing the generalization ability of argumentative information extraction.

[0216] See Figure 4 , Figure 4 A flowchart of a data processing method for information extraction according to an embodiment of this specification is shown. The method is applied to a cloud-side device and includes the following specific steps:

[0217] Step 402: Obtain the sample dataset, which includes multiple sample statements, each labeled with a sample statement relationship tag. The sample statements include the sample topic statement and the sample initial statement corresponding to the sample topic statement.

[0218] Step 404: Input the sample topic statement and the sample initial statement into the encoder of the information extraction model to encode the sample statement feature set. The information extraction model also includes a relation extraction layer and a decoder.

[0219] Step 406: Input the sample statement feature set into the relationship extraction layer, extract the sample statement relationship between each sample statement, and construct the sample statement relationship feature, where the sample statement relationship is the logical relationship between sample statements.

[0220] Step 408: Input the sample statement feature set and sample statement relationship features into the decoder to decode and obtain the sample target text information and sample information type of the sample target text information in multiple sample initial statements.

[0221] Step 410: Calculate the first loss value based on the sample target text information and sample sentences, and calculate the second loss value based on the sample information type and sample sentence relationship label.

[0222] Step 412: Based on the first loss value and the second loss value, adjust the model parameters of the encoder, relation extraction layer and decoder, and obtain the trained information extraction model when the preset training termination condition is met.

[0223] Step 414: Send the model parameters of the information extraction model to the end device.

[0224] The embodiments in this specification apply to cloud-side devices with model training capabilities. A cloud-side device is a network cloud device that provides model training functionality; it is a virtual device. An edge device is a terminal device that provides information extraction functionality; it is a physical device. The edge device and the cloud-side device are connected via a network channel for data transmission. The computing power of the cloud-side device is higher than that of the edge device.

[0225] It should be noted that steps 402 to 414 have already been described above. Figure 1 The examples are described in detail and will not be repeated here.

[0226] In this embodiment, a sample dataset is acquired, comprising multiple sample statements labeled with sample statement relationship tags. Each sample statement includes a sample topic statement and its corresponding initial sample statement. The sample topic statement and the initial sample statement are input into the encoder of an information extraction model to encode a sample statement feature set. The information extraction model further includes a relationship extraction layer and a decoder. The sample statement feature set is input into the relationship extraction layer to extract the sample statement relationships between each sample statement, constructing sample statement relationship features, where the sample statement relationship is the logical relationship between sample statements. The sample statement feature set and the sample statement relationship features are input into the decoder to decode the sample target text information and the sample information type of the sample target text information in multiple initial sample statements. Based on the sample target text information and sample statements, a first loss value is calculated, and based on the sample information type and sample statement relationship tags, a second loss value is calculated. Based on the first and second loss values, the model parameters of the encoder, relationship extraction layer, and decoder are adjusted. Under the condition of satisfying a preset training termination condition, a trained information extraction model is obtained. The model parameters of the information extraction model are sent to the edge device. The model fully explores the relationship between sample topic sentences and initial sentences of each sample, constructing sample sentence relationship features corresponding to the sample sentence features. Based on the sample sentence relationship features and the sample sentence feature set, the model decodes the sample target text information and sample information type. It utilizes the logical relationship between sample sentences, which contains richer semantic information, for model training, improving the model's accuracy. On the basis of the text generation model, it adaptively determines the sample sentence relationship between sample sentences for model training, rather than filling in content based on a pre-set table template. This more flexibly captures the structural logical relationship between each sample sentence, improving the model's generalization ability. Based on the first loss value of the information itself and the second loss value of the information type, the model is adjusted to improve the model's accuracy. At the same time, the model training is implemented on cloud-side devices with higher computing power, improving the efficiency of model training and reducing the computing power cost of edge devices.

[0227] Figure 5 A schematic diagram illustrating data processing in an information extraction method according to an embodiment of this specification is shown. For example... Figure 3 As shown:

[0228] The embedded sequence of statements (start delimiter, #0, topic statement, end delimiter...start delimiter, #7, initial statement, end delimiter) is input into the encoding layer of the T5 model to obtain a set of statement features (h1, h2...h7). This set of statement features is then input into the first and second feature transformation layers, followed by a feature classification layer to determine the relational labels for each statement: "+" for supporting the position, "-" for opposing the position, "investigation" for investigation-type argument type, "example" for example-type argument type, "expert" for expert-type argument type, and "0" for an empty label. This constructs the statement relational features. The statement relational features can be abstracted into a 7x8 table. The cells in the first column are used to solve the statement position detection problem, and the remaining cells in the table will be used to solve the joint labeling task. Statement 3 is an argument statement supporting the topic statement, and it is supported by two supporting statements (statements 1 and 2), both of which belong to the survey type. Statement 6 is an argument statement opposing the topic statement, and it is supported by two supporting statements (statements 4 and 7). Statement 4 belongs to the example type, and statement 7 belongs to the expert type. For a non-argument statement, all cells in that row are labeled with an empty label. The statement relationship features and statement feature set are input into the decoder of the T5 model to decode the target information: (#6, opposing argument statement, :, #4, example type supporting statement, \, #7, expert type supporting statement, terminating separator) and (#3, supporting argument statement, :, #1, survey type supporting statement, \, #2, survey type supporting statement, terminating separator).

[0229] Figure 6 This diagram illustrates a front-end interaction of an information extraction method according to an embodiment of this specification. Figure 4 As shown:

[0230] The front-end interactive interface includes a prompt box for "Please enter a topic statement," an input box for the topic statement, and a selection control for "Search and extract information." After the user enters a topic statement and selects the selection control for "Search and extract information," the front-end interactive interface displays "The document currently found to be extracted is: XXX" and the target information in the document to be extracted XXX: "Supporting argument statements: Survey type argument statement 1; Survey type argument statement 2" and "Opposing argument statements: Example type argument statement 3; Expert type argument statement 4."

[0231] The following is in conjunction with the appendix Figure 7 Taking the application of the information extraction method provided in this specification in argumentative essay writing as an example, the information extraction method will be further explained. Figure 7 The present specification illustrates a process flowchart of an information extraction method for argumentative essay writing according to an embodiment of this specification, including the following specific steps:

[0232] Step 702: Receive the data document to be extracted and the thesis topic statements uploaded by the user. The data document to be extracted includes multiple data statements.

[0233] Step 704: Input the paper's main statement and multiple supporting statements into the embedding layer of the information extraction model, and concatenate the paper's main statement and multiple supporting statements to obtain a statement embedding sequence. The information extraction model is a pre-trained text generation model. The argument information extraction model also includes an attention feature encoding layer, an argument feature transformation layer, an argument feature transformation layer, a feature classification layer, and an attention feature decoder.

[0234] Step 706: Input the sentence embedding sequence into the attention feature encoding layer, perform attention feature encoding on the sentence embedding sequence, and obtain the sentence feature set.

[0235] Step 708: Input the statement features of each data statement into the argument feature transformation layer and the evidence feature transformation layer respectively to obtain the argument statement transformation features and the evidence statement expression transformation features.

[0236] Step 710: Input the argument statement transformation features and the evidence statement transformation features into the feature classification layer to obtain the argumentative relationship between the statements in each document.

[0237] Step 712: Based on the argumentative relationship between the statements in each document, determine the corresponding argumentative relationship label, and annotate each document statement based on the argumentative relationship label to construct the argumentative relationship feature.

[0238] Step 714: Input the statement feature set and argument relationship features into the attention feature decoder, perform attention decoding on the statement feature set and argument relationship features, and obtain the target data information in multiple data statements.

[0239] Step 716: Based on the information type of the target data text information, group the target data text information to obtain multiple target data information groups.

[0240] Step 718: Feed back multiple target data information groups to the user.

[0241] In this embodiment, the argumentative relationships between the main statements of the paper and the statements in the document to be extracted are fully explored. Argumentative relationship features corresponding to the statement features are constructed. Based on the argumentative relationship features and the target data information decoded from the statement feature set, the argumentative relationship, which contains richer semantic information, is used to extract the target data information, thereby improving the accuracy of target data information extraction. On the basis of the text generation model, the argumentative relationship between statements is adaptively determined to extract target data information, rather than based on a pre-set table template for content filling. This more flexibly captures the structural argumentative relationship between the statements in the document to be extracted, improving the generalization ability of the model and the generalization ability of target data information extraction.

[0242] Corresponding to the above method embodiments, this specification also provides embodiments of information extraction devices. Figure 8 A schematic diagram of an information extraction device according to one embodiment of this specification is shown. Figure 8 As shown, the device includes:

[0243] The first acquisition module 802 is configured to acquire the document to be extracted and the topic statements, wherein the document to be extracted contains multiple initial statements;

[0244] The first encoding module 804 is configured to input the topic statement and multiple initial statements into the encoder of the information extraction model to encode a set of statement features. The information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0245] The first extraction and construction module 806 is configured to input the statement feature set into the relation extraction layer, extract the statement relations between each statement, and construct the statement relation features, wherein the statement relations are the logical relations between statements;

[0246] The first decoding module 808 is configured to input the statement feature set and statement relationship features into the decoder to decode and obtain target information from multiple initial statements.

[0247] Optionally, the relation extraction layer includes a first feature transformation layer, a second feature transformation layer, and a feature classification layer;

[0248] Correspondingly, the first extraction and construction module 806 is further configured to: input the first statement feature of the first statement into the first feature transformation layer to obtain the first statement transformation feature; input the second statement feature of the second statement into the second feature transformation layer to obtain the second statement transformation feature, wherein the first statement and the second statement are any two statements among the topic statement and multiple initial statements; input the first statement transformation feature and the second statement transformation feature into the feature classification layer to obtain the statement relationship between the first statement and the second statement; and construct statement relationship features based on the statement relationship between each statement.

[0249] Optionally, the first extraction and construction module 806 is further configured to: determine the corresponding type of statement relationship label based on the statement relationship between each statement; and annotate each statement based on the statement relationship label to construct statement relationship features.

[0250] Optionally, the encoder includes an embedding layer and an encoding layer;

[0251] Correspondingly, the first encoding module 804 is further configured to: input the topic statement and multiple initial statements into the embedding layer, concatenate the topic statement and multiple initial statements to obtain a statement embedding sequence; input the statement embedding sequence into the encoding layer to encode a statement feature set.

[0252] Optionally, the encoding layer is an attention feature encoding layer;

[0253] Correspondingly, the first encoding module 804 is further configured to: input the statement embedding sequence into the attention feature encoding layer, perform attention feature encoding on the statement embedding sequence, and obtain a statement feature set.

[0254] Optionally, the decoder includes an attention feature decoding layer;

[0255] Correspondingly, the first decoding module 808 is further configured to: input the statement feature set and statement relationship features into the attention feature decoder, perform attention decoding on the statement feature set and statement relationship features, and obtain target information in multiple initial statements.

[0256] Optionally, the target information includes target text information and the information type of the target text information;

[0257] Correspondingly, the device also includes a grouping module, configured to group the target text information based on the information type of the target text information to obtain at least one target information group.

[0258] Optionally, the device further includes: a training module configured to acquire a sample dataset, wherein the sample dataset includes multiple sample sentences labeled with sample sentence relationship tags, and the sample sentences include sample topic sentences and sample initial sentences corresponding to the sample topic sentences; inputting the sample topic sentences and sample initial sentences into the encoder of the information extraction model to encode a set of sample sentence features, wherein the information extraction model further includes a relationship extraction layer and a decoder; inputting the set of sample sentence features into the relationship extraction layer to extract the sample sentence relationships between each sample sentence, constructing sample sentence relationship features, wherein the sample sentence relationships are logical relationships between sample sentences; inputting the set of sample sentence features and the sample sentence relationship features into the decoder to decode the sample target text information and the sample information type of the sample target text information in multiple sample initial sentences; calculating a first loss value based on the sample target text information and sample sentences, and calculating a second loss value based on the sample information type and sample sentence relationship tags; adjusting the model parameters of the encoder, relationship extraction layer and decoder based on the first loss value and the second loss value, and obtaining a trained information extraction model when the preset training termination condition is met.

[0259] Optionally, the training module is further configured to: sample empty labels based on the number of non-empty labels to obtain target empty labels; and calculate a second loss value based on the sample information type, non-empty labels, and target empty labels.

[0260] In the embodiments of this specification, the relationship between the topic statement and the initial statements in the document to be extracted is fully explored, and the statement relationship features corresponding to the statement features are constructed. Based on the target information obtained by decoding the statement relationship features and the statement feature set, the logical relationship between statements containing richer semantic information is used to extract information, which improves the accuracy of information extraction. On the basis of the text generation model, the relationship between statements is adaptively determined to extract information, rather than based on a pre-set table template for content filling. This more flexibly captures the structural logical relationship between the statements in the document to be extracted, improves the generalization ability of the model, and improves the generalization ability of information extraction.

[0261] Corresponding to the above method embodiments, this specification also provides embodiments of information extraction devices. Figure 9 A schematic diagram of another information extraction device provided in one embodiment of this specification is shown. Figure 9 As shown, this device is applied to cloud-side equipment, and the device includes:

[0262] The receiving module 902 is configured to receive the document to be extracted and the topic statement sent by the end-side device, wherein the document to be extracted contains multiple initial statements;

[0263] The second encoding module 904 is configured to input the topic statement and multiple initial statements into the encoder of the information extraction model to encode a set of statement features. The information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0264] The second extraction and construction module 906 is configured to input the statement feature set into the relationship extraction layer, extract the statement relationships between each statement, and construct statement relationship features, wherein the statement relationship is the logical relationship between statements;

[0265] The second decoding module 908 is configured to input the statement feature set and statement relationship features into the decoder to decode and obtain target information in multiple initial statements;

[0266] Feedback module 910 is configured to feed back target information to the end-side device.

[0267] In the embodiments of this specification, the relationship between the topic statement and the initial statements in the document to be extracted is fully explored, and the statement relationship features corresponding to the statement features are constructed. Based on the target information obtained by decoding the statement relationship features and the statement feature set, the logical relationship between statements containing richer semantic information is used to extract information, which improves the accuracy of information extraction. On the basis of the text generation model, the relationship between statements is adaptively determined to extract information, rather than filling in content based on a pre-set table template. This more flexibly captures the structural logical relationship between the statements in the document to be extracted, improves the generalization ability of the model, and improves the generalization ability of information extraction. At the same time, information extraction is implemented on cloud-side devices with higher computing power, which improves the efficiency of information extraction and reduces the computing power cost of edge devices.

[0268] The above is a schematic scheme of an information extraction device according to this embodiment. It should be noted that the technical solution of this information extraction device and the technical solution of the information extraction method described above belong to the same concept. For details not described in detail in the technical solution of the information extraction device, please refer to the description of the technical solution of the information extraction method described above.

[0269] Corresponding to the above method embodiments, this specification also provides embodiments of an argument information extraction device. Figure 10 A schematic diagram of an argument information extraction device according to one embodiment of this specification is shown. Figure 10 As shown, the device includes:

[0270] The second acquisition module 1002 is configured to acquire the document to be extracted and the topic statement, wherein the document to be extracted contains multiple initial statements;

[0271] The third encoding module 1004 is configured to input the topic statement and multiple initial statements into the encoder of the argument information extraction model to encode the statement feature set. The argument information extraction model is a pre-trained text generation model, which also includes a relation extraction layer and a decoder.

[0272] The third extraction and construction module 1006 is configured to input the set of statement features into the relation extraction layer, extract the argumentative relationship between each statement, and construct the argumentative relationship features, wherein the argumentative relationship is the argumentative logical relationship between statements;

[0273] The third decoding module 1008 is configured to input the statement feature set and the argument relationship feature into the decoder to decode and obtain the target argument information in multiple initial statements.

[0274] In this embodiment, the argumentative relationships between the topic statements and the initial statements in the document to be extracted are fully explored, and argumentative relationship features corresponding to the statement features are constructed. Based on the argumentative relationship features and the target argumentative information decoded from the statement feature set, the argumentative relationship, which contains richer semantic information, is used to extract the argumentative information, thereby improving the accuracy of argumentative information extraction. On the basis of the text generation model, the argumentative relationships between statements are adaptively determined to extract argumentative information, rather than filling in content based on a pre-set table template. This more flexibly captures the structural argumentative logical relationships between the statements in the document to be extracted, improving the generalization ability of the model and the generalization ability of argumentative information extraction.

[0275] The above is a schematic scheme of an argument information extraction device according to this embodiment. It should be noted that the technical solution of this argument information extraction device and the technical solution of the argument information extraction method described above belong to the same concept. For details not described in detail in the technical solution of the argument information extraction device, please refer to the description of the technical solution of the argument information extraction method described above.

[0276] Corresponding to the above method embodiments, this specification also provides embodiments of a data processing apparatus for information extraction. Figure 11 A schematic diagram of the structure of a data processing apparatus for information extraction according to one embodiment of this specification is shown. Figure 11 As shown, this device is applied to cloud-side equipment, and the device includes:

[0277] The third acquisition module 1102 is configured to acquire a sample dataset, wherein the sample dataset includes multiple sample statements, the multiple sample statements are labeled with sample statement relationship tags, and the sample statements include sample topic statements and sample initial statements corresponding to the sample topic statements.

[0278] The fourth encoding module 1104 is configured to input the sample topic statement and the sample initial statement into the encoder of the information extraction model and encode them to obtain the sample statement feature set. The information extraction model also includes a relation extraction layer and a decoder.

[0279] The fourth extraction and construction module 1106 is configured to input the sample statement feature set into the relationship extraction layer, extract the sample statement relationship between each sample statement, and construct the sample statement relationship feature, wherein the sample statement relationship is the logical relationship between sample statements;

[0280] The fourth decoding module 1108 is configured to input the sample statement feature set and sample statement relationship features into the decoder to decode and obtain the sample target text information and sample information type of the sample target text information in multiple sample initial statements;

[0281] The calculation module 1110 is configured to calculate a first loss value based on the sample target text information and sample sentences, and to calculate a second loss value based on the sample information type and sample sentence relationship label.

[0282] The adjustment module 1112 is configured to adjust the model parameters of the encoder, relation extraction layer and decoder based on the third loss value, so as to obtain the trained information extraction model when the preset training termination condition is met.

[0283] The sending module 1114 is configured to send the model parameters of the information extraction model to the end device.

[0284] In the embodiments of this specification, the relationship between sample topic sentences and initial sentences of each sample is fully explored, and sample sentence relationship features corresponding to sample sentence features are constructed. Based on the sample sentence relationship features and the sample sentence feature set decoded to obtain sample target text information and sample information type, the logical relationship between sample sentences containing richer semantic information is used for model training, which improves the accuracy of the model. On the basis of the text generation model, the sample sentence relationship between sample sentences is adaptively determined for model training, rather than based on a pre-set table template for content filling. This more flexibly captures the structural logical relationship between each sample sentence, improving the generalization ability of the model. Based on the first loss value of the information itself and the second loss value of the information type, the model is adjusted to improve the accuracy of the model. At the same time, the model training is implemented on cloud-side devices with higher computing power, which improves the efficiency of model training and reduces the computing power cost of edge devices.

[0285] The above is a schematic diagram of a data processing apparatus for information extraction according to this embodiment. It should be noted that the technical solution of this data processing apparatus for information extraction belongs to the same concept as the technical solution of the data processing method for information extraction described above. Details not described in detail in the technical solution of the data processing apparatus for information extraction can be found in the description of the technical solution of the data processing method for information extraction described above.

[0286] Figure 12 A structural block diagram of a computing device according to one embodiment of this specification is shown. The components of the computing device 1200 include, but are not limited to, a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and a database 1250 is used to store data.

[0287] The computing device 1200 also includes an access device 1240, which enables the computing device 1200 to communicate via one or more networks 1260. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1240 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 1202.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0288] In one embodiment of this specification, the aforementioned components of the computing device 1200 and Figure 12 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 12 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0289] The computing device 1200 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 1200 can also be a mobile or stationary server.

[0290] The processor 1220 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the aforementioned information extraction method, argument information extraction method, or information extraction data processing method.

[0291] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solutions of the above-described information extraction method, argument information extraction method, and information extraction data processing method. For details not described in detail in the technical solution of the computing device, please refer to the descriptions of the technical solutions of the above-described information extraction method, argument information extraction method, or information extraction data processing method.

[0292] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described information extraction method, argument information extraction method, or information extraction data processing method.

[0293] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the above-described information extraction method, argument information extraction method, and information extraction data processing method. For details not described in detail in the technical solution of the storage medium, please refer to the descriptions of the technical solutions of the above-described information extraction method, argument information extraction method, or information extraction data processing method.

[0294] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described information extraction method, argument information extraction method, or information extraction data processing method.

[0295] The above is an illustrative scheme of a computer program according to this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solutions of the above-described information extraction method, argument information extraction method, and information extraction data processing method. For details not described in detail in the technical solution of the computer program, please refer to the descriptions of the technical solutions of the above-described information extraction method, argument information extraction method, or information extraction data processing method.

[0296] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0297] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0298] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0299] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0300] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.< / se> < / ss> < / se> < / ss>

Claims

1. An information extraction method, comprising: Obtain the document to be extracted and the topic statements, wherein the document to be extracted contains multiple initial statements; The topic statement and the plurality of initial statements are input into the encoder of the information extraction model to encode and obtain a set of statement features. The information extraction model is a pre-trained text generation model, and the information extraction model also includes a relation extraction layer and a decoder. The statement feature set is input into the relation extraction layer to extract the statement relationships between statements and construct statement relationship features, wherein the statement relationship is the logical relationship between statements; The statement feature set and the statement relationship features are input into the decoder to decode and obtain the target information in the plurality of initial statements.

2. The method according to claim 1, wherein, The relation extraction layer includes a first feature transformation layer, a second feature transformation layer, and a feature classification layer; The step of inputting the set of statement features into the relation extraction layer, extracting the statement relationships between statements, and constructing statement relationship features includes: The first statement feature of the first statement is input into the first feature transformation layer to obtain the first statement transformation feature, and the second statement feature of the second statement is input into the second feature transformation layer to obtain the second statement transformation feature, wherein the first statement and the second statement are any two statements among the topic statement and the plurality of initial statements; The first statement transformation feature and the second statement transformation feature are input into the feature classification layer to obtain the statement relationship between the first statement and the second statement. Based on the relationships between statements, statement relationship features are constructed.

3. The method according to claim 2, wherein constructing statement relationship features based on the statement relationships between statements includes: Based on the relationships between statements, determine the corresponding type of statement relationship label; Each statement is labeled based on the statement relationship tags to construct statement relationship features.

4. The method according to claim 1, wherein, The encoder includes an embedding layer and an encoding layer; The encoder of the model that extracts the topic statement and the multiple initial statement input information encodes the statement feature set, including: The topic statement and the plurality of initial statements are input into the embedding layer, and the topic statement and the plurality of initial statements are concatenated to obtain a statement embedding sequence; The embedded sequence of the statement is input into the encoding layer to obtain a set of statement features.

5. The method according to claim 4, wherein, The encoding layer is an attention feature encoding layer; The step of inputting the embedded sequence of the statement into the encoding layer to encode a set of statement features includes: The embedded sequence of statements is input into the attention feature encoding layer, and attention feature encoding is performed on the embedded sequence of statements to obtain a set of statement features.

6. The method according to claim 4 or 5, wherein, The decoder includes an attention feature decoding layer; The step of inputting the statement feature set and the statement relationship features into the decoder to decode and obtain the target information in the plurality of initial statements includes: The statement feature set and the statement relationship features are input into the attention feature decoding layer, and attention decoding is performed on the statement feature set and the statement relationship features to obtain the target information in the multiple initial statements.

7. The method according to claim 1, wherein, The target information includes target text information and the information type of the target text information; After obtaining the target information from the plurality of initial statements, the method further includes: Based on the information type of the target text information, the target text information is grouped to obtain at least one target information group.

8. The method according to claim 1, wherein the information extraction model is trained through the following steps: Obtain the sample dataset, where, The sample dataset includes multiple sample statements, each sample statement is labeled with a sample statement relationship tag, and the sample statements include sample topic statements and multiple sample initial statements corresponding to the sample topic statements. The sample topic statement and the multiple sample initial statements are input into the encoder of the information extraction model to encode and obtain the sample statement feature set. The information extraction model also includes a relation extraction layer and a decoder. The sample statement feature set is input into the relation extraction layer to extract the sample statement relationships between each sample statement and construct the sample statement relationship features, wherein the sample statement relationship is the logical relationship between sample statements; The sample statement feature set and the sample statement relationship features are input into the decoder to decode and obtain the sample target text information and the sample information type of the sample target text information in the multiple sample initial statements. Based on the target text information of the sample and the sample statement, a first loss value is calculated, and based on the sample information type and the relationship label of the sample statement, a second loss value is calculated. Based on the first loss value and the second loss value, the model parameters of the encoder, the relation extraction layer and the decoder are adjusted, and the trained information extraction model is obtained when the preset training termination condition is met.

9. The method according to claim 8, wherein, The sample statement relational tags include empty tags and non-empty tags; The calculation of the second loss value based on the sample information type and the sample statement relationship label includes: Based on the number of non-empty tags, the empty tags are sampled to obtain the target empty tags; Based on the sample information type, the non-empty label, and the target empty label, a second loss value is calculated.

10. An information extraction method, applied to cloud-side devices, comprising: The receiving end device sends the document to be extracted and the topic statement, wherein the document to be extracted contains multiple initial statements; The topic statement and the plurality of initial statements are input into the encoder of the information extraction model to encode and obtain a set of statement features. The information extraction model is a pre-trained text generation model, and the information extraction model also includes a relation extraction layer and a decoder. The statement feature set is input into the relation extraction layer to extract the statement relationships between statements and construct statement relationship features, wherein the statement relationship is the logical relationship between statements; The statement feature set and the statement relationship features are input into the decoder to decode and obtain the target information in the plurality of initial statements; The target information is fed back to the end-side device.

11. A method for extracting argumentative information, comprising: Obtain the document to be extracted and the topic statements, wherein the document to be extracted contains multiple initial statements; The topic statement and the plurality of initial statements are input into the encoder of the argument information extraction model to encode and obtain a set of statement features. The argument information extraction model is a pre-trained text generation model, and the argument information extraction model also includes a relation extraction layer and a decoder. The set of statement features is input into the relation extraction layer to extract the argumentative relationships between statements and construct the argumentative relationship features, wherein the argumentative relationship is the argumentative logical relationship between statements; The statement feature set and the argument relationship feature are input into the decoder to decode and obtain the target argument information in the multiple initial statements.

12. A data processing method for information extraction, applied to cloud-side devices, comprising: Obtain a sample dataset, wherein the sample dataset includes multiple sample statements, the multiple sample statements are labeled with sample statement relationship tags, and the sample statements include sample topic statements and multiple sample initial statements corresponding to the sample topic statements; The sample topic statement and the multiple sample initial statements are input into the encoder of the information extraction model to encode and obtain the sample statement feature set. The information extraction model also includes a relation extraction layer and a decoder. The sample statement feature set is input into the relation extraction layer to extract the sample statement relationships between each sample statement and construct the sample statement relationship features, wherein the sample statement relationship is the logical relationship between sample statements; The sample statement feature set and the sample statement relationship features are input into the decoder to decode and obtain the sample target text information and the sample information type of the sample target text information in the multiple sample initial statements. Based on the target text information of the sample and the sample statement, a first loss value is calculated, and based on the sample information type and the relationship label of the sample statement, a second loss value is calculated. Based on the first loss value and the second loss value, the model parameters of the encoder, the relation extraction layer and the decoder are adjusted, and the trained information extraction model is obtained when the preset training termination condition is met. The model parameters of the information extraction model are sent to the end device.

13. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 12.

14. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

Images