Information extraction method, method and device for training information extraction model

Abstract

Embodiments of the present application disclose an information extraction method, a method and device for training an information extraction model. The main technical solutions include: obtaining a text to be extracted; inputting a first text sequence obtained by splicing target type information and the text to be extracted into an information extraction model, obtaining a first extraction result output by the information extraction model, the first extraction result including a target corresponding to the target type information extracted from the text to be extracted; inputting a second text sequence obtained by splicing the first text sequence and the first extraction result into the information extraction model, obtaining a second extraction result output by the information extraction model, the second extraction result including a target corresponding to the target type information extracted from the text to be extracted; and the information extraction model is pre-trained based on a pre-trained language model. The accuracy of information extraction can be improved by the present application.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an information extraction method, a method and apparatus for training an information extraction model. Background Technology

[0002] Information extraction (IE) refers to extracting specific types of targets from text, thereby automatically classifying, extracting, and reconstructing massive amounts of content. Examples include named entity recognition, entity relation extraction, event extraction, and sentiment extraction. Most existing information extraction techniques train models for specific types of targets, resulting in poor transferability. To address this issue, a unified information extraction technique has been proposed, which uses a single model to solve extraction tasks for various information types.

[0003] Existing unified information extraction schemes all adopt a one-step generation method, but this method has poor target accuracy, especially when extracting complex types of targets. Summary of the Invention

[0004] In view of this, this application provides an information extraction method, a method and apparatus for training an information extraction model, in order to improve the accuracy of information extraction.

[0005] This application provides the following solution:

[0006] Firstly, a method for information extraction is provided, the method comprising:

[0007] Get the text to be extracted;

[0008] The first text sequence obtained by concatenating the target type information and the text to be extracted is input into the information extraction model, and the first extraction result output by the information extraction model is obtained. The first extraction result includes the target corresponding to the target type information extracted from the text to be extracted.

[0009] The second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes the target corresponding to the target type information extracted from the text to be extracted.

[0010] The information extraction model mentioned above is pre-trained based on a pre-trained language model.

[0011] According to one achievable method in an embodiment of this application, the target type information includes information on multiple target types.

[0012] According to one achievable method in an embodiment of this application, the information extraction model includes: an encoding network and a decoding network;

[0013] The encoding network is used to encode the features of each element Token in the input text sequence to obtain the feature representation of each Token. A preset separator is inserted in the input text sequence at the concatenation position.

[0014] The decoding network is used to perform decoding processing using the feature representation of each token to obtain an output text sequence. The output text sequence includes the target extracted from the text to be extracted and its corresponding target type identifier.

[0015] Secondly, an entity extraction method is provided, the method comprising:

[0016] The first text sequence obtained by concatenating entity type information and the text to be extracted is input into the information extraction model, and the first extraction result output by the information extraction model is obtained. The first extraction result includes entity words corresponding to the entity type information extracted from the text to be extracted.

[0017] The second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes entity words corresponding to the entity type information extracted from the text to be extracted.

[0018] The information extraction model mentioned above is pre-trained based on a pre-trained language model.

[0019] Thirdly, a method for extracting entity relations is provided, the method comprising:

[0020] The first text sequence obtained by concatenating triple information and the text to be extracted is input into the information extraction model, and the first extraction result output by the information extraction model is obtained. The first extraction result includes entity words and entity relation words extracted from the text to be extracted that correspond to the triple information.

[0021] The second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes entity words and entity relation words extracted from the text to be extracted that correspond to the triple information.

[0022] The information extraction model mentioned above is pre-trained based on a pre-trained language model.

[0023] Fourthly, an emotion extraction method is provided, the method comprising:

[0024] The first text sequence obtained by concatenating the sentiment element type information and the text to be extracted is input into the information extraction model, and the first extraction result output by the information extraction model is obtained. The first extraction result includes the sentiment element corresponding to the sentiment element type information extracted from the text to be extracted.

[0025] The second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes the emotional element corresponding to the emotional element type information extracted from the text to be extracted.

[0026] The information extraction model mentioned above is pre-trained based on a pre-trained language model.

[0027] Fifthly, a method for training an information extraction model is provided, the method comprising:

[0028] Acquire training data containing multiple training samples, wherein the training samples include text samples and extraction result samples corresponding to the text samples, and the extraction result samples include targets corresponding to preset target type information in the text samples;

[0029] An information extraction model based on a pre-trained language model is trained using the training data. A first text sequence, obtained by concatenating target type information and text samples, is input into the information extraction model to obtain a first extraction result. The first extraction result includes the target corresponding to the target type information extracted from the text samples. A second text sequence, obtained by concatenating the first text sequence and the first extraction result, is input into the information extraction model to obtain a second extraction result. The second extraction result includes the target corresponding to the target type information extracted from the text samples. The training objective includes minimizing the difference between the second extraction result and the extraction result samples corresponding to the text samples.

[0030] According to one possible implementation in this application embodiment, the training data includes a support set and a query set;

[0031] Training an information extraction model based on a pre-trained language model using the training data includes:

[0032] Initialize the model parameters of the information extraction model;

[0033] Backup the currently obtained information extraction model to obtain a backup model;

[0034] The text samples are sampled from the support set and input into the backup model. The loss function value is determined based on the training objective. The gradient is calculated for the model parameters of the backup model using the loss function value, and the model parameters of the backup model are updated until the preset first training termination condition is met.

[0035] Text samples are sampled from the query set and input into the currently trained backup model. The loss function value is determined based on the training objective. The gradient of the model parameters of the currently obtained information extraction model is calculated using the loss function value, and the model parameters of the information extraction model are updated until the preset second training termination condition is met.

[0036] The process continues until the preset third training termination condition is met, then proceeds to the step of extracting the currently obtained information from the backup model.

[0037] According to one achievable method in an embodiment of this application, the loss function value is obtained based on the difference between the second extraction result and the extraction result sample corresponding to the text sample; or...

[0038] The loss function value is obtained based on a first loss function value and a second loss function value, wherein the first loss function value is obtained based on the difference between the first extraction result and the extraction result sample corresponding to the text sample, and the second loss function value is obtained based on the difference between the second extraction result and the extraction result sample corresponding to the text sample.

[0039] According to one achievable method in an embodiment of this application, the information extraction model includes: an encoding network and a decoding network;

[0040] The encoding network is used to encode the features of each token in the input text sequence to obtain the feature representation of each token. A preset separator is inserted in the input text sequence at the concatenation position.

[0041] The decoding network is used to perform decoding processing using the feature representations of each token to obtain a text sequence. The output text sequence includes the target extracted from the text sample and its corresponding target type identifier.

[0042] Sixthly, an information extraction device is provided, the device comprising:

[0043] The text acquisition unit is configured to acquire the text to be extracted.

[0044] The first extraction unit is configured to input a first text sequence obtained by concatenating target type information and text to be extracted into an information extraction model, and obtain a first extraction result output by the information extraction model. The first extraction result includes the target corresponding to the target type information extracted from the text to be extracted.

[0045] The second extraction unit is configured to input a second text sequence obtained by concatenating the first text sequence and the first extraction result into the information extraction model, and obtain a second extraction result output by the information extraction model. The second extraction result includes the target corresponding to the target type information extracted from the text to be extracted.

[0046] The information extraction model mentioned above is pre-trained based on a pre-trained language model.

[0047] In a seventh aspect, an apparatus for training an information extraction model is provided, the apparatus comprising:

[0048] The sample acquisition unit is configured to acquire training data containing multiple training samples, wherein the training samples include text samples and extraction result samples corresponding to the text samples, and the extraction result samples include targets corresponding to preset target type information in the text samples;

[0049] The model training unit is configured to train an information extraction model based on a pre-trained language model using the training data. A first text sequence, obtained by concatenating target type information and text samples, is input into the information extraction model to obtain a first extraction result, which includes the target corresponding to the target type information extracted from the text samples. A second text sequence, obtained by concatenating the first text sequence and the first extraction result, is input into the information extraction model to obtain a second extraction result, which includes the target corresponding to the target type information extracted from the text samples. The training objective includes minimizing the difference between the second extraction result and the extraction result sample corresponding to the text samples.

[0050] According to an eighth aspect, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any one of the first to fifth aspects.

[0051] According to the ninth aspect, an electronic device is provided, comprising:

[0052] One or more processors; and

[0053] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first to fifth aspects.

[0054] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0055] 1) This application provides a two-step information extraction method. First, an information extraction model is used to extract the target type information corresponding to the target from the text to be extracted as the first extraction result. Then, the first extraction result is used as a basic semantic reference to assist the information extraction model in the second step to extract the target type information corresponding to the target from the text to be extracted again, thereby improving the accuracy of information extraction. This method is particularly effective in cases of complex target types.

[0056] 2) This application provides a model training mode based on meta-learning, which combines an inner loop with an outer loop. This enables the model to learn quickly in downstream application scenarios. It can achieve good results on the query set after training only a few times on the support set, reducing the amount of training data required and improving model performance.

[0057] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a system architecture diagram applicable to the embodiments of this application;

[0060] Figure 2 A flowchart of the information extraction method provided in the embodiments of this application;

[0061] Figure 3 This is a schematic diagram of the structure of the information extraction model provided in the embodiments of this application;

[0062] Figure 4 A flowchart illustrating the method for extracting training information from a model provided in this application embodiment;

[0063] Figure 5 A schematic diagram illustrating a preferred method for training an information extraction model provided in an embodiment of this application;

[0064] Figure 6 A schematic diagram illustrating the principle of entity extraction provided in the embodiments of this application;

[0065] Figure 7 A schematic block diagram of an information extraction device provided in the embodiments of this application;

[0066] Figure 8 A schematic block diagram of a training information extraction model provided in an embodiment of this application;

[0067] Figure 9 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0068] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0069] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0070] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0071] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0072] To facilitate understanding of this application, a brief description of the system architecture to which this application applies will be given first. Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown, such as Figure 1 As shown, the system architecture includes a model training device that builds an information extraction model offline, and an information extraction device that extracts information from the text to be extracted online.

[0073] After acquiring training data, the model training device can use the method provided in the embodiments of this application to train the model and obtain an information extraction model.

[0074] The information extraction device utilizes an established information extraction model to extract information from the text to be extracted, obtaining targets of a specified information type. For example, entity recognition is performed on the text to obtain entity words of a specified entity type. Another example is entity triple extraction, yielding two entity words and entity relation words. Yet another example is sentiment extraction, obtaining sentiment elements of a specified sentiment element type. Details will be provided in subsequent embodiments.

[0075] The model training device and information extraction device can be configured as separate servers, or they can be configured on the same server or server group, or they can be configured on a separate or the same cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability. The model training device and prediction device can also be configured on computer terminals with strong computing capabilities.

[0076] It should be noted that, in addition to online information extraction, the aforementioned information extraction device can also perform information extraction offline, such as extracting information from batches of text to be extracted.

[0077] It should be understood that Figure 1 The number of model training devices, information extraction devices, and information extraction models shown in the diagram is merely illustrative. Depending on the implementation requirements, any number of model training devices, information extraction devices, and information extraction models can be included.

[0078] Figure 2 This is a flowchart of an information extraction method provided in an embodiment of this application. The method flow can be... Figure 1 The information extraction device in the system architecture shown is executed. For example... Figure 2 As shown, the method may include the following steps:

[0079] Step 202: Obtain the text to be extracted.

[0080] Step 204: Input the first text sequence obtained by concatenating the target type information and the text to be extracted into the information extraction model, and obtain the first extraction result output by the information extraction model. The first extraction result includes the target corresponding to the above-mentioned target type information extracted from the text to be extracted.

[0081] Step 206: Input the second text sequence obtained by concatenating the first text sequence and the first extraction result into the information extraction model, and obtain the second extraction result output by the information extraction model. The second extraction result includes the target corresponding to the above-mentioned target type information extracted from the text to be extracted.

[0082] As can be seen from the above process, this application provides a two-step information extraction method. First, the information extraction model extracts the target type information corresponding to the target from the text to be extracted as the first extraction result. Then, using the first extraction result as a basic semantic reference, the information extraction model is assisted in the second step to extract the target type information corresponding to the target from the text to be extracted again, thereby improving the accuracy of information extraction. This method is particularly effective in cases of complex target types.

[0083] It should be noted that the terms "first" and "second" used in this disclosure do not have any limitations on size, order, or quantity; they are merely used to distinguish between the two. For example, "first extraction result" and "second extraction result" are used to distinguish between two extraction results in terms of name. Similarly, "first text sequence" and "second text sequence" are used to distinguish between two text sequences in terms of name.

[0084] The following describes each step in the above method flow. First, step 204, "obtaining the text to be extracted," will be described in detail with reference to the embodiment.

[0085] The text to be extracted in the embodiments of this application can be text obtained from a user device. For example, if a user wants to extract information of a specific target type from the text, the user device can upload the text to the information extraction device on the server side. The information extraction device will then use the text as the text to be extracted and extract the target corresponding to the specific target type information from the text to be extracted using the method provided in the embodiments of this application.

[0086] The text to be extracted in this embodiment can also be text obtained from a database. For example, if it is necessary to extract information of a specific target type from a batch of text in a database, each text obtained from the database can be used as the text to be extracted, and the target corresponding to the specific target type information can be extracted from the text to be extracted using the method provided in this embodiment.

[0087] The text to be extracted can be sentences, paragraphs composed of sentences, documents, or even smaller units of text, such as phrases.

[0088] The following describes in detail step 204, namely, "inputting the first text sequence obtained by concatenating the target type information and the text to be extracted into the information extraction model, and obtaining the first extraction result output by the information extraction model," with reference to the embodiments.

[0089] The target type involved in the embodiments of this application can be specified by the user or a default target type can be used. It can be information of one target type or information of multiple target types.

[0090] Depending on the application scenario, different target information types can be used. For example, in entity recognition, the target information type can be one or more combinations of names such as person names, place names, organization names, work names, brand names, addresses, dates, etc. In entity triple recognition, the target information type can be entity triple information. In emotion recognition, the target information type can be one or more combinations of emotion expressers, emotion objects, emotion words, etc.

[0091] One feasible approach is to place the target type information before the text to be extracted, forming the first text sequence. For example, suppose we want to extract the target types of a person's name, place name, and work title from the sentence "Yue Fei, a native of Tangyin, Xiangzhou, whose masterpiece 'Man Jiang Hong' is a patriotic classic passed down through the ages." We can then concatenate the target type information with the text to be extracted, resulting in the following first text sequence:

[0092] "Personal name, place name, and work title; Yue Fei, a native of Tangyin, Xiangzhou, whose masterpiece, 'Man Jiang Hong,' is a patriotic classic that has been passed down through the ages."

[0093] In the first text sequence described above, target type information is separated by commas (,), and the target type information and the text to be extracted are separated by semicolons (;). Other symbols can also be used for separation. If the target type information is the default target type information, no symbols are required for separation. In the first text sequence described above, target type information is in the form of text such as "person's name," "place name," or "work title," but other forms can also be used, such as symbols.

[0094] In addition to placing the target type information before the text to be extracted, other orders can also be used, such as placing the target type information after the text to be extracted. The key is to ensure that the order used in the information extraction process is consistent with the order used in the model training process.

[0095] The information extraction model provided in this application is based on a pre-trained language model. Specifically, the information extraction model is a Seq2Seq (Sequence to Sequence) model. Its structure can be as follows: Figure 3 As shown, it includes an encoding network (Encoder) and a decoding network (Decoder).

[0096] The encoding network is used to encode the features of each token (element) in the input text sequence (which is the first text sequence in this case) to obtain the feature representation of each token.

[0097] The encoding network can first perform embedding processing on each token in the input text sequence. This embedding process can include at least word embedding and position embedding. Word embedding involves encoding each token into word vectors to obtain word vector representations. Position embedding involves encoding the position of each token in the text sequence to be predicted to obtain position representations.

[0098] The decoding network is used to perform decoding processing using the feature representations of each token, and the output is also a text sequence, which is used as the first extraction result. The output text sequence may include the target extracted from the text to be extracted and its corresponding target type identifier.

[0099] Continuing the previous example, suppose the first extraction result obtained from the first text sequence is: "Personal Name: Yue Fei; Place Name:; Work Title: Man Jiang Hong". Here, "Personal Name:", "Place Name:", and "Work Title:" are textual target type identifiers. Other forms of target type identifiers can also be used, for example, the first extraction result could be: <n>Yue Fei; <pn> ； <wt>"Man Jiang Hong", in which " <n> ”、" <pn>"and" <wt>"These are type identifiers for personal names, place names, and work titles, respectively."

[0100] As one possible approach, the decoding network can classify each token in the text to be identified based on the feature representation of each token in the target type information, that is, map each token in the text to be processed to the target type space to obtain the classification result of whether each token belongs to the specified target type, and then obtain the target corresponding to the target type information extracted from the text to be extracted as the first extraction result.

[0101] The information extraction model described in this application embodiment can be trained based on a pre-trained language model. It employs pre-trained language models such as T5 (TransferText-to-Text Transformer), BERT (Bidirectional Encoder Representation from Transformers), XLNet (an autoregressive model that uses permutations of language models to achieve bidirectional contextual information), and GPT (Generative Pre-Training) as its model framework, and then performs further training on top of these. BERT is a bidirectional pre-trained language model that uses a Transformer Encoder as its model structure, and it effectively utilizes contextual information for feature learning. XLNet is a model similar to BERT, a more generalized autoregressive pre-trained model. GPT uses a Transformer Decoder structure and retains only masked multi-head attention within the Transformer Decoder. T5 is still an encoder-decoder structure consisting of stacked Transformer layers, but the Decoder has a standard attention layer after the self-attention layer. This standard attention layer will take the encoder's output into the attention calculation. The last layer of the Decoder outputs the probability of each token belonging to the target type through the Softmax classifier.

[0102] Each token in a text sequence refers to an element that constitutes the text sequence. For a text sequence, if the text sequence is divided into character or word sequences, then the characters or words in the text sequence, as well as the start character and the separator, are all tokens.

[0103] Since the pre-trained language model is implemented based on the Transformer network, the self-attention mechanism determines the feature representation of each token in the text to be extracted by referring to the influence of each token in the entire first text sequence, that is, it fully considers the semantic association of all the content in the first text sequence.

[0104] Following the traditional one-step information extraction approach, the first extraction result is directly used as the final extraction result. However, the first extraction result obtained in this way is often inaccurate, and problems such as errors in the extracted target or failure to extract some targets may occur. Therefore, this embodiment of the application continues to execute step 206. The following describes in detail step 206, namely, "inputting the second text sequence obtained by concatenating the first text sequence and the first extraction result into the information extraction model, and obtaining the second extraction result output by the information extraction model," with reference to the embodiment.

[0105] One feasible approach is to place the first extraction result after the first text sequence to form the second text sequence. Continuing the previous example, suppose the first extraction result obtained from the first text sequence through step 204 is: "Personal name: Yue Fei; Place name:; Work title: Man Jiang Hong", where the place name recognition result is empty. The second text sequence obtained by concatenating the first text sequence and the first extraction result can be as follows:

[0106] "Personal name, place name, work title; Yue Fei, a native of Tangyin, Xiangzhou, whose representative work, 'Man Jiang Hong,' is a patriotic masterpiece that has been passed down through the ages; Personal name: Yue Fei; Place name: ; Work title: 'Man Jiang Hong'"

[0107] In the second text sequence mentioned above, the first text sequence and the first extraction result are separated by ";", but other symbols can also be used for separation.

[0108] Alternatively, other orders can be used, such as placing the first extraction result before the first text sequence. The key is to ensure that the order used in the information extraction process is consistent with the order used in the model training process.

[0109] In this scenario, the encoding network in the information extraction model considers the influence of all tokens in the entire second text sequence when extracting feature representations of each token in the text to be extracted. That is, in addition to fully considering the semantic relationships of all content in the first text sequence, it also considers the semantic content of the first extraction result. Even if the first extraction result is inaccurate, it can be used as a basic semantic reference (which can be understood as basic knowledge retrieved from the model) during the second information extraction step, thereby improving the accuracy of the second step.

[0110] Continuing with the previous example, in the second step of information extraction, the information extraction model can obtain the second extraction result for the second text sequence as "person name: Yue Fei; place name: Xiangzhou Tangyin; work name: Man Jiang Hong".

[0111] Since the knowledge contained in pre-trained language models is not always relevant to downstream application tasks, it contains a large number of irrelevant instructions. Extracting irrelevant knowledge will negatively impact the model's information extraction performance. To ensure that the model extracts only knowledge relevant to a specified target type, this application proposes a training method for an information extraction model. This method can be implemented by... Figure 1 The model training device in the system architecture shown is executed. For example... Figure 4 As shown, the method may include the following steps:

[0112] Step 402: Obtain training data containing multiple training samples. The training samples include text samples and the extracted result samples corresponding to the text samples. The extracted result samples include the targets corresponding to the preset target type information in the text samples.

[0113] The training samples obtained in this step can be manually labeled, that is, manually identifying the targets corresponding to the preset target type information in the text samples, thereby obtaining the extracted result samples.

[0114] Step 404: Train an information extraction model based on a pre-trained language model using training data. This involves inputting a first text sequence (concatenated from target type information and text samples) into the information extraction model to obtain a first extraction result. The first extraction result includes the target corresponding to the target type information extracted from the text samples. Then, input a second text sequence (concatenated from the first text sequence and the first extraction result) into the information extraction model to obtain a second extraction result. The second extraction result includes the target corresponding to the target type information extracted from the text samples. The training objective is to minimize the difference between the second extraction result and the extraction result samples corresponding to the text samples.

[0115] As one feasible approach, the information extraction model can construct a loss function using the training objectives described above, taking the parameters of the pre-trained language model as initial parameters. In each iteration, the model parameters are updated using the loss function value and methods such as gradient descent, until a preset training termination condition is met. This termination condition may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0116] However, to enable the information extraction model to learn quickly and rapidly extract target semantics in downstream application scenarios, this application provides a more preferred implementation method. The information extraction model is trained using a meta-pre-training algorithm that incorporates two loops: an inner loop and an outer loop, simulating the model fine-tuning and prediction stages in downstream application scenarios, respectively. In this implementation, the training data is divided into a support set and a query set. The specific training process can be as follows: Figure 5 The process shown includes the following:

[0117] First, initialize the model parameters of the information extraction model.

[0118] One possible approach is to use the model parameters of a pre-trained language model to initialize the model parameters of the information extraction model.

[0119] Since information extraction tasks may be an ongoing process of development and evolution, involving the extraction of new target types, existing information extraction models can be migrated to the new target types. In this case, as another feasible approach, the model parameters of the information extraction model to be trained can be initialized using the model parameters of other information extraction models that have already been trained.

[0120] Then, the inner loop process is executed: backing up the currently obtained information extraction model to obtain a backup model; training the backup model with training samples from the support set to obtain the model parameters of the backup model. That is, inputting text samples from the training samples from the support set into the backup model, determining the loss function value constructed according to the training objective, using the loss function value to calculate the gradient of the model parameters of the backup model and updating the model parameters of the backup model, until the preset first training termination condition is met.

[0121] Each time the inner loop is executed, the initial parameters of the backup model are determined by the model parameters of the currently obtained information extraction model. During the first execution of the inner loop, the initial parameters of the backup model are actually the same as the model parameters initialized in the information extraction model. Subsequent executions of the inner loop use the same initial parameters obtained from the updates performed in the outer loop during the current training process.

[0122] As a preferred implementation, the value of the loss function Loss, constructed based on the training objective, can be obtained from the value of the first loss function Loss1 and the value of the second loss function Loss2. For example:

[0123] Loss = αLoss1 × βLoss2

[0124] Here, α and β are preset weighting coefficients. The value of the first loss function, Loss1, can be obtained based on the difference between the first extraction result corresponding to the text sample and the extraction result sample corresponding to the current sample. The value of the second loss function, Loss2, can be obtained based on the difference between the second extraction result corresponding to the text sample and the extraction result sample corresponding to the current text sample. Both Loss1 and Loss2 can be implemented using functions such as cross-entropy loss. This loss function, Loss, can comprehensively consider the accuracy of both the first and second extraction results.

[0125] The aforementioned first training termination condition can be, for example, the value of the loss function being less than or equal to a preset first loss function threshold, the number of iterations reaching a preset first iteration threshold, or the training samples being sampled from the support set after iteration, etc.

[0126] After the inner loop process is completed, the outer loop process is executed: training samples are sampled from the query set, text samples from the training samples are input into the currently trained backup model, the loss function value constructed based on the training objective is determined, the gradient of the model parameters of the currently obtained information extraction model is calculated using the loss function value, and the model parameters of the information extraction model are updated until the preset second training termination condition is met.

[0127] The loss function used in the outer loop is the same as that used in the inner loop, so it will not be elaborated here.

[0128] In the outer loop, the backup model is actually used to calculate the loss function value, and this value is used to update the parameters of the information extraction model. In other words, throughout the entire inner and outer loop process, the backup model trained in the inner loop merely serves as an intermediate parameter for calculating the loss function. The gradient calculation using the loss function is performed on the parameters of the information extraction model and used to update those parameters in reverse. This process can be found in [reference needed]. Figure 5 As shown in the image.

[0129] The second training termination condition mentioned above can be, for example, the value of the loss function being less than or equal to a preset second loss function threshold, the number of iterations reaching a preset second iteration threshold, or the training samples sampled from the query set having been completed, etc.

[0130] After the outer loop is completed, the process can return to the inner loop until the preset third training termination condition is met. For example, after the outer loop is completed, it can first be determined whether the preset third training termination condition is met. If it is met, the entire training process of the information extraction model ends, and the information extraction model is obtained. If it is not met, the process returns to the inner loop.

[0131] The third training termination condition mentioned above could be, for example, the number of iterations of the inner or outer loop process reaching a preset threshold, or the iterations of all training samples in the query set or support set N times, where N is a preset positive integer, etc.

[0132] The aforementioned model training mode, combining inner and outer loops, enables the model to learn rapidly in downstream application scenarios. Experiments have verified that training only a few times on the support set yields excellent results on the query set, reducing the amount of training data required and improving model performance. After training, the obtained information can be extracted and used by the model for further applications. Figure 2 Information extraction is performed in the process shown.

[0133] The methods provided in this application embodiment can be applied to various application scenarios, including but not limited to:

[0134] Application Scenario 1: Entity Extraction

[0135] Entity extraction refers to the extraction of entity words of a specific entity type from text to be extracted. It is an important task in the field of natural language processing and has been widely used in knowledge graph construction, question answering systems, syntactic analysis, machine translation and other fields.

[0136] like Figure 6 As shown, after obtaining the text to be extracted and the entity type information to be extracted, the first text sequence obtained by concatenating the entity type information and the text to be extracted can be input into the information extraction model to obtain the first extraction result output by the information extraction model. The first extraction result includes the entity words corresponding to the above-mentioned entity type information extracted from the text to be extracted.

[0137] Then, the second text sequence (i.e., the concatenation of entity type information, text to be extracted, and first extraction result) obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes entity words corresponding to the entity type information extracted from the text to be extracted.

[0138] The target type information mentioned above can be one or more combinations of names such as people, places, organizations, works, brands, addresses, dates, etc.

[0139] The above method performs two-step entity extraction. First, entity words corresponding to entity type information are extracted from the text to be extracted as the first extraction result. Then, the first extraction result is used as a basic semantic reference. In the second step of entity extraction, the auxiliary information extraction model extracts entity words corresponding to entity type information from the text to be extracted again, thereby improving the accuracy of entity extraction.

[0140] Application Scenario 2: Entity Relationship Extraction

[0141] Entity relation extraction involves not only extracting entity words but also the relationships between them—in other words, extracting triple information. Entity relation extraction is a fundamental task in knowledge graph construction.

[0142] In this application scenario, the triple information and the text to be extracted are concatenated to obtain the first text sequence, which is then input into the information extraction model to obtain the first extraction result output by the information extraction model. The first extraction result includes entity words and entity relation words extracted from the text to be extracted.

[0143] Then, the second text sequence (i.e., the sequence formed by concatenating the first text sequence and the first extraction result) is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes entity words and entity relation words extracted from the text to be extracted.

[0144] The triple information mentioned above can be, for example, head entity, entity relation, and tail entity. It can also be, for example, head entity, tail entity, and entity relation, etc.

[0145] The above method performs a two-step entity relation extraction. First, entity words and entity relation words are extracted from the text to be extracted as the first extraction result. Then, the first extraction result is used as a basic semantic reference. In the second step of entity extraction, the auxiliary information extraction model extracts entity words and entity relation words from the text to be extracted again, thereby improving the accuracy of entity relation extraction.

[0146] Application Scenario 3: Sentiment Extraction

[0147] Sentiment extraction refers to the process of extracting specific types of sentiment elements from text, primarily determining the role of words in expressing emotion. Sentiment extraction has been widely applied in various fields such as intelligent dialogue systems, e-commerce platforms, and trend prediction. For example, sentiment elements can be extracted from text data such as user dialogue data and evaluation data, and targeted services can be provided to users based on the extracted sentiment elements.

[0148] After obtaining the text to be extracted and the sentiment element type information, the first text sequence obtained by concatenating the sentiment element type information and the text to be extracted can be input into the information extraction model to obtain the first extraction result output by the information extraction model. The first extraction result includes the sentiment elements corresponding to the sentiment element type information extracted from the text to be extracted.

[0149] Then, the second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes the sentiment elements corresponding to the sentiment element type information extracted from the text to be extracted.

[0150] The target type information mentioned above can be one or more combinations of emotional holders, emotional objects, emotional words, etc.

[0151] The above method performs two-step sentiment extraction. First, sentiment elements corresponding to the sentiment element type information are extracted from the text to be extracted as the first extraction result. Then, the first extraction result is used as a basic semantic reference. In the second step of sentiment extraction, the auxiliary information extraction model extracts sentiment elements corresponding to the sentiment element type information from the text to be extracted again, thereby improving the accuracy of sentiment extraction.

[0152] As one possible implementation method, the information extraction method described above in this application embodiment can be executed by a cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and virtual private servers (VPS) services, such as high management difficulty and weak service scalability.

[0153] After the cloud server receives the text to be extracted sent by the user through the terminal device, it performs information extraction using the method provided in this application embodiment. Specifically, a first text sequence, obtained by concatenating target type information and the text to be extracted, is input into the information extraction model to obtain a first extraction result. The first extraction result includes the target corresponding to the target type information extracted from the text to be extracted. A second text sequence, obtained by concatenating the first text sequence and the first extraction result, is input into the information extraction model to obtain a second extraction result. The second extraction result includes the target corresponding to the target type information extracted from the text to be extracted. The second extraction result is then returned to the terminal device.

[0154] The aforementioned terminal devices may include, but are not limited to, smart mobile terminals, smart home devices, wearable devices, and PCs (Personal Computers). Smart mobile devices may include mobile phones, tablets, laptops, PDAs (Personal Digital Assistants), and connected cars. Smart home devices may include smart TVs, smart speakers, and smart refrigerators. Wearable devices may include smartwatches, smart glasses, virtual reality devices, augmented reality devices, and mixed reality devices (i.e., devices that support both virtual and augmented reality).

[0155] The above describes embodiments of the methods provided in this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0156] According to another embodiment, an information extraction device is provided. Figure 7 A schematic block diagram of an information extraction apparatus according to one embodiment is shown. Figure 7 As shown, the device 700 includes: a text acquisition unit 701, a first extraction unit 702, and a second extraction unit 703. The main functions of each component are as follows:

[0157] The text acquisition unit 701 is configured to acquire the text to be extracted.

[0158] The first extraction unit 702 is configured to input a first text sequence obtained by concatenating target type information and text to be extracted into an information extraction model, and obtain a first extraction result output by the information extraction model. The first extraction result includes the target corresponding to the target type information extracted from the text to be extracted.

[0159] The second extraction unit 703 is configured to input a second text sequence obtained by concatenating the first text sequence and the first extraction result into the information extraction model, and obtain the second extraction result output by the information extraction model. The second extraction result includes the target corresponding to the target type information extracted from the text to be extracted.

[0160] The target type information mentioned above may include information about one target type or information about multiple target types.

[0161] The information extraction model described above is pre-trained based on a pre-trained language model. The relevant principles and structure of the information extraction model can be found in the descriptions in the method embodiments, and will not be elaborated upon here.

[0162] According to another embodiment, an apparatus for training an information extraction model is provided, the apparatus corresponding to Figure 1 The model training device in the system architecture shown. Figure 8 A schematic block diagram of the apparatus for extracting training information from the model provided in this application. Figure 8 As shown, the device 800 includes a sample acquisition unit 801 and a model training unit 802. The main functions of each component are as follows:

[0163] The sample acquisition unit 801 is configured to acquire training data containing multiple training samples, including text samples and extraction result samples corresponding to the text samples, and the extraction result samples include targets corresponding to preset target type information in the text samples.

[0164] The model training unit 802 is configured to train an information extraction model based on a pre-trained language model using training data. A first text sequence, obtained by concatenating target type information and text samples, is input into the information extraction model to obtain a first extraction result, which includes the target corresponding to the target type information extracted from the text samples. A second text sequence, obtained by concatenating the first text sequence and the first extraction result, is input into the information extraction model to obtain a second extraction result, which also includes the target corresponding to the target type information extracted from the text samples. The training objective includes minimizing the difference between the second extraction result and the extraction result samples corresponding to the text samples.

[0165] As one possible implementation, the training data acquired by the sample acquisition unit 801 may include a support set and a query set. The model training unit 802 may be specifically configured to perform the following processes to train the information extraction model:

[0166] Initialize the model parameters of the information extraction model;

[0167] Backup the currently obtained information extraction model to obtain a backup model;

[0168] The system starts by sampling text samples from the support set and inputting them into the backup model. It then determines the loss function value based on the training objective, uses the loss function value to calculate the gradient for the model parameters of the backup model, and updates the model parameters of the backup model until the preset first training termination condition is met.

[0169] The system samples text samples from the query set and inputs them into the currently trained backup model. It determines the loss function value constructed based on the training objective, uses the loss function value to calculate the gradient of the model parameters of the currently obtained information extraction model, and updates the model parameters of the information extraction model until the preset second training termination condition is met.

[0170] The process continues until the third preset training termination condition is met, then proceeds to the backup model by extracting the currently obtained information.

[0171] As one possible approach, the aforementioned loss function value is obtained based on the difference between the second extraction result and the extraction result sample corresponding to the text sample.

[0172] As another more preferred implementation, the above loss function value is obtained based on the first loss function value and the second loss function value, wherein the first loss function value is obtained based on the difference between the first extraction result and the extraction result sample corresponding to the text sample, and the second loss function value is obtained based on the difference between the second extraction result and the extraction result sample corresponding to the text sample.

[0173] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0174] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0175] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0176] And an electronic device, comprising:

[0177] One or more processors; and

[0178] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0179] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0180] in, Figure 9 An exemplary architecture of an electronic device is shown, which may include a processor 910, a video display adapter 911, a disk drive 912, an input / output interface 913, a network interface 914, and a memory 920. The processor 910, video display adapter 911, disk drive 912, input / output interface 913, network interface 914, and memory 920 can communicate with each other via a communication bus 930.

[0181] The processor 910 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0182] The memory 920 can be implemented in the form of ROM (Read-Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 920 can store the operating system 921 for controlling the operation of the electronic device 900, and the basic input / output system (BIOS) 922 for controlling the low-level operations of the electronic device 900. Additionally, it can store a web browser 923, a data storage management system 924, and an information extraction device / model training device 925, etc. The aforementioned information extraction device / model training device 925 can be the application program that specifically implements the aforementioned steps in the embodiments of this application. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 920 and is called and executed by the processor 910.

[0183] Input / output interface 913 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0184] Network interface 914 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0185] Bus 930 includes a pathway for transmitting information between various components of the device, such as processor 910, video display adapter 911, disk drive 912, input / output interface 913, network interface 914, and memory 920.

[0186] It should be noted that although the above-described device only shows the processor 910, video display adapter 911, disk drive 912, input / output interface 913, network interface 914, memory 920, bus 930, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0187] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0188] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.< / wt> < / pn> < / n> < / wt> < / pn> < / n>

Claims

1. A method for information extraction, characterized in that, The method includes: Get the text to be extracted; The first text sequence obtained by concatenating the target type information and the text to be extracted is input into the information extraction model, and the first extraction result output by the information extraction model is obtained. The first extraction result includes the target corresponding to the target type information extracted from the text to be extracted. The second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes the target corresponding to the target type information extracted from the text to be extracted. The information extraction model is obtained by pre-training a pre-trained language model based on training data. The training data includes a support set and a query set. The support set is used to update the model parameters of the backup model, and the query set is used to update the model parameters of the information extraction model. The backup model is obtained by backing up the information extraction model.

2. The method according to claim 1, characterized in that, The target type information includes information on multiple target types.

3. The method according to claim 1, characterized in that, The information extraction model includes: an encoding network and a decoding network; The encoding network is used to encode the features of each element Token in the input text sequence to obtain the feature representation of each Token. A preset separator is inserted in the input text sequence at the concatenation position. The decoding network is used to perform decoding processing using the feature representation of each token to obtain an output text sequence. The output text sequence includes the target extracted from the text to be extracted and its corresponding target type identifier.

4. An entity extraction method, characterized in that, The method includes: The first text sequence obtained by concatenating entity type information and the text to be extracted is input into the information extraction model, and the first extraction result output by the information extraction model is obtained. The first extraction result includes entity words corresponding to the entity type information extracted from the text to be extracted. The second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes entity words corresponding to the entity type information extracted from the text to be extracted. The information extraction model is obtained by pre-training a pre-trained language model based on training data. The training data includes a support set and a query set. The support set is used to update the model parameters of the backup model, and the query set is used to update the model parameters of the information extraction model. The backup model is obtained by backing up the information extraction model.

5. A method for extracting entity relations, characterized in that, The method includes: The first text sequence obtained by concatenating triple information and the text to be extracted is input into the information extraction model, and the first extraction result output by the information extraction model is obtained. The first extraction result includes entity words and entity relation words extracted from the text to be extracted that correspond to the triple information. The second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes entity words and entity relation words extracted from the text to be extracted that correspond to the triple information. The information extraction model is obtained by pre-training a pre-trained language model based on training data. The training data includes a support set and a query set. The support set is used to update the model parameters of the backup model, and the query set is used to update the model parameters of the information extraction model. The backup model is obtained by backing up the information extraction model.

6. An emotion extraction method, characterized in that, The method includes: The first text sequence obtained by concatenating the sentiment element type information and the text to be extracted is input into the information extraction model, and the first extraction result output by the information extraction model is obtained. The first extraction result includes the sentiment element corresponding to the sentiment element type information extracted from the text to be extracted. The second text sequence obtained by concatenating the first text sequence and the first extraction result is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes the emotional element corresponding to the emotional element type information extracted from the text to be extracted. The information extraction model is obtained by pre-training a pre-trained language model based on training data. The training data includes a support set and a query set. The support set is used to update the model parameters of the backup model, and the query set is used to update the model parameters of the information extraction model. The backup model is obtained by backing up the information extraction model.

7. A method for training an information extraction model, characterized in that, The method includes: Acquire training data containing multiple training samples, wherein the training samples include text samples and extraction result samples corresponding to the text samples, and the extraction result samples include targets corresponding to preset target type information in the text samples; The training data is used to train an information extraction model based on a pre-trained language model, wherein a first text sequence obtained by concatenating target type information and text samples is input into the information extraction model, and a first extraction result output by the information extraction model is obtained. The first extraction result includes the target corresponding to the target type information extracted from the text samples. The second text sequence, obtained by concatenating the first text sequence and the first extraction result, is input into the information extraction model to obtain the second extraction result output by the information extraction model. The second extraction result includes the target corresponding to the target type information extracted from the text sample. The training objective includes minimizing the difference between the second extraction result and the extraction result sample corresponding to the text sample. The training data includes a support set and a query set. The support set is used to update the model parameters of the backup model, and the query set is used to update the model parameters of the information extraction model. The backup model is obtained by backing up the information extraction model.

8. The method according to claim 7, characterized in that, The method further includes: Training an information extraction model based on a pre-trained language model using the training data includes: Initialize the model parameters of the information extraction model; Backup the currently obtained information extraction model to obtain a backup model; The text samples are sampled from the support set and input into the backup model. The loss function value is determined based on the training objective. The gradient is calculated for the model parameters of the backup model using the loss function value, and the model parameters of the backup model are updated until the preset first training termination condition is met. Text samples are sampled from the query set and input into the currently trained backup model. The loss function value is determined based on the training objective. The gradient of the model parameters of the currently obtained information extraction model is calculated using the loss function value, and the model parameters of the information extraction model are updated until the preset second training termination condition is met. The process continues until the preset third training termination condition is met, then proceeds to the step of extracting the currently obtained information from the backup model.

9. The method according to claim 8, characterized in that, The loss function value is obtained based on the difference between the second extraction result and the extraction result sample corresponding to the text sample; or... The loss function value is obtained based on a first loss function value and a second loss function value, wherein the first loss function value is obtained based on the difference between the first extraction result and the extraction result sample corresponding to the text sample, and the second loss function value is obtained based on the difference between the second extraction result and the extraction result sample corresponding to the text sample.

10. The method according to any one of claims 7 to 9, characterized in that, The information extraction model includes: an encoding network and a decoding network; The encoding network is used to encode the features of each token in the input text sequence to obtain the feature representation of each token. A preset separator is inserted in the input text sequence at the concatenation position. The decoding network is used to perform decoding processing using the feature representations of each token to obtain a text sequence. The output text sequence includes the target extracted from the text sample and its corresponding target type identifier.

11. An information extraction device, characterized in that, The device includes: The text acquisition unit is configured to acquire the text to be extracted. The first extraction unit is configured to input a first text sequence obtained by concatenating target type information and text to be extracted into an information extraction model, and obtain a first extraction result output by the information extraction model. The first extraction result includes the target corresponding to the target type information extracted from the text to be extracted. The second extraction unit is configured to input a second text sequence obtained by concatenating the first text sequence and the first extraction result into the information extraction model, and obtain a second extraction result output by the information extraction model. The second extraction result includes the target corresponding to the target type information extracted from the text to be extracted. The information extraction model is obtained by pre-training a pre-trained language model based on training data. The training data includes a support set and a query set. The support set is used to update the model parameters of the backup model, and the query set is used to update the model parameters of the information extraction model. The backup model is obtained by backing up the information extraction model.

12. An apparatus for training an information extraction model, characterized in that, The device includes: The sample acquisition unit is configured to acquire training data containing multiple training samples, wherein the training samples include text samples and extraction result samples corresponding to the text samples, and the extraction result samples include targets corresponding to preset target type information in the text samples; The model training unit is configured to train an information extraction model based on a pre-trained language model using the training data. Specifically, a first text sequence, obtained by concatenating target type information and text samples, is input into the information extraction model to obtain a first extraction result, which includes the target corresponding to the target type information extracted from the text samples. A second text sequence, obtained by concatenating the first text sequence and the first extraction result, is input into the information extraction model to obtain a second extraction result, which includes the target corresponding to the target type information extracted from the text samples. The training objective includes minimizing the difference between the second extraction result and the extraction result sample corresponding to the text samples. The training data includes a support set and a query set. The support set is used to update the model parameters of a backup model, and the query set is used to update the model parameters of the information extraction model. The backup model is obtained by backing up the information extraction model.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 10.

14. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 10.

Images