Information extraction method, method and device for training question and answer processing model

Abstract

The embodiment of the application discloses an information extraction method, a method and device for training a question and answer processing model. The main technical scheme comprises: obtaining a text to be extracted and first target content corresponding to a first target type in the text to be extracted; inputting a first text sequence obtained by splicing the text to be extracted and the first prompt text into the question and answer processing model; obtaining a first answer text containing second target content corresponding to a second target type output by the question and answer processing model; inputting a second text sequence obtained by splicing the text to be extracted, the first answer text and the second prompt text into the question and answer processing model; obtaining a second answer text containing third target content corresponding to a third target type output by the question and answer processing model; wherein the question and answer processing model is obtained by pre-training based on a pre-trained language model. The application improves the accuracy of information extraction through a chain reasoning method based on prompt learning.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to an information extraction method, a method and apparatus for training a question-answering processing model. Background Technology

[0002] With the widespread use of the internet, a massive amount of text data is generated online every day, containing immense value, giving rise to information extraction technology. Information extraction (IE) refers to extracting specific types of targets from text, thereby automatically classifying, extracting, and reconstructing massive amounts of content, such as named entity recognition, entity relationship extraction, event extraction, and sentiment extraction.

[0003] With the rapid development of deep learning technology, information extraction techniques have gradually evolved from dictionary-based and rule-based methods to deep learning model-based methods. However, while deep learning model-based information extraction performs well in extracting explicit information, it relies heavily on a deep semantic understanding of the input text for implicit information extraction, where current techniques are less effective. For example, when extracting sentiment polarity, existing techniques can accurately extract "positive" sentiment polarity for explicit expressions like "This phone is great." However, for implicit expressions like "My phone fits perfectly in my pocket," which actually means the phone is the right size and should also have a "positive" sentiment polarity, current techniques struggle to extract this accurately, often resulting in "neutral" sentiment polarity. Summary of the Invention

[0004] In view of this, this application provides an information extraction method, a method and apparatus for training a question-answering processing model, in order to improve the accuracy of information extraction.

[0005] This application provides the following solution:

[0006] Firstly, a method for information extraction is provided, the method comprising:

[0007] Obtain the text to be extracted and the first target content corresponding to the first target type in the text to be extracted;

[0008] The first text sequence, obtained by concatenating the text to be extracted and the first prompt text, is input into the question-answering processing model. The first prompt text contains information about the first target content and the second target type. The first answer text, which contains the second target content corresponding to the second target type, is obtained from the output of the question-answering processing model.

[0009] The second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, is input into the question-and-answer processing model. The second prompt text contains information about the first target content and the third target type. The second answer text, which contains the third target content corresponding to the third target type, is then obtained from the output of the question-and-answer processing model.

[0010] The question-answering processing model mentioned above is pre-trained based on a pre-trained language model.

[0011] According to one achievable method in this application embodiment, the first target content corresponding to the first target type in the text to be extracted is obtained in the following way:

[0012] Retrieve the first target content specified by the user; or,

[0013] The first target content corresponding to the first target type is extracted from the text to be extracted using an information extraction model; or...

[0014] The fourth text sequence, obtained by concatenating the text to be extracted and the fourth prompt text, is input into the question-and-answer processing model. The fourth prompt text contains information about the first target type. The first target content corresponding to the first target type is obtained from the fourth answer text output by the question-and-answer processing model.

[0015] According to one achievable method in the embodiments of this application, the method further includes:

[0016] The third text sequence, obtained by concatenating the text to be extracted, the first answer text, the second answer text, and the third prompt text, is input into the question-and-answer processing model. The third prompt text contains information about the first target content and the fourth target type. The third answer text, which contains the fourth target content corresponding to the fourth target type, is then obtained from the output of the question-and-answer processing model.

[0017] According to one achievable method in the embodiments of this application, the first prompt text is obtained by filling the information of the first target content and the second target type into the corresponding slots of a preset first question template;

[0018] The second prompt text is obtained by filling the information of the first target content and the third target type into the corresponding slots of the preset second question template.

[0019] Secondly, a method for information extraction is provided, the method being executed by a cloud server, the method comprising:

[0020] Obtain the text to be extracted from the user terminal, and obtain the first target content corresponding to the first target type in the text to be extracted;

[0021] The first text sequence, obtained by concatenating the text to be extracted and the first prompt text, is input into the question-answering processing model. The first prompt text contains information about the first target content and the second target type. The first answer text, which contains the second target content corresponding to the second target type, is obtained from the output of the question-answering processing model.

[0022] The second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, is input into the question-and-answer processing model. The second prompt text contains information about the first target content and the third target type. The second answer text, which contains the third target content corresponding to the third target type, is then obtained from the output of the question-and-answer processing model.

[0023] The extraction result is generated using the third target content, and the extraction result is returned to the user terminal.

[0024] Thirdly, an emotion extraction method is provided, the method comprising:

[0025] Obtain the text to be extracted and the sentiment object words in the text to be extracted;

[0026] The first text sequence, obtained by concatenating the text to be extracted and the first prompt text, is input into the question-and-answer processing model. The first prompt text contains information about the sentiment object words and the opinion words. The first answer text containing the opinion words is then obtained from the output of the question-and-answer processing model.

[0027] The second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, is input into the question-and-answer processing model. The second prompt text contains the sentiment object words and information indicating sentiment polarity. The second answer text containing sentiment polarity is then obtained from the output of the question-and-answer processing model.

[0028] The emotional polarity is obtained from the second answer text.

[0029] Fourthly, an emotion extraction method is provided, the method comprising:

[0030] Obtain the text to be extracted and the sentiment object words in the text to be extracted;

[0031] The first text sequence, obtained by concatenating the text to be extracted and the first prompt text, is input into the question-answering processing model. The first prompt text contains the sentiment object words and information indicating aspects. The first answer text, containing aspect words, is then obtained from the output of the question-answering processing model.

[0032] The second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, is input into the question-and-answer processing model. The second prompt text contains type information of the sentiment object words and the opinion words. The second answer text containing opinion words is then obtained from the output of the question-and-answer processing model.

[0033] The third text sequence, obtained by concatenating the text to be extracted, the first answer text, the second answer text, and the third prompt text, is input into the question-and-answer processing model. The third prompt text contains the sentiment object words and information indicating sentiment polarity. The third answer text containing sentiment polarity is then obtained from the output of the question-and-answer processing model.

[0034] The emotional polarity is obtained from the third answer text.

[0035] Fifthly, a method for training a question-answering processing model is provided, the method comprising:

[0036] Acquire training data including multiple training samples, wherein the training samples include text samples and third target content labels corresponding to the third target type labeled on the text samples;

[0037] A question-answering processing model based on a pre-trained language model is trained using the training data. The process includes: obtaining the first target content corresponding to a first target type from the text samples; inputting a first text sequence obtained by concatenating the text samples and a first prompt text into the question-answering processing model, where the first prompt text contains information about the first target content and the second target type; obtaining the first answer text output by the question-answering processing model, which contains the second target content corresponding to the second target type; inputting a second text sequence obtained by concatenating the text samples, the first answer text, and the second prompt text into the question-answering processing model, where the second prompt text contains information about the first target content and the third target type; and obtaining the second answer text output by the question-answering processing model, which contains the third target content corresponding to the third target type.

[0038] The training objective includes minimizing the difference between the third target content contained in the second answer text and the corresponding third target content label.

[0039] According to one achievable method in an embodiment of this application, the training sample further includes a second target content label corresponding to the second target type labeled in the text sample;

[0040] The training objective also includes minimizing the difference between the second target content contained in the first answer text and the corresponding second target content label.

[0041] According to one achievable method in the embodiments of this application, the first prompt text is obtained by filling the information of the first target content and the second target type into the corresponding slots of a preset first question template;

[0042] The second prompt text is obtained by filling the information of the first target content and the third target type into the corresponding slots of the preset second question template.

[0043] Sixthly, an information extraction device is provided, the device comprising:

[0044] The text acquisition unit is configured to acquire the text to be extracted.

[0045] The first target acquisition unit is configured to acquire the first target content corresponding to the first target type in the text to be extracted;

[0046] The second target acquisition unit is configured to input a first text sequence obtained by concatenating the text to be extracted and the first prompt text into the question-and-answer processing model, wherein the first prompt text contains information about the first target content and the second target type; and to acquire the first answer text output by the question-and-answer processing model, which contains the second target content corresponding to the second target type.

[0047] The third target acquisition unit is configured to input a second text sequence obtained by concatenating the text to be extracted, the first answer text, and the second prompt text into the question-and-answer processing model, wherein the second prompt text contains information about the first target content and the third target type; and to acquire the second answer text output by the question-and-answer processing model, which contains the third target content corresponding to the third target type.

[0048] The question-answering processing model mentioned above is pre-trained based on a pre-trained language model.

[0049] In a seventh aspect, an apparatus for training a question-answering processing model is provided, the apparatus comprising:

[0050] The sample acquisition unit is configured to acquire training data including multiple training samples, wherein the training samples include text samples and third target content tags corresponding to the third target type labeled on the text samples;

[0051] The model training unit is configured to train a question-answering processing model based on a pre-trained language model using the training data. Specifically, it acquires first target content corresponding to a first target type from the text samples; inputs a first text sequence obtained by concatenating the text samples and a first prompt text into the question-answering processing model, where the first prompt text contains information about the first target content and the second target type; acquires a first answer text output by the question-answering processing model containing second target content corresponding to the second target type; inputs a second text sequence obtained by concatenating the text samples, the first answer text, and the second prompt text into the question-answering processing model, where the second prompt text contains information about the first target content and a third target type; acquires a second answer text output by the question-answering processing model containing third target content corresponding to the third target type; the training objective includes minimizing the difference between the third target content contained in the second answer text and the corresponding third target content label.

[0052] According to an eighth aspect, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any one of the first to fifth aspects.

[0053] According to the ninth aspect, an electronic device is provided, comprising:

[0054] One or more processors; and

[0055] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first to fifth aspects.

[0056] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0057] 1) This application extracts the first target content from the text to be extracted, and then inputs the first text sequence obtained by concatenating the text to be extracted and the first prompt text into the question-answering model to help the model obtain the first answer text containing the second target content. Then, the second text sequence obtained by concatenating the text to be extracted, the first answer text, and the second prompt text into the question-answering model is input into the model to help it obtain the second answer text containing the third target content. This chain-like reasoning method based on prompt learning helps the question-answering model fully utilize the powerful understanding ability of the pre-trained language model to gradually understand the deep semantics of the text to be extracted, thereby improving the accuracy of information extraction. In other words, the question-answering model obtains the target content corresponding to one target type in one step, and gradually obtains the target content corresponding to each target type in multiple steps. The obtained target content will assist the question-answering model in understanding the next target type. Therefore, when extracting implicit information, the model can effectively utilize the already obtained target content to understand the implicit information, resulting in a particularly outstanding extraction effect.

[0058] 2) This application can utilize a preset question template to fill in the obtained target content and the target type information to be extracted into the question template, thereby automatically generating prompt text, which is then concatenated with the text to be extracted or further with the already obtained answer text and input into the question-answering processing model. The entire process can be automated without manual intervention.

[0059] 3) This application first uses a question-and-answer processing model based on a prompting mechanism to infer the aspect words of the emotional object targeted by the sentence. Then, it further uses a question-and-answer processing model based on a prompting mechanism to infer the opinion words for that aspect word. Finally, it uses a question-and-answer processing model based on a prompting mechanism to infer the emotional polarity. By using three progressive inference steps, this application fully utilizes the understanding ability of the language model to gradually understand the deep semantics of the sentence, thereby accurately extracting the implicitly expressed emotional polarity.

[0060] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a system architecture diagram applicable to the embodiments of this application;

[0063] Figure 2 A flowchart of the information extraction method provided in the embodiments of this application;

[0064] Figure 3 A schematic diagram illustrating the principle of the question-answering processing model provided in the embodiments of this application;

[0065] Figure 4 A flowchart of the emotion extraction method provided in this application embodiment;

[0066] Figure 5 A flowchart illustrating the method for training a question-answering processing model provided in this application embodiment;

[0067] Figure 6 A schematic block diagram of an information extraction device provided in the embodiments of this application;

[0068] Figure 7 A schematic block diagram of an apparatus for training a question-answering processing model provided in an embodiment of this application;

[0069] Figure 8 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0070] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0071] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0072] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0073] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0074] With the emergence of pre-trained language models, especially large-scale language models, research and applications of Natural Language Processing (NLP) have begun to focus on pre-trained language models. Because pre-trained language models possess powerful understanding and question-answering capabilities, prompt learning mechanisms have emerged. Prompt learning, without significantly altering the structure and parameters of the pre-trained language model, transforms the downstream task into a text generation task by adding prompt information to the input. This application references the prompt learning mechanism and, based on it, provides a chain-reasoning approach to achieve information extraction.

[0075] To facilitate understanding of this application, a brief description of the system architecture to which this application applies will be given first. Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown, such as Figure 1 As shown, the system architecture includes a model training device that builds a question-answering processing model offline, and an information extraction device that performs information extraction online.

[0076] The information extraction method provided in this application embodiment is implemented using a question-answering processing model, that is, the information extraction task is transformed into a question-answering task. After acquiring training data, the model training device pre-trains the question-answering processing model using the method provided in this application embodiment.

[0077] The question-answering processing model uses the method provided in the embodiments of this application to obtain an output text sequence from an input text sequence, wherein the output text sequence contains target content corresponding to a specific target type. The target can correspond to different content in different application scenarios, such as entity words, entity relationships, sentiment elements, event information, etc., which will be detailed in subsequent embodiments. The information extraction device can use the above-described question-answering processing model to extract target content of a preset target type from the text to be extracted.

[0078] The model training device and the information extraction device can be configured as separate servers, or they can be configured on the same server or server group, or they can be configured on a separate or the same cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPS) services, such as high management difficulty and weak service scalability. The model training device or information extraction device can also be configured on a computer terminal with strong computing power.

[0079] In one common scenario, users can use a user terminal to interact with an information extraction device set up on the server. Various applications can be installed on the user terminal, such as voice interaction applications, web browser applications, and communication applications.

[0080] User terminals can be various electronic devices, including those with screens and those without. These include, but are not limited to, smartphones, tablets, smart speakers, smart TVs, PCs (Personal Computers), wearable devices, and so on.

[0081] Users can input text to be extracted through their user terminals. This text is sent to an information extraction device on a server via the network. The information extraction device then extracts the target content of a preset target type from the text and returns the extracted target content to the user terminal. The preset target type can be a user-specified target type or a default target type.

[0082] It should be understood that Figure 1 The number of model training devices, information extraction devices, user terminals, and question-answering models shown in the diagram is merely illustrative. Depending on implementation needs, any number of model training devices, information extraction devices, user terminals, and question-answering models can be included.

[0083] Figure 2 This is a flowchart of an information extraction method provided in an embodiment of this application. The method flow can be... Figure 1 The information extraction device in the system architecture shown is executed. For example... Figure 2 As shown, the method may include the following steps:

[0084] Step 202: Obtain the text to be extracted and the content of the first target type in the text to be extracted.

[0085] Step 204: Input the first text sequence obtained by concatenating the text to be extracted and the first prompt text into the question answering model. The first prompt text contains information about the first target content and the second target type. Obtain the first answer text output by the question answering model, which contains the second target content corresponding to the second target type. The question answering model is pre-trained based on a pre-trained language model.

[0086] Step 206: Input the second text sequence obtained by concatenating the text to be extracted, the first answer text, and the second prompt text into the question-answering processing model. The second prompt text contains information about the first target content and the third target type. Obtain the second answer text corresponding to the third target content of the third target type output by the question-answering processing model.

[0087] As can be seen from the above process, after extracting the first target content from the text to be extracted, this application inputs the first text sequence obtained by concatenating the text to be extracted and the first prompt text into the question-answering model, which helps the question-answering model obtain the first answer text containing the second target content; then, the second text sequence obtained by concatenating the text to be extracted, the first answer text, and the second prompt text into the question-answering model, which helps the question-answering model obtain the second answer text containing the third target content. This chain reasoning method based on prompt learning helps the question-answering model fully utilize the powerful understanding ability of the pre-trained language model to gradually understand the deep semantics of the text to be extracted, thereby improving the accuracy of information extraction.

[0088] It should be noted that the terms "first," "second," etc., used in this disclosure do not imply any limitation in size, order, or quantity; they are merely used to distinguish between them by name. For example, "first target type," "second target type," and "third target type" distinguish three target types by name. Similarly, "first prompt text" and "second prompt text" distinguish two prompt texts by name. And again, "first text sequence" and "second text sequence" distinguish two text sequences by name, and so on.

[0089] The following describes each step in the above method flow. First, step 202, namely "obtaining the text to be extracted and the first target content corresponding to the first target type in the text to be extracted", will be described in detail with reference to the embodiment.

[0090] The text to be extracted in this application embodiment can be text obtained from a user device. For example, if a user wants to extract information of a specific target type from the text, the user device can upload the text to the information extraction device on the server side. The information extraction device will then use the text as the text to be extracted and extract the target content of the specific target type from the text to be extracted using the method provided in this application embodiment.

[0091] The text to be extracted in this embodiment can also be text obtained from a database. For example, if it is necessary to extract information of a specific target type from a batch of text in a database, each text obtained from the database can be used as the text to be extracted, and the target content of a specific target type can be extracted from the text to be extracted using the method provided in this embodiment.

[0092] The text to be extracted can be sentences, paragraphs composed of sentences, documents, or even smaller units of text, such as phrases.

[0093] The information extraction involved in this application aims to extract information of a specific target type. The first target content of the first target type obtained in this step can be a target type that is semantically basic, explicit, and easy to understand. Taking sentiment extraction as an example, sentiment extraction typically involves extracting sentiment elements such as sentiment object words, aspect words, evaluation words, and sentiment polarity. Among these, aspect words, evaluation words, and sentiment polarity are all based on sentiment objects, and sentiment objects are usually explicitly expressed in text, making extraction relatively easy. Therefore, sentiment objects can be used as the first target type for initial extraction. A detailed description using sentiment extraction as an example will follow.

[0094] The following methods can be used, but are not limited to, to obtain the content of the first target type in the text to be extracted:

[0095] The first method: Obtain the first target content specified by the user.

[0096] When extracting information, the primary target content can be specified by the user. This is equivalent to the user determining what other target content of different types they want to extract from the text based on the primary target content. For example, if a user wants to extract the emotional polarity of a specific emotional object from the text to be extracted, they can input the emotional object words from the text.

[0097] The second approach is to use an information extraction model to extract the first target content corresponding to the first target type from the text to be extracted.

[0098] Since the primary target content is usually explicit and relatively easy to understand in the text, existing information extraction models are quite accurate in identifying the primary target content of the primary target type. Therefore, existing information extraction models can be used to extract the primary target content corresponding to the primary target type from the text to be extracted. For example, a sentiment extraction model can be used to extract sentiment objects from the text to be extracted.

[0099] The third method involves inputting the fourth text sequence, obtained by concatenating the text to be extracted and the fourth prompt text, into the question-answering processing model. The fourth prompt text contains information about the first target type. The first target content corresponding to the first target type is then obtained from the fourth answer text output by the question-answering processing model.

[0100] Since the third extraction method uses a cue-based learning mechanism, it will be explained in detail in the description of step 204.

[0101] The following describes in detail step 204, namely, "inputting the first text sequence obtained by concatenating the text to be extracted and the first prompt text into the question-and-answer processing model, wherein the first prompt text contains information about the first target content and the second target type; and obtaining the first answer text output by the question-and-answer processing model that contains the second target content corresponding to the second target type".

[0102] While some techniques already exist in the NLP field that utilize cue learning mechanisms, they are primarily used to adapt pre-trained language models to specific tasks, exhibiting limited reasoning capabilities for tasks related to deep semantic understanding. This application combines cue learning mechanisms with a chain-based reasoning approach. The chain-based reasoning in this application includes at least two reasoning steps; this step pertains to one of those steps.

[0103] First, the text to be extracted and the first prompt text are concatenated to obtain the first text sequence. The first prompt text includes the first target content already extracted in the previous step, as well as information about the second target type needed for reasoning in this step.

[0104] As one possible approach, the aforementioned first target type, second target type, third target type, etc., can progressively increase in difficulty of understanding. That is, first, the target content corresponding to the relatively easier target type is extracted, and then the extracted target content is used to help understand the target content corresponding to the relatively difficult target type, and so on. This implementation method is a preferred embodiment, but this application is not limited to this implementation method.

[0105] The first prompt text can be a preset first question template. The first target content and the information indicating the second target type are filled into the corresponding slots in the preset first question template to obtain the first prompt text.

[0106] For example, the template for the first question corresponding to the first prompt text could be: "Which of [T2] in [t1] might be mentioned?"

[0107] In the first question template mentioned above, the slot corresponding to [t1] can be filled with the content of the first target, and the slot corresponding to [T2] can be filled with information of the second target type.

[0108] In addition, when concatenating the text to be extracted and the first prompt text, you can use a format such as "'Given sentence [S]' + first prompt text", where the slot corresponding to [S] can be filled with the text to be extracted.

[0109] The following section will use sentiment extraction as a specific application scenario as an example to illustrate the concept.

[0110] As one feasible approach, the aforementioned first question template can be a template set by human experience. It can be seen that, in addition to the first target content and information indicating the second target type, the first question template also includes some context to reflect the semantic relationships between the first target content, the second target type, and the text to be extracted. This context can be set by human experience.

[0111] Besides filling the first target content and the information indicating the second target type into the corresponding slots in the preset first question template to obtain the first prompt text, this application is not limited to other methods of obtaining the first prompt text. For example, specific wording can be used to concatenate the first target content and the information indicating the second target type to obtain the first prompt text, and so on.

[0112] This application's embodiments transform the original information extraction task into a question-answering task by adding prompt text. The question-answering processing model uses a Seq2Seq (Sequence to Sequence) model. Its structure can be as follows: Figure 3 As shown, it includes an encoding network (Encoder) and a decoding network (Decoder).

[0113] The encoding network is used to encode the features of each element Token in the input text sequence (which is the first text sequence in this case) to obtain the feature representation of each Token.

[0114] The encoding network first performs embedding processing on each token in the input text sequence. Each token in the text sequence refers to an element that constitutes the text sequence. For a text sequence, if the text sequence is segmented into a sequence of characters or words, then the characters or words in the text sequence, as well as the start character and the separator, are all tokens.

[0115] The above embedding process can include at least two parts: word embedding and position embedding. Word embedding involves encoding each token into a word vector representation. Position embedding involves encoding the position of each token within the text sequence to be predicted, thus obtaining a position representation.

[0116] The decoding network is used to perform decoding processing using the characteristic representations of each token, and the output is also a text sequence. This output text sequence is used as the answer text (the first answer text is output in this step). The first answer text contains the second target content corresponding to the second target type.

[0117] The first text sequence is equivalent to a question about the second target type in the text to be extracted, and the question-answering model outputs an answer about the second target type in the text to be extracted, i.e., the content of the second target.

[0118] For example, for a first text sequence of the form "Given sentence [S], which [T2] of [t1] might be mentioned in it?", the first answer text could be something like "The T2 of t1 mentioned in the sentence is t2". Here, t2 is the second target content corresponding to the second target type T2 in the text to be extracted.

[0119] The question-answering processing model described in this application embodiment can be trained based on a pre-trained language model. It employs pre-trained language models such as T5 (TransferText-to-Text Transformer), BERT (Bidirectional Encoder Representation from Transformers), XLNet (an autoregressive model that uses permutations of language models to achieve bidirectional contextual information), and GPT (Generative Pre-Training) as its model framework, and then performs further training on top of these. BERT is a bidirectional pre-trained language model that uses a Transformer Encoder as its model structure, and it effectively utilizes contextual information for feature learning. XLNet is a model similar to BERT, a more generalized autoregressive pre-trained model. GPT uses a Transformer Decoder structure and retains only masked multi-head attention within the Transformer Decoder. T5 is still an encoder-decoder structure consisting of stacked Transformer layers, but the Decoder has a standard attention layer after the self-attention layer. This standard attention layer will incorporate the encoder's output into the attention calculation.

[0120] Let's now go back to describe the third way of obtaining the first target content involved in step 202. In a similar way, the fourth text sequence obtained by concatenating the text to be extracted and the fourth prompt text is input into the question-answering processing model. The fourth prompt text contains information about the first target type. The first target content corresponding to the first target type is obtained from the fourth answer text output by the question-answering processing model.

[0121] For example, a fourth text sequence could be used, such as "Given sentence [S], what is [T1] contained within it?", where the slot corresponding to [S] is filled with the text to be extracted, and the slot corresponding to [T1] can be filled with information of the first target type. The fourth answer text output by the question-answering model could be something like "The T1 contained in the sentence is t1". Here, t1 is the first target content corresponding to the first target type T1 in the text to be extracted.

[0122] The following describes in detail step 206, namely, "inputting the second text sequence obtained by concatenating the text to be extracted, the first answer text, and the second prompt text into the question-and-answer processing model, wherein the second prompt text contains information about the first target content and the third target type; and obtaining the second answer text corresponding to the third target content output by the question-and-answer processing model."

[0123] This step is similar to the previous step, except that when generating the second text sequence, the text to be extracted, the first answer text, and the second prompt text will be concatenated.

[0124] The second prompt text can be a preset second question template. The content of the first target and the information indicating the type of the third target are filled into the corresponding slots in the preset second question template to obtain the second prompt text.

[0125] For example, the template for the second question corresponding to the second prompt text could be: "What is the most likely [T3] for [t1]?"

[0126] In the second question template mentioned above, the slot corresponding to [t1] can be filled with the content of the first target, and the slot corresponding to [T3] can be filled with information of the third target type.

[0127] When concatenating the text to be extracted, the first answer text, and the first prompt text, it can be in the form of "'Given sentence [S]' + first answer text + first prompt text", where the slot corresponding to [S] can be filled with the text to be extracted.

[0128] Continuing from the previous example, the second text sequence could be in the form of "Given a sentence [S], where T1 is t1, what is the most likely [T3] for t1?" The following will use sentiment extraction as a specific application scenario as an example to illustrate this.

[0129] As one feasible approach, the aforementioned second question template can be a template set by human experience. It can be seen that, in addition to the information about the first target content and the third target type, the second question template also includes some context to reflect the semantic relationships between the first target content, the third target type, and the text to be extracted. This context can be set by human experience.

[0130] The question-answering processing model used in step 204 is the same as the question-answering processing model used in step 204, and will not be described in detail here.

[0131] Continuing with the previous example, the second answer text generated by the question-answering processing model can be something like "t1 is very likely to be t3" or "t1's T3 is t3", where t3 is the third target content corresponding to the third target type T3.

[0132] If the desired target content of the desired type has been deduced after this step, such as wanting to extract the third target content from the text to be extracted, then the chain reasoning can be ended and the third target content can be obtained from the third answer text.

[0133] However, in some cases, the desired target content may not yet be obtained through reasoning. In such cases, chain reasoning can continue. This involves inputting a third text sequence, obtained by concatenating the text to be extracted, the first answer text, the second answer text, and the third prompt text, into the question-answering model. The third prompt text contains information about the first target content and the fourth target type. The third answer text, which contains the fourth target content corresponding to the fourth target type, is then obtained from the output of the question-answering model. This process continues in this manner, without exhaustive enumeration.

[0134] The information extraction method provided in this application can be applied to various application scenarios, such as entity relationship extraction, event extraction, sentiment extraction, and so on. Here, sentiment extraction is used as an example to illustrate the above information extraction method.

[0135] Sentiment extraction refers to the process of extracting sentiment elements from a given input text S. These sentiment elements can include sentiment objects, opinion words, and sentiment polarities.

[0136] In this context, the emotion object can include emotion object words, also known as target words, which refer to the specific matter to which the emotion is directed. For example, in the text "This dress doesn't look good," the emotion object is the matter of "dress." In some finer-grained emotion recognition, the emotion object includes not only emotion object words but also aspect words. An aspect word can be understood as an attribute of the emotion object, that is, a certain attribute of the target of the emotion. For example, in the text "This dress is too small," the emotion object is an attribute of the matter, namely, "the size of the dress."

[0137] Opinion words, also known as affective words or evaluative words, refer to words that express feelings, evaluations, or opinions. For example, in the sentence "The apple is so sweet," the affective word is "sweet."

[0138] Emotional polarity refers to the discrete values ​​that classify emotions, such as positive, negative, and neutral. Emotional polarity is usually implicitly expressed; that is, it doesn't appear directly in the sentence but is conveyed through the semantics of the entire text. For example, the emotional polarity of "The apple is so sweet!" is "positive." Similarly, the implicit expression "My phone fits perfectly in my pocket" actually means the phone is a suitable size, and its emotional polarity should also be "positive."

[0139] Because the implicit expression of sentiment polarity often places very high demands on semantic understanding, it relies on a deep understanding of the semantics within the text to infer the true intent behind it. Existing technologies, whether dictionary-plus-rule methods or deep learning approaches, struggle to accurately extract sentiment polarity. Therefore, the chain-like reasoning method based on cue learning provided in this application can be adopted.

[0140] If fine-grained emotion recognition is used, the specific process can be as follows: Figure 4 As shown, the steps include:

[0141] Step 402: Obtain the text to be extracted and the sentiment object words in the text to be extracted.

[0142] In this embodiment, the text to be extracted can be text obtained from a user device. For example, if a user wants to extract sentiment from a text, the user can upload the text to an information extraction device on the server through the user device, and the information extraction device will use the text as the text to be extracted.

[0143] The text to be extracted in this embodiment can also be text obtained from a database. For example, if sentiment extraction is required from a batch of text in a database, each text obtained from the database can be used as the text to be extracted.

[0144] The text to be extracted can be a sentence, a paragraph composed of sentences, a document, or even a smaller unit of text, such as a phrase. In this embodiment, a sentence is used as an example.

[0145] When extracting sentiment object words from the text to be extracted, the following methods can be used, but are not limited to:

[0146] The first method: Obtain the sentiment object words specified by the user.

[0147] When performing sentiment extraction, the sentiment object words can be specified by the user. This is equivalent to the user specifying the sentiment polarity they want to extract from the text for a specific sentiment object, and they can input the sentiment object words from the text to be extracted. For example, for the text to be extracted, "My phone can just fit in my pocket," the user can specify the sentiment object word "phone."

[0148] The second approach is to use an information extraction model to extract sentiment words from the text to be extracted.

[0149] Since sentiment words are usually explicit and relatively easy to understand in text, existing information extraction models are quite accurate in identifying sentiment words. Therefore, existing information extraction models (i.e., event extraction models) can be used to extract sentiment words from the text to be extracted.

[0150] The third approach involves inputting the fourth text sequence, obtained by concatenating the text to be extracted and the fourth prompt text, into the question-and-answer processing model. The fourth prompt text contains information indicating the sentiment object, and the sentiment object words are extracted from the fourth answer text output by the question-and-answer processing model.

[0151] For example, input the following fourth text sequence into the question-answering processing model:

[0152] Given the sentence "My phone can just fit in my pocket," what is the emotional object word it contains?

[0153] For this fourth text sequence, the question-answering model can generate a fourth answer text such as:

[0154] "The emotional object word contained in the sentence is 'mobile phone'."

[0155] Step 404: Input the first text sequence obtained by concatenating the text to be extracted and the first prompt text into the question-answering processing model. The first prompt text contains sentiment object words and indicative information; obtain the first answer text containing aspect words output by the question-answering processing model.

[0156] Continuing with the previous example, input the following first text sequence into the question-answering processing model:

[0157] Given the sentence 'My phone fits perfectly into my pocket,' which specific aspect of the phone might be mentioned?

[0158] For this first text sequence, the question-answering processing model can generate the first answer text, which could be something like:

[0159] The specific aspect of the phone mentioned in the sentence is its size or portability.

[0160] It should be noted that if coarse-grained sentiment extraction is used, aspect words can be omitted, i.e., this step can be skipped. Instead, in the next step, the text to be extracted is directly concatenated with the second prompt text containing type information such as sentiment words, object words, and indicative opinion words to obtain the second text sequence, which is then input into the processing model.

[0161] Step 406: Input the second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, into the question-and-answer processing model. The second prompt text contains type information of sentiment object words and opinion words. Obtain the second answer text containing opinion words output by the question-and-answer processing model.

[0162] Continuing with the previous example, input the following second text sequence into the question-answering processing model:

[0163] Given the sentence "My phone fits just into my pocket," the specific aspect of the phone mentioned in the sentence is its size or portability. What are the evaluative words for this specific aspect of the phone, and why?

[0164] For this second text sequence, the question-answering processing model can generate a second answer text such as:

[0165] "Regarding the specific aspect of the phone, the sentence means that the phone is small enough to fit perfectly in the speaker's pocket. Generally speaking, the speaker is expressing the point that it is very portable because it is easy for the speaker to carry the phone and put it in their pocket, and they find the phone convenient and easy to use."

[0166] Step 408: Input the third text sequence, which is obtained by concatenating the text to be extracted, the first answer text, the second answer text, and the third prompt text, into the question-and-answer processing model. The third prompt text contains sentiment object words and information indicating sentiment polarity; obtain the third answer text containing sentiment polarity output by the question-and-answer processing model.

[0167] Continuing with the previous example, input the following third text sequence into the question-answering processing model:

[0168] "Given the sentence 'My phone fits perfectly in my pocket,' the specific aspect of the phone mentioned in the sentence is its size or portability. Regarding this specific aspect of the phone, the sentence means that the phone is small enough to fit comfortably in the speaker's pocket. Generally, the speaker is expressing the view that it is very portable because it is easy for the speaker to carry and put in their pocket, and they find the phone convenient and easy to use. Based on this viewpoint, what is the emotional polarity regarding this phone?"

[0169] For this third text sequence, the question-answering model can generate a third answer text such as:

[0170] "The emotional polarity regarding this phone is positive. The speaker finds the phone convenient to carry and use, expressing their liking for it."

[0171] Step 410: Obtain the sentiment polarity from the third answer text.

[0172] Continuing with the previous example, the sentiment polarity can be determined as "positive" from the third answer text. This sentiment polarity information can be fed back to the user, or it can be associated with the extracted text and stored for later querying or use.

[0173] As can be seen from the above process, this application first uses a question-and-answer processing model based on a prompting mechanism to infer the aspect words of the emotional object targeted by the sentence. Then, it further uses the question-and-answer processing model based on a prompting mechanism to infer the opinion words related to that aspect word. Finally, it uses the question-and-answer processing model based on a prompting mechanism to infer the emotional polarity. By using three progressive inference steps, the understanding ability of the language model is fully utilized to gradually understand the deep semantics of the sentence, thereby accurately extracting the implicitly expressed emotional polarity.

[0174] The question-answering processing model used in this embodiment is based on a pre-trained language model. As one possible approach, the question-answering processing model can employ existing question-answering processing models.

[0175] As another feasible approach, question-answering models can be fine-tuned using a small amount of sentiment-related text, building upon existing models. Training can employ the MLM (Masked Language Model) approach commonly used in pre-trained language models. This involves masking a portion of the tokens in the text sample to construct a prompt text that asks a question about the content of that token. The text sample and the prompt text are concatenated and used as the input sequence to train the question-answering model. The model answers the question, and the resulting answer text contains the masked portion. The training objective is to minimize the difference between the masked portion obtained by the question-answering model and the actual masked portion in the text sample.

[0176] As another feasible approach, the question-answering processing model can be trained on top of a pre-trained language model using training data specifically designed for information extraction. This training process can be as follows: Figure 5 As shown, the training process can be performed by Figure 1 The model training device in the system architecture shown is executed. For example... Figure 5 As shown, the method may include the following steps:

[0177] Step 502: Obtain training data including multiple training samples. The training samples include text samples and the third target content labels corresponding to the third target type labeled on the text samples.

[0178] The training samples obtained in this step can be manually labeled, that is, manually identify the third target content corresponding to the third target type in the text sample and label it. For example, in the case of sentiment extraction, sentiment polarity labels are labeled for the text sample.

[0179] Furthermore, the training samples may also include second target content labels corresponding to the second target type labeled on the text samples. For example, in the case of sentiment extraction, the text samples may be labeled with evaluation word labels and / or aspect word labels.

[0180] Step 504: Train a question-answering processing model based on a pre-trained language model using training data. This includes: obtaining the first target content corresponding to the first target type from the text samples; inputting the first text sequence obtained by concatenating the text samples and the first prompt text into the question-answering processing model, where the first prompt text contains information about the first target content and the second target type; obtaining the first answer text output by the question-answering processing model, which contains the second target content corresponding to the second target type; inputting the second text sequence obtained by concatenating the text samples, the first answer text, and the second prompt text into the question-answering processing model, where the second prompt text contains information about the first target content and the third target type; obtaining the second answer text output by the question-answering processing model, which contains the third target content corresponding to the third target type; the training objective includes minimizing the difference between the third target content contained in the second answer text and the corresponding third target content label.

[0181] Furthermore, the training objective mentioned above can also include minimizing the difference between the second target content contained in the first answer text and the corresponding second target content label.

[0182] As one possible approach, the question-answering model can construct a loss function using the training objectives described above, taking the parameters of the pre-trained language model as initial parameters. In each iteration, the model parameters are updated using the loss function value and methods such as gradient descent, until a preset training termination condition is met. This termination condition may include, for example, the loss function value being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0183] For example, the value of the loss function Loss, which can be constructed based on the training objective, can be obtained from the value of the first loss function Loss1 and the value of the second loss function Loss2. For example:

[0184] Loss = αLoss1 × βLoss2

[0185] Here, α and β are preset weighting coefficients. The value of the first loss function, Loss1, can be obtained based on the difference between the second target content contained in the first answer text and the corresponding second target content label. The value of the second loss function, Loss2, can be obtained based on the difference between the third target content contained in the second answer text and the corresponding third target content label. Both Loss1 and Loss2 can be implemented using functions such as cross-entropy loss. This loss function, Loss, can comprehensively consider the accuracy of both the second and third target content.

[0186] The first training termination condition mentioned above can be, for example, the value of the loss function being less than or equal to a preset first loss function threshold, or the number of iterations reaching a preset first iteration threshold, etc.

[0187] The above describes embodiments of the methods provided in this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0188] According to another embodiment, an information extraction device is provided. Figure 6 A schematic block diagram of an information extraction apparatus according to one embodiment is shown. Figure 6 As shown, the device 600 includes: a text acquisition unit 601, a first target acquisition unit 602, a second target acquisition unit 603, and a third target acquisition unit 604, and may further include a fourth target acquisition unit 605. The main functions of each component are as follows:

[0189] The text acquisition unit 601 is configured to acquire the text to be extracted.

[0190] The first target acquisition unit 602 is configured to acquire the first target content corresponding to the first target type in the text to be extracted.

[0191] The second target acquisition unit 603 is configured to input a first text sequence obtained by concatenating the text to be extracted and the first prompt text into the question-and-answer processing model, wherein the first prompt text contains information about the first target content and the second target type; and to acquire the first answer text output by the question-and-answer processing model, which contains the second target content corresponding to the second target type.

[0192] The third target acquisition unit 604 is configured to input a second text sequence obtained by concatenating the text to be extracted, the first answer text, and the second prompt text into the question-and-answer processing model, wherein the second prompt text contains information about the first target content and the third target type; and to acquire the second answer text output by the question-and-answer processing model, which contains the third target content corresponding to the third target type.

[0193] The question-answering model mentioned above is obtained by pre-training a pre-trained language model.

[0194] The first target acquisition unit 602 may acquire the first target content corresponding to the first target type in the text to be extracted using, but not limited to, the following methods:

[0195] The first method: Obtain the first target content specified by the user.

[0196] The second approach is to use an information extraction model to extract the first target content corresponding to the first target type from the text to be extracted.

[0197] The third method involves inputting the fourth text sequence, obtained by concatenating the text to be extracted and the fourth prompt text, into the question-answering processing model. The fourth prompt text contains information about the first target type. The first target content corresponding to the first target type is then obtained from the fourth answer text output by the question-answering processing model.

[0198] Furthermore, the fourth target acquisition unit 605 can be configured to input a third text sequence obtained by concatenating the text to be extracted, the first answer text, the second answer text, and the third prompt text into the question-and-answer processing model, wherein the third prompt text contains information about the first target content and the fourth target type; and acquire the third answer text output by the question-and-answer processing model, which contains the fourth target content corresponding to the fourth target type.

[0199] As one possible approach, the first prompt text is obtained by filling the first target content and the second target type information into the corresponding slots in the preset first question template.

[0200] The second prompt text is obtained by filling the information of the first target content and the third target type into the corresponding slots in the preset second question template.

[0201] The question-answering processing model described above is pre-trained based on a pre-trained language model. The relevant principles and structure of the question-answering processing model can be found in the descriptions in the method embodiments, and will not be elaborated upon here.

[0202] According to another embodiment, an apparatus for training a question-answering processing model is provided, the apparatus corresponding to Figure 1 The model training device in the system architecture shown. Figure 7A schematic block diagram of an apparatus for training an information question-answering processing model provided in an embodiment of this application. Figure 7 As shown, the device 700 includes a sample acquisition unit 701 and a model training unit 702. The main functions of each component are as follows:

[0203] The sample acquisition unit 701 is configured to acquire training data including multiple training samples, the training samples including text samples and third target content labels corresponding to the third target type labeled on the text samples;

[0204] The model training unit 702 is configured to train a question-answering processing model based on a pre-trained language model using training data. Specifically, it acquires the first target content corresponding to a first target type from text samples; inputs a first text sequence obtained by concatenating text samples and first prompt text into the question-answering processing model, where the first prompt text contains information about the first target content and the second target type; acquires the first answer text output by the question-answering processing model, containing the second target content corresponding to the second target type; inputs a second text sequence obtained by concatenating text samples, the first answer text, and the second prompt text into the question-answering processing model, where the second prompt text contains information about the first target content and the third target type; and acquires the second answer text output by the question-answering processing model, containing the third target content corresponding to the third target type. The training objective includes minimizing the difference between the third target content contained in the second answer text and the corresponding third target content label.

[0205] Furthermore, the training samples also include second target content labels corresponding to the second target type annotated in the text samples.

[0206] Accordingly, the training objective also includes minimizing the difference between the second target content contained in the first answer text and the corresponding second target content label.

[0207] As one possible approach, the first prompt text is obtained by filling the first target content and the second target type information into the corresponding slots in the preset first question template.

[0208] The second prompt text is obtained by filling the information of the first target content and the third target type into the corresponding slots in the preset second question template.

[0209] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0210] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0211] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0212] And an electronic device, comprising:

[0213] One or more processors; and

[0214] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0215] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0216] in, Figure 8An exemplary architecture of an electronic device is shown, which may include a processor 810, a video display adapter 811, a disk drive 812, an input / output interface 813, a network interface 814, and a memory 820. The processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, and memory 820 can communicate with each other via a communication bus 830.

[0217] The processor 810 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0218] The memory 820 can be implemented in the form of ROM (Read-Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 820 can store the operating system 821 for controlling the operation of the electronic device 800, and the basic input / output system (BIOS) 822 for controlling the low-level operations of the electronic device 800. Additionally, it can store a web browser 823, a data storage management system 824, and an information extraction device / model training device 825, etc. The aforementioned information extraction device / model training device 825 can be the application program that specifically implements the aforementioned steps in the embodiments of this application. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 820 and is called and executed by the processor 810.

[0219] The input / output interface 813 is used to connect input / output modules to enable information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0220] Network interface 814 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0221] Bus 830 includes a pathway for transmitting information between various components of the device, such as processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, and memory 820.

[0222] It should be noted that although the above-described device only shows the processor 810, video display adapter 811, disk drive 812, input / output interface 813, network interface 814, memory 820, bus 830, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0223] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0224] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for information extraction, characterized in that, The method includes: Obtain the text to be extracted and the first target content corresponding to the first target type in the text to be extracted; The first text sequence, obtained by concatenating the text to be extracted and the first prompt text, is input into the question-and-answer processing model. The first prompt text contains information about the first target content and the second target type. The first answer text, which contains the second target content corresponding to the second target type, is obtained from the output of the question-and-answer processing model. The first prompt text is obtained by filling the first target content and the second target type information into the corresponding slots in a preset first question template. The second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, is input into the question-and-answer processing model. The second prompt text contains information about the first target content and the third target type. The second answer text, which contains the third target content corresponding to the third target type, is obtained from the output of the question-and-answer processing model. The second prompt text is obtained by filling the first target content and the third target type information into the corresponding slots in a preset second question template. The question-answering processing model mentioned above is pre-trained based on a pre-trained language model.

2. The method according to claim 1, characterized in that, The first target content corresponding to the first target type in the text to be extracted is obtained in the following way: Retrieve the first target content specified by the user; or, The first target content corresponding to the first target type is extracted from the text to be extracted using an information extraction model; or, The fourth text sequence, obtained by concatenating the text to be extracted and the fourth prompt text, is input into the question-and-answer processing model. The fourth prompt text contains information about the first target type. The first target content corresponding to the first target type is obtained from the fourth answer text output by the question-and-answer processing model.

3. The method according to claim 1, characterized in that, The method further includes: The third text sequence, obtained by concatenating the text to be extracted, the first answer text, the second answer text, and the third prompt text, is input into the question-and-answer processing model. The third prompt text contains information about the first target content and the fourth target type. The third answer text, which contains the fourth target content corresponding to the fourth target type, is then obtained from the output of the question-and-answer processing model.

4. A method for information extraction, wherein the method is executed by a cloud server, characterized in that, The method includes: Obtain the text to be extracted from the user terminal, and obtain the first target content corresponding to the first target type in the text to be extracted; The first text sequence, obtained by concatenating the text to be extracted and the first prompt text, is input into the question-and-answer processing model. The first prompt text contains information about the first target content and the second target type. The first answer text, which contains the second target content corresponding to the second target type, is obtained from the output of the question-and-answer processing model. The first prompt text is obtained by filling the first target content and the second target type information into the corresponding slots in a preset first question template. The second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, is input into the question-and-answer processing model. The second prompt text contains information about the first target content and the third target type. The second answer text, which contains the third target content corresponding to the third target type, is obtained from the output of the question-and-answer processing model. The second prompt text is obtained by filling the first target content and the third target type information into the corresponding slots in a preset second question template. The extraction result is generated using the third target content, and the extraction result is returned to the user terminal.

5. An emotion extraction method, characterized in that, The method includes: Obtain the text to be extracted and the sentiment object words in the text to be extracted; The first text sequence, obtained by concatenating the text to be extracted and the first prompt text, is input into the question-and-answer processing model. The first prompt text contains information about the sentiment object words and the opinion words. The first answer text, which contains opinion words, is output by the question-and-answer processing model. The first prompt text is obtained by filling the information about the sentiment object words and the opinion words into the corresponding slots in a preset first question template. The second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, is input into the question-and-answer processing model. The second prompt text contains the sentiment object words and information indicating the sentiment polarity. The second answer text containing the sentiment polarity is obtained from the output of the question-and-answer processing model. The second prompt text is obtained by filling the sentiment object words and the information indicating the sentiment polarity into the corresponding slots in the preset second question template. The emotional polarity is obtained from the second answer text.

6. An emotion extraction method, characterized in that, The method includes: Obtain the text to be extracted and the sentiment object words in the text to be extracted; The first text sequence, obtained by concatenating the text to be extracted and the first prompt text, is input into the question-and-answer processing model. The first prompt text contains the sentiment object words and information on the indicative aspect. The first answer text, which contains the indicative aspect words, is obtained from the output of the question-and-answer processing model. The first prompt text is obtained by filling the sentiment object words and information on the indicative aspect into the corresponding slots in the preset first question template. The second text sequence, obtained by concatenating the text to be extracted, the first answer text, and the second prompt text, is input into the question-and-answer processing model. The second prompt text contains type information of the sentiment object words and the opinion words. The second answer text containing opinion words is obtained from the output of the question-and-answer processing model. The second prompt text is obtained by filling the information of the sentiment object words and the opinion words into the corresponding slots in the preset second question template. The third text sequence, obtained by concatenating the text to be extracted, the first answer text, the second answer text, and the third prompt text, is input into the question-and-answer processing model. The third prompt text contains the sentiment object words and information indicating sentiment polarity. The third answer text containing sentiment polarity is then obtained from the output of the question-and-answer processing model. The emotional polarity is obtained from the third answer text.

7. A method for training a question-answering processing model, characterized in that, The method includes: Acquire training data including multiple training samples, wherein the training samples include text samples and third target content labels corresponding to the third target type labeled on the text samples; A question-answering processing model based on a pre-trained language model is trained using the training data. The process includes: obtaining the first target content corresponding to a first target type from the text samples; inputting a first text sequence obtained by concatenating the text samples and a first prompt text into the question-answering processing model, where the first prompt text contains information about the first target content and the second target type; obtaining a first answer text output by the question-answering processing model that contains the second target content corresponding to the second target type; inputting a second text sequence obtained by concatenating the text samples, the first answer text, and the second prompt text into the question-answering processing model, where the second prompt text contains information about the first target content and the third target type; and obtaining a second answer text output by the question-answering processing model that contains the third target content corresponding to the third target type. The first prompt text is obtained by filling the first target content and the second target type information into the corresponding slots of a preset first question template, and the second prompt text is obtained by filling the first target content and the third target type information into the corresponding slots of a preset second question template. The training objective includes minimizing the difference between the third target content contained in the second answer text and the corresponding third target content label.

8. The method according to claim 7, characterized in that, The training samples also include second target content tags corresponding to the second target type labeled in the text samples; The training objective also includes minimizing the difference between the second target content contained in the first answer text and the corresponding second target content label.

9. The method according to claim 7, characterized in that, The first prompt text is obtained by filling the information of the first target content and the second target type into the corresponding slots of the preset first question template; The second prompt text is obtained by filling the information of the first target content and the third target type into the corresponding slots of the preset second question template.

10. An information extraction device, characterized in that, The device includes: The text acquisition unit is configured to acquire the text to be extracted. The first target acquisition unit is configured to acquire the first target content corresponding to the first target type in the text to be extracted; The second target acquisition unit is configured to input a first text sequence obtained by concatenating the text to be extracted and the first prompt text into a question-and-answer processing model, wherein the first prompt text contains information about a first target content and a second target type; and to acquire a first answer text output by the question-and-answer processing model that contains the second target content corresponding to the second target type; wherein the first prompt text is obtained by filling the first target content and the information about the second target type into the corresponding slots in a preset first question template; The third target acquisition unit is configured to input a second text sequence obtained by concatenating the text to be extracted, the first answer text, and the second prompt text into the question-and-answer processing model, wherein the second prompt text contains information about the first target content and the third target type; and to acquire the second answer text output by the question-and-answer processing model, which contains the third target content corresponding to the third target type; wherein the second prompt text is obtained by filling the first target content and the information about the third target type into the corresponding slots in a preset second question template; The question-answering processing model mentioned above is pre-trained based on a pre-trained language model.

11. An apparatus for training a question-answering processing model, characterized in that, The device includes: The sample acquisition unit is configured to acquire training data including multiple training samples, wherein the training samples include text samples and third target content tags corresponding to the third target type labeled on the text samples; The model training unit is configured to train a question-answering processing model based on a pre-trained language model using the training data. Specifically, it acquires first target content corresponding to a first target type from the text samples; inputs a first text sequence obtained by concatenating the text samples and a first prompt text into the question-answering processing model, where the first prompt text contains information about the first target content and the second target type; acquires a first answer text output by the question-answering processing model containing second target content corresponding to the second target type; inputs a second text sequence obtained by concatenating the text samples, the first answer text, and the second prompt text into the question-answering processing model, where the second prompt text contains information about the first target content and a third target type; and acquires a second answer text output by the question-answering processing model containing third target content corresponding to the third target type. The training objective includes minimizing the difference between the third target content contained in the second answer text and the corresponding third target content label. The first prompt text is obtained by filling the first target content and the second target type information into the corresponding slots of a preset first question template; the second prompt text is obtained by filling the first target content and the third target type information into the corresponding slots of a preset second question template.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the steps of the method described in any one of claims 1 to 9.

13. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 9.

Images