Information extraction method and device, storage medium and electronic equipment

Abstract

The invention discloses an information extraction method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a to-be-extracted text, and determining a plurality of target extraction categories; and for each target extraction category, determining an identifier corresponding to the target extraction category according to a preset corresponding relationship, and taking the identifier as prompt information. The prompt information and the to-be-extracted text are input into an information extraction model, an extraction result of the to-be-extracted text output by the information extraction model is obtained, and the information extraction model is obtained through training of the identifier determined through the preset corresponding relation. Due to the fact that the number of the extraction categories needing to be extracted is large, the length of the prompt information is too long, and therefore the length of the prompt information is reduced by converting the extraction categories into the identifiers. Besides, the information extraction model is also obtained by training the identifier, so that the identifier is input into the information extraction model as the prompt information, and the information of the extraction category corresponding to the identifier can be extracted from the to-be-extracted text.

Description

Technical Field

[0001] The present invention relates to the field of computers, and more particularly to an information extraction method, device, storage medium and electronic device. Background Art

[0002] Information Extraction (IE) is an important application direction in natural language processing. It focuses on identifying and extracting valuable, structured information from unstructured or semi-structured text data, such as extracting entities in the text, determining the relationships, events, emotions, etc. between entities. This information may include user privacy information.

[0003] At present, using information extraction models for extraction is a relatively efficient extraction method. Generally speaking, when extracting, first define the information categories that need to be extracted, such as people, organizations, geographic locations, etc., as prompt information, and then input the prompt information and the text to be extracted into the information extraction model to obtain the information extraction result. However, when there are many categories to be extracted, the text length of the prompt information is long, which will cause the length of the text input into the information extraction model to exceed the input length limit of the information extraction model, and the information extraction cannot be completed. If the prompt information is divided into several subsets and extracted separately, the extraction efficiency will be reduced.

[0004] Based on this, this specification provides an information extraction method. Summary of the invention

[0005] This specification provides an information extraction method, device, storage medium and electronic device to at least partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides an information extraction method, including:

[0008] Get the text to be extracted and determine several target extraction categories;

[0009] For each target extraction category, according to a preset corresponding relationship, an identifier corresponding to the target extraction category is determined as prompt information;

[0010] The prompt information and the text to be extracted are input into a pre-trained information extraction model to obtain an extraction result of the text to be extracted output by the information extraction model, wherein the information extraction model is trained by an identifier determined by the preset corresponding relationship.

[0011] Optionally, the information extraction model includes an encoder and a decoder;

[0012] Inputting the prompt information and the text to be extracted into a pre-trained information extraction model to obtain an extraction result of the text to be extracted output by the information extraction model, specifically including:

[0013] Inputting the prompt information and the text to be extracted into an encoder in a pre-trained information extraction model, the encoder encodes the prompt information and the text to be extracted to obtain an encoding result of the text to be extracted;

[0014] The encoding result of the text to be extracted is input into the decoder in the information extraction model, and the decoder determines the extraction result of the text to be extracted based on a replication mechanism.

[0015] Optionally, the encoder includes a text encoder and a prompt encoder;

[0016] Inputting the prompt information and the text to be extracted into an encoder in a pre-trained information extraction model, the encoder encodes the prompt information and the text to be extracted to obtain an encoding result of the text to be extracted, specifically including:

[0017] Inputting the prompt information into a prompt encoder in a pre-trained information extraction model, wherein the prompt encoder encodes the prompt information to obtain a prompt encoding result; and

[0018] Inputting the text to be extracted into a text encoder in a pre-trained information extraction model, wherein the text encoder encodes the text to be extracted to obtain a text encoding result;

[0019] The prompt encoding result and the text encoding result are determined as the encoding result of the text to be extracted.

[0020] Optionally, the text encoder encodes the text to be extracted to obtain a text encoding result, which specifically includes:

[0021] The text encoder performs word segmentation on the text to be extracted to obtain a word segmentation result including a plurality of text word segmentations;

[0022] For each text segmentation, determine the segmentation vector of the text segmentation as the text encoding result.

[0023] Optionally, the encoding result of the text to be extracted is input into a decoder in the information extraction model, and the decoder determines the extraction result of the text to be extracted based on a replication mechanism, specifically including:

[0024] By using the copy mode in the copy mechanism, according to the encoding result and the prompt information, the probability of the extraction category to which each text segmentation in the to-be-extracted text belongs is determined as the copy probability;

[0025] For each text segmentation, determine a target replication probability among the replication probabilities of the text segmentation;

[0026] Copy the identifier corresponding to the extraction category to which the target copy probability belongs, and obtain a decoding result, wherein the decoding result includes each text segmentation in the text to be extracted and the identifier corresponding to each text segmentation;

[0027] According to the decoding result, an extraction result of the text to be extracted is determined.

[0028] Optionally, determining the extraction result of the text to be extracted according to the decoding result specifically includes:

[0029] Determining, according to the preset corresponding relationship, an extraction category corresponding to each identifier in the decoding result;

[0030] The extraction category corresponding to each identifier and each text segment in the text to be extracted are determined as the extraction result of the text to be extracted.

[0031] Optionally, training an information extraction model includes:

[0032] Obtaining a sample text to be extracted and determining a label of the sample extracted text;

[0033] Determine a number of categories to be extracted, and determine identifiers corresponding to the categories to be extracted as sample prompt information;

[0034] Input the sample prompt information and the sample text to be extracted into an information extraction model to obtain a predicted extraction result of the sample text to be extracted output by the information extraction model;

[0035] The information extraction model is trained according to the predicted extraction result and the label.

[0036] Optionally, determining a label of the sample extracted text specifically includes:

[0037] Determining a sample extraction category for each text segmentation in the sample extraction text;

[0038] For each sample extraction category, determine a sample identifier corresponding to the sample extraction category according to a preset corresponding relationship;

[0039] Each text segmentation in the sample extracted text and a sample identifier corresponding to the text segmentation are determined as a label of the sample extracted text.

[0040] This specification provides an information extraction device, the device comprising:

[0041] The input acquisition module is used to obtain the text to be extracted and determine several target extraction categories;

[0042] A prompt information determination module, used to determine, for each target extraction category, an identifier corresponding to the target extraction category according to a preset corresponding relationship as prompt information;

[0043] The extraction module is used to input the prompt information and the text to be extracted into a pre-trained information extraction model to obtain the extraction result of the text to be extracted output by the information extraction model. The information extraction model is trained by the identifier determined by the preset corresponding relationship.

[0044] Optionally, the information extraction model includes an encoder and a decoder;

[0045] The extraction module is specifically used to input the prompt information and the text to be extracted into an encoder in a pre-trained information extraction model, and the encoder encodes the prompt information and the text to be extracted to obtain the encoding result of the text to be extracted; input the encoding result of the text to be extracted into a decoder in the information extraction model, and the decoder determines the extraction result of the text to be extracted based on a copy mechanism.

[0046] Optionally, the encoder includes a text encoder and a prompt encoder;

[0047] The extraction module is specifically used to input the prompt information into a prompt encoder in a pre-trained information extraction model, and the prompt encoder encodes the prompt information to obtain a prompt encoding result; and input the text to be extracted into a text encoder in a pre-trained information extraction model, and the text encoder encodes the text to be extracted to obtain a text encoding result; and determine the prompt encoding result and the text encoding result as the encoding result of the text to be extracted.

[0048] Optionally, the extraction module is specifically used for the text encoder to segment the text to be extracted to obtain a segmentation result including several text segmentations; for each text segmentation, a segmentation vector of the text segmentation is determined as a text encoding result.

[0049] Optionally, the extraction module is specifically used to determine the probability of the extraction category to which each text segmentation in the text to be extracted belongs as the copy probability according to the encoding result and the prompt information through the copy mode in the copy mechanism; for each text segmentation, determine the target copy probability among the various copy probabilities of the text segmentation; copy the identifier corresponding to the extraction category to which the target copy probability belongs to obtain a decoding result, the decoding result including each text segmentation in the text to be extracted and the identifier corresponding to each text segmentation; and determine the extraction result of the text to be extracted based on the decoding result.

[0050] Optionally, the extraction module is specifically used to determine the extraction category corresponding to each identifier in the decoding result according to the preset corresponding relationship; and determine the extraction category corresponding to each identifier and each text segment in the text to be extracted as the extraction result of the text to be extracted.

[0051] Optionally, the device further comprises:

[0052] The training module is used to obtain sample text to be extracted and determine the label of the sample extracted text; determine a number of categories to be extracted and determine the identifiers corresponding to the categories to be extracted as sample prompt information; input the sample prompt information and the sample text to be extracted into the information extraction model to obtain the predicted extraction result of the sample text to be extracted output by the information extraction model; and train the information extraction model according to the predicted extraction result and the label.

[0053] Optionally, the training module is specifically used to determine the sample extraction category of each text segmentation in the sample extraction text; for each sample extraction category, determine the sample identifier corresponding to the sample extraction category according to a preset correspondence; and determine each text segmentation in the sample extraction text and the sample identifier corresponding to the text segmentation as the label of the sample extraction text.

[0054] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned information extraction method is implemented.

[0055] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned information extraction method when executing the program.

[0056] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0057] In the information extraction method provided in this specification, a text to be extracted is obtained, and several target extraction categories are determined. For each target extraction category, an identifier corresponding to the target extraction category is determined according to a preset corresponding relationship as prompt information. The prompt information and the text to be extracted are input into an information extraction model to obtain an extraction result of the text to be extracted output by the information extraction model, and the information extraction model is trained by the identifier determined by the preset corresponding relationship.

[0058] It can be seen from the above method that since there are many extraction categories to be extracted, the length of the prompt information is too long. Therefore, this method reduces the length of the prompt information by converting the extraction categories into identifiers. In addition, the information extraction model is also trained by identifiers. Therefore, by inputting the identifier as prompt information into the information extraction model, the information of the extraction category corresponding to the identifier can be extracted from the text to be extracted. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The exemplary embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification.

[0060] In the figure:

[0061] Figure 1 A flowchart of an information extraction method provided in this specification;

[0062] Figure 2 A character conversion diagram provided for this manual;

[0063] Figure 3 A schematic diagram of the structure of the information extraction model provided in this manual;

[0064] Figure 4 A schematic diagram of the internal structure of the information extraction model provided in this manual;

[0065] Figure 5 A schematic diagram of an information extraction device provided in this specification;

[0066] Figure 6 The corresponding Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0069] Figure 1 A flowchart of an information extraction method provided in this specification specifically includes the following steps:

[0070] S100: Obtain the text to be extracted and determine several target extraction categories.

[0071] When extracting information from a text, the text to be extracted and the prompt information including the target extraction category are usually input into the information extraction model together. When there are many extraction categories to be extracted, that is, in the extreme label scenario, the length of the text input into the information extraction model is too long and will exceed the input length limit of the information extraction model, and the information extraction cannot be completed. Therefore, this specification provides an information extraction method to reduce the length of the prompt information so that the information extraction model can extract information from the text to be extracted and ensure the efficiency of information extraction. The execution subject of this specification can be a computing device deployed with an information extraction model for information extraction, such as a server, or a device such as a desktop computer, a laptop computer, etc., and this specification does not limit this. For the convenience of description, the information extraction method provided in this specification is described below with the server as the execution subject only.

[0072] First, the server obtains the text to be extracted and determines several target extraction categories. The target extraction category refers to the category of information that the information extraction model needs to extract when extracting information from the text to be extracted, such as names of people, places, events, emotions, etc. The target extraction category is used to prompt the information extraction model which categories of information the model needs to extract. The target extraction category is the category that the information extraction model can extract. For example, after training the information extraction model, the categories that the information extraction model can extract include a, b, c, and d, and the categories that cannot be extracted include f and h. Then, the target extraction category can be any one or more of a, b, c, and d, and cannot include any one or more of f and h. Among them, the server can train the information extraction model as needed so that the information extraction model has the ability to extract the extraction categories that need to be extracted.

[0073] In addition, the target extraction category may not be included in the text to be extracted. For example, the target extraction categories include a and b, and the extraction categories included in the text to be extracted are f and h. Then, the extraction result obtained by the information extraction model from the text to be extracted may be that the target extraction category information is not extracted.

[0074] S102: For each target extraction category, according to a preset corresponding relationship, determine an identifier corresponding to the target extraction category as prompt information.

[0075] In one or more embodiments of the present specification, due to the large number of target extraction categories, the length of the input information extraction model is too long and may exceed the input length limit of the information extraction model. Therefore, the server can determine the identifier corresponding to the target extraction category for each target extraction category according to the preset correspondence relationship as prompt information. The preset correspondence relationship can be set as needed, but the preset correspondence relationship used when training the information extraction model and the preset relationship when the information extraction model training is completed and applied need to be consistent. For example, for the extraction category of person, the preset correspondence relationship used when training the information extraction model is that the identifier corresponding to person is A. Then, when applying the information extraction model, if the target extraction category is person, the identifier of the input information extraction model obtained according to the preset correspondence relationship is A. When the target extraction category is converted into an identifier, the length of the prompt information is greatly reduced, thereby reducing the length of the input information extraction model.

[0076] Figure 2 The character conversion diagram provided for this manual is as follows: Figure 2 As shown, for the extraction category of person, the number of characters in the extraction category itself is 6. If the identifier corresponding to person is A in the preset correspondence, then the number of characters in the extraction category becomes 1. When inputting the information extraction model, the character length is shortened by 5.

[0077] It should be noted that the length in this specification generally refers to the number of characters in the text, and characters include letters, numbers, punctuation marks, spaces, etc. It can also refer to the length of the token (text segmentation) obtained by converting the text to be extracted after the text to be extracted is input into the information extraction model, and it can also refer to the length of other data forms that can be understood by the information extraction model after the text to be extracted is converted, etc. This specification does not limit this, but the lengths of different data forms can be converted to each other. For example, the number of characters in the text and the length of the text segmentation can be converted to each other, and the specific conversion relationship can be set as needed. Specifically, one or more characters can form a text segmentation, and a text segmentation may also be composed of multiple characters, forming a special case of text segmentation merging, etc.

[0078] S104: Inputting the prompt information and the text to be extracted into a pre-trained information extraction model to obtain an extraction result of the text to be extracted output by the information extraction model, wherein the information extraction model is trained by the identifier determined by the preset corresponding relationship.

[0079] Since the information extraction model is trained by using identifiers determined by preset corresponding relationships, the information extraction model can extract information from the text to be extracted based on prompt information including the identifiers.

[0080] Specifically, Figure 3 The structural diagram of the information extraction model provided in this manual is as follows: Figure 3 As shown, the information extraction model includes an encoder and a decoder. Then, the server can input the prompt information and the text to be extracted into the encoder of the pre-trained information extraction model, and the encoder encodes the prompt information and the text to be extracted to obtain the encoding result of the text to be extracted, and inputs the encoding result of the text to be extracted into the decoder of the information extraction model. The decoder determines the extraction result of the text to be extracted based on the copy mechanism.

[0081] based on Figure 1 The information extraction method shown obtains the text to be extracted and determines several target extraction categories. For each target extraction category, the identifier corresponding to the target extraction category is determined according to the preset corresponding relationship as prompt information. The prompt information and the text to be extracted are input into the information extraction model to obtain the extraction result of the text to be extracted output by the information extraction model, and the information extraction model is trained by the identifier determined by the preset corresponding relationship. Since there are many extraction categories to be extracted, the length of the prompt information is too long. Therefore, the method reduces the length of the prompt information by converting the extraction category into an identifier. In addition, the information extraction model is also obtained by identifier training. Therefore, the identifier is input into the information extraction model as prompt information, and the information of the extraction category corresponding to the identifier can be extracted in the text to be extracted.

[0082] For step S104, Figure 4 The internal structure diagram of the information extraction model provided in this manual is as follows: Figure 4 As shown, the encoder of the information extraction model includes a text encoder and a prompt encoder. Then, the server can input the prompt information into the prompt encoder in the pre-trained information extraction model, and the prompt encoder encodes the prompt information to obtain a prompt encoding result. And the text to be extracted is input into the text encoder in the pre-trained information extraction model, and the text encoder encodes the text to be extracted to obtain a text encoding result. Afterwards, the prompt encoding result and the text encoding result are determined as the encoding result of the text to be extracted.

[0083] By using multiple encoders, the text to be extracted and the prompt information can be encoded separately. For the same text to be extracted, if it is extracted multiple times and the target extraction category of each extraction changes, only the prompt information encoder can be used to re-encode the target extraction category, and there is no need to repeatedly encode the text to be extracted through the text encoder, thereby improving the efficiency of information extraction. For example, for the same text to be extracted, the target extraction categories include a and b in the first extraction, and the target extraction categories include a, b and c in the second extraction. Then, since the text encoder can only encode the text to be extracted in the first extraction, the encoding result obtained by the first encoding can be used in the second extraction. However, since the target extraction category has changed, the prompt encoder needs to encode the prompt information in both the first and second extractions.

[0084] When encoding the text to be extracted, the text encoder performs word segmentation on the text to be extracted to obtain a word segmentation result including several text word segments, and then determines the word segmentation vector of each text word segment as the text encoding result.

[0085] In the process of decoding the encoding result, the copy mode in the copy mechanism can be used to screen the identifier in the prompt information, determine the identifier corresponding to the extraction category to which each text segmentation in the text to be extracted belongs, and copy it to obtain the decoding result. In other words, through the copy mode in the copy mechanism, according to the encoding result and the prompt information, the probability of the extraction category to which each text segmentation in the text to be extracted belongs is determined as the copy probability. Afterwards, for each text segmentation, the target copy probability is determined among the various copy probabilities of the text segmentation, and the one with the highest copy probability can be used as the target copy probability. Then, the identifier corresponding to the extraction category to which the target copy probability belongs is copied to obtain the decoding result, which includes each text segmentation in the text to be extracted and the identifier corresponding to each text segmentation.

[0086] For example, the identifiers included in the prompt information are A, B, and C. For text segmentation 1, the information extraction model determines that the probability that the text segmentation 1 belongs to the extraction category a corresponding to identifier A is 10%, the probability that it belongs to the extraction category b corresponding to identifier B is 5%, and the probability that it belongs to the extraction category c corresponding to identifier C is 85%. The probability of the extraction category corresponding to identifier C is used as the target copy probability. Then, the identifier C is copied and output, and the output decoding result includes identifier C and text segmentation 1.

[0087] In one or more embodiments of the present specification, the replication mechanism is used to filter the identifiers in the prompt information to determine the extraction categories corresponding to the text segmentation, that is, to exclude the extraction categories that do not exist in the text to be extracted. Then, the extraction categories in the final extraction result are the common categories.

[0088] The copy mechanism is usually used to jointly use the Generate Mode and the Copy Mode to solve the OOV (Out-of-Vocabulary, OOV) problem that often occurs in text generation tasks. On the one hand, the generation probability of text segmentation in the original vocabulary is calculated and generated using the Generate Mode. On the other hand, the probability of OOV text segmentation is calculated from the pre-defined unknown vocabulary (UnknownWords, UNK) based on the joint information and copied to the output using the Copy Mode. The original vocabulary includes the text segmentation that may appear in the pre-set text to be extracted. The text segmentation in the original vocabulary is usually the text segmentation extracted during the training of the information extraction model. The unknown vocabulary includes the text segmentation of the extraction category that the information extraction model may not be able to recognize.

[0089] Formally, the final probability of duplication of a text segmentation is as follows:

[0090] p=p gen ×logits LM +(1-p gen )×logits Copy

[0091] where p gen =softmax(FFN(c t )) is the generation probability, is the context vector, The attention h of the i-th token in the extracted text can be determined by generating a token at time t in the decoder. i That is the hidden layer state of the token, that is, the text encoding result, logits LM The output of the decoder decoding the text encoding result, logits Copy The decoder performs matrix multiplication on the cross attention matrix of the prompt encoding result and the one-hot vector matrix embedded in the prompt input, which can also be understood as the decoder decoding the prompt encoding result.

[0092] The decoding result may be in the form of a structural extraction language (SEL). For example, the text to be extracted is: In 1997, Steve was excited to become the CEO of Apple, and the prompt information is<label_id_7><label_id_23><label_id_54> , the extraction category corresponding to identifier 7 is person, the extraction category corresponding to identifier 23 is organization, and the extraction category corresponding to identifier 54 is location. Since the text to be extracted only includes identifiers 7 and 23, the decoding result is<label_id_7> Steve<label_id_23> Apple, the decoding result is in SEL format.

[0093] In addition, the information extraction model will not extract extraction categories other than the target extraction category. That is to say, in the process of using the copy mechanism, when determining the target copy probability, in addition to judging whether the copy probability is the maximum probability of a certain text segmentation, it will also judge whether the extraction category to which the copy probability belongs belongs to the target extraction category. If so, the copy probability will be determined as the target copy probability. For example, the identifiers corresponding to the categories that can be extracted by the information extraction model include A, B, C, and F. The identifiers included in the encoding result of the text to be extracted are A, B, and F. The identifiers corresponding to the target extraction category include A, B, and C. Although the information extraction model can extract category F, since F is not included in the target extraction category, after decoding through the copy mechanism, it is determined that the common identifiers are A and B, and the decoding result will only include identifiers A and B, that is, text segmentations.

[0094] That is to say, in the process of using the copy mechanism, when determining the target copy probability, in addition to judging whether the copy probability is the maximum probability of a certain text segmentation, it is also judged whether the extraction category to which the copy probability belongs belongs to the target extraction category. If so, the copy probability is determined as the target copy probability. Of course, after determining the target copy probability, it is also possible to judge whether the extraction category to which the target copy probability belongs belongs to the target extraction category. If so, the identifier corresponding to the extraction category to which the target copy probability belongs is copied. If not, the identifier corresponding to the extraction category to which the target copy probability belongs is not copied.

[0095] It should be noted that, since the input prompt information is an identifier rather than an extraction category, the extraction category described in the decoding process of this specification appears in the form of an identifier in the information extraction model.

[0096] Furthermore, the decoder outputs the identifier and the text segmentation corresponding to the identifier. Therefore, the server also needs to convert the identifier into an extraction category, that is, determine the extraction result of the text to be extracted according to the decoding result. Specifically, the server can determine the extraction category corresponding to each identifier in the decoding result according to the preset corresponding relationship, and determine the extraction category corresponding to each identifier and each text segmentation in the text to be extracted as the extraction result of the text to be extracted.

[0097] This specification also provides a training method for an information extraction model. First, the server obtains a sample text to be extracted and determines a label of the sample text to be extracted. Then, a number of categories to be extracted are determined, and identifiers corresponding to the categories to be extracted are determined as sample prompt information. The sample prompt information and the sample text to be extracted are input into the information extraction model to obtain a predicted extraction result of the sample text to be extracted output by the information extraction model. The information extraction model is trained based on the predicted extraction result and the label.

[0098] When determining the label, determine the sample extraction category of each text segmentation in the sample extraction text. For each sample extraction category, determine the sample identifier corresponding to the sample extraction category according to a preset correspondence relationship. Each text segmentation in the sample extraction text and the sample identifier corresponding to the text segmentation are determined as the label of the sample extraction text.

[0099] Since different encoders are used for text encoding and prompt information encoding in the information extraction model of this manual, when extracting text from the same sample and when it is necessary to extract some categories in a targeted manner, only the prompt information encoder needs to be used to encode the prompt information, thereby improving the efficiency of model training. If it is necessary to migrate to a new category system, only the prompt information encoder needs to be used to encode the prompt information to fine-tune the information extraction model. Migrating to a new category system is to expect the information extraction model to have the ability to extract new categories of information, and the new categories refer to the categories of information that the information extraction model cannot extract before the information extraction model is subjected to migration training.

[0100] In addition, mapping the extraction categories to different identifiers can not only reduce the length of the input information extraction model, but also avoid the information extraction model only learning the incorrect correlation information between the natural language categories and the input text to be extracted, while ignoring the true semantics, further improving the accuracy of the model extraction results.

[0101] In the early stage of training, since the identifiers corresponding to the categories to be extracted are not in the original vocabulary, the information extraction model may classify the identifiers corresponding to the categories to be extracted as unknown vocabulary, and the information extraction model may not output information that it cannot recognize, resulting in the final output extraction result not including the identifiers corresponding to the categories to be extracted. For example, the sample identifiers corresponding to the extraction categories included in the sample text to be extracted are A and B, and the identifiers corresponding to the categories to be extracted include A and B. Since the information extraction model cannot recognize identifier A and identifier B, the information extraction model will classify identifier A and identifier B as unknown vocabulary, and the decoder in the information extraction model will not output identifier A and identifier B, resulting in the final prediction extraction result not including the extraction category corresponding to the text segmentation.

[0102] Based on the above problems, this manual will also use a copy mechanism during the model training stage, through which the data classified as an unknown vocabulary by the information extraction model can be directly copied. Then, the identifier corresponding to the category to be extracted will be copied and output, so that the predicted extraction result includes the identifier.

[0103] In addition, when copying an identifier, the probability that the text segmentation belongs to a certain category will be determined to obtain the copy probability, which is used to determine whether to copy the identifier corresponding to the aforementioned category. Usually, the identifier with the highest copy probability can be copied. For example, the identifiers included in the sample prompt information are D, E, and F. For text segmentation 2, the information extraction model determines that the probability that the text segmentation 1 belongs to the category d to be extracted corresponding to the identifier D is 3%, the probability that it belongs to the category e to be extracted corresponding to the identifier E is 7%, and the probability that the identifier F corresponds to the category f to be extracted is 90%. Therefore, the information extraction model can copy the identifier F and output the identifier F and the text segmentation 2.

[0104] The above is an information extraction method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding information extraction device, such as Figure 5 shown.

[0105] Figure 5 This specification provides a schematic diagram of an information extraction device, the device comprising:

[0106] An input acquisition module 500 is used to acquire the text to be extracted and determine a number of target extraction categories;

[0107] A prompt information determination module 502 is used to determine, for each target extraction category, an identifier corresponding to the target extraction category according to a preset corresponding relationship as prompt information;

[0108] The extraction module 504 is used to input the prompt information and the text to be extracted into a pre-trained information extraction model to obtain the extraction result of the text to be extracted output by the information extraction model. The information extraction model is trained by the identifier determined by the preset corresponding relationship.

[0109] Optionally, the information extraction model includes an encoder and a decoder;

[0110] The extraction module 504 is specifically used to input the prompt information and the text to be extracted into an encoder in a pre-trained information extraction model, and the encoder encodes the prompt information and the text to be extracted to obtain the encoding result of the text to be extracted; input the encoding result of the text to be extracted into a decoder in the information extraction model, and the decoder determines the extraction result of the text to be extracted based on a copy mechanism.

[0111] Optionally, the encoder includes a text encoder and a prompt encoder;

[0112] The extraction module 504 is specifically used to input the prompt information into a prompt encoder in a pre-trained information extraction model, and the prompt encoder encodes the prompt information to obtain a prompt encoding result; and input the text to be extracted into a text encoder in a pre-trained information extraction model, and the text encoder encodes the text to be extracted to obtain a text encoding result; and determine the prompt encoding result and the text encoding result as the encoding result of the text to be extracted.

[0113] Optionally, the extraction module 504 is specifically used for the text encoder to segment the text to be extracted to obtain a segmentation result including a plurality of text segmentations; for each text segmentation, a segmentation vector of the text segmentation is determined as a text encoding result.

[0114] Optionally, the extraction module 504 is specifically used to determine the probability of the extraction category to which each text segmentation in the text to be extracted belongs as the copy probability according to the encoding result and the prompt information through the copy mode in the copy mechanism; for each text segmentation, determine the target copy probability among the various copy probabilities of the text segmentation; copy the identifier corresponding to the extraction category to which the target copy probability belongs to obtain a decoding result, the decoding result including each text segmentation in the text to be extracted and the identifier corresponding to each text segmentation; and determine the extraction result of the text to be extracted based on the decoding result.

[0115] Optionally, the extraction module 504 is specifically used to determine the extraction category corresponding to each identifier in the decoding result according to the preset corresponding relationship; and determine the extraction category corresponding to each identifier and each text segment in the text to be extracted as the extraction result of the text to be extracted.

[0116] Optionally, the device further comprises:

[0117] The training module 506 is used to obtain sample text to be extracted and determine the label of the sample extracted text; determine a number of categories to be extracted and determine identifiers corresponding to the categories to be extracted as sample prompt information; input the sample prompt information and the sample text to be extracted into the information extraction model to obtain the predicted extraction result of the sample text to be extracted output by the information extraction model; and train the information extraction model according to the predicted extraction result and the label.

[0118] Optionally, the training module 506 is specifically used to determine the sample extraction category of each text segmentation in the sample extraction text; for each sample extraction category, determine the sample identifier corresponding to the sample extraction category according to a preset correspondence; and determine each text segmentation in the sample extraction text and the sample identifier corresponding to the text segmentation as the label of the sample extraction text.

[0119] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides information extraction methods.

[0120] This manual also provides Figure 6 The structural diagram of the electronic device shown in FIG. Figure 6 As shown, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0121] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0122] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0123] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0124] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0125] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0127] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0128] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0129] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0130] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0131] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0132] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0133] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0135] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0136] The above description is only an embodiment of this specification and is not intended to limit this specification. For those skilled in the art, this specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this specification should be included in the scope of the claims of this application.

Claims

1. An information extraction method, the method comprising: Get the text to be extracted and determine several target extraction categories; For each target extraction category, according to a preset corresponding relationship, an identifier corresponding to the target extraction category is determined as prompt information; The prompt information and the text to be extracted are input into a pre-trained information extraction model to obtain an extraction result of the text to be extracted output by the information extraction model, wherein the information extraction model is trained by an identifier determined by the preset corresponding relationship.

2. The method according to claim 1, wherein the information extraction model comprises an encoder and a decoder; Inputting the prompt information and the text to be extracted into a pre-trained information extraction model to obtain an extraction result of the text to be extracted output by the information extraction model, specifically including: Inputting the prompt information and the text to be extracted into an encoder in a pre-trained information extraction model, the encoder encodes the prompt information and the text to be extracted to obtain an encoding result of the text to be extracted; The encoding result of the text to be extracted is input into the decoder in the information extraction model, and the decoder determines the extraction result of the text to be extracted based on a replication mechanism.

3. The method of claim 2, wherein the encoder comprises a text encoder and a prompt encoder; Inputting the prompt information and the text to be extracted into an encoder in a pre-trained information extraction model, the encoder encodes the prompt information and the text to be extracted to obtain an encoding result of the text to be extracted, specifically including: Inputting the prompt information into a prompt encoder in a pre-trained information extraction model, wherein the prompt encoder encodes the prompt information to obtain a prompt encoding result; and Inputting the text to be extracted into a text encoder in a pre-trained information extraction model, wherein the text encoder encodes the text to be extracted to obtain a text encoding result; The prompt encoding result and the text encoding result are determined as the encoding result of the text to be extracted.

4. The method according to claim 3, wherein the text encoder encodes the text to be extracted to obtain a text encoding result, specifically comprising: The text encoder performs word segmentation on the text to be extracted to obtain a word segmentation result including a plurality of text word segmentations; For each text segmentation, determine the segmentation vector of the text segmentation as the text encoding result.

5. The method according to claim 2, wherein the encoding result of the text to be extracted is input into a decoder in the information extraction model, and the decoder determines the extraction result of the text to be extracted based on a replication mechanism, specifically comprising: By using the copy mode in the copy mechanism, according to the encoding result and the prompt information, the probability of the extraction category to which each text segmentation in the to-be-extracted text belongs is determined as the copy probability; For each text segmentation, determine a target replication probability among the replication probabilities of the text segmentation; Copy the identifier corresponding to the extraction category to which the target copy probability belongs, and obtain a decoding result, wherein the decoding result includes each text segmentation in the text to be extracted and the identifier corresponding to each text segmentation; According to the decoding result, an extraction result of the text to be extracted is determined.

6. The method according to claim 5, determining the extraction result of the text to be extracted according to the decoding result, specifically comprising: Determining, according to the preset corresponding relationship, an extraction category corresponding to each identifier in the decoding result; The extraction category corresponding to each identifier and each text segment in the text to be extracted are determined as the extraction result of the text to be extracted.

7. The method according to claim 1, wherein training the information extraction model comprises: Obtaining a sample text to be extracted and determining a label of the sample extracted text; Determine a number of categories to be extracted, and determine identifiers corresponding to the categories to be extracted as sample prompt information; Input the sample prompt information and the sample text to be extracted into an information extraction model to obtain a predicted extraction result of the sample text to be extracted output by the information extraction model; The information extraction model is trained according to the predicted extraction result and the label.

8. The method according to claim 7, wherein determining the label of the sample extracted text comprises: Determining a sample extraction category for each text segmentation in the sample extraction text; For each sample extraction category, determine a sample identifier corresponding to the sample extraction category according to a preset corresponding relationship; Each text segmentation in the sample extracted text and a sample identifier corresponding to the text segmentation are determined as a label of the sample extracted text.

9. An information extraction device, comprising: The input acquisition module is used to obtain the text to be extracted and determine several target extraction categories; A prompt information determination module, used to determine, for each target extraction category, an identifier corresponding to the target extraction category according to a preset corresponding relationship as prompt information; The extraction module is used to input the prompt information and the text to be extracted into a pre-trained information extraction model to obtain the extraction result of the text to be extracted output by the information extraction model. The information extraction model is trained by the identifier determined by the preset corresponding relationship.

10. The apparatus of claim 9, wherein the information extraction model comprises an encoder and a decoder; The extraction module is specifically used to input the prompt information and the text to be extracted into an encoder in a pre-trained information extraction model, and the encoder encodes the prompt information and the text to be extracted to obtain the encoding result of the text to be extracted; input the encoding result of the text to be extracted into a decoder in the information extraction model, and the decoder determines the extraction result of the text to be extracted based on a copy mechanism.

11. The apparatus of claim 10, wherein the encoder comprises a text encoder and a prompt encoder; The extraction module is specifically used to input the prompt information into a prompt encoder in a pre-trained information extraction model, and the prompt encoder encodes the prompt information to obtain a prompt encoding result; and input the text to be extracted into a text encoder in a pre-trained information extraction model, and the text encoder encodes the text to be extracted to obtain a text encoding result; and determine the prompt encoding result and the text encoding result as the encoding result of the text to be extracted.

12. In the device as described in claim 11, the extraction module is specifically used for the text encoder to segment the text to be extracted to obtain a segmentation result including several text segmentations; for each text segmentation, the segmentation vector of the text segmentation is determined as the text encoding result.

13. The device as described in claim 10, wherein the extraction module is specifically used to determine the probability of the extraction category to which each text segmentation in the text to be extracted belongs as the copy probability according to the encoding result and the prompt information through the copy mode in the copy mechanism; for each text segmentation, determine the target copy probability among the various copy probabilities of the text segmentation; copy the identifier corresponding to the extraction category to which the target copy probability belongs to obtain a decoding result, wherein the decoding result includes each text segmentation in the text to be extracted and the identifier corresponding to each text segmentation; and determine the extraction result of the text to be extracted according to the decoding result.

14. In the device as described in claim 13, the extraction module is specifically used to determine the extraction category corresponding to each identifier in the decoding result according to the preset corresponding relationship; and determine the extraction category corresponding to each identifier and each text segment in the text to be extracted as the extraction result of the text to be extracted.

15. The apparatus of claim 9, further comprising: A training module is used to obtain sample text to be extracted and determine the label of the sample extracted text; Determine a number of categories to be extracted, and determine identifiers corresponding to the categories to be extracted as sample prompt information; The sample prompt information and the sample text to be extracted are input into an information extraction model to obtain a predicted extraction result of the sample text to be extracted output by the information extraction model; and the information extraction model is trained according to the predicted extraction result and the label.

16. In the device as described in claim 15, the training module is specifically used to determine the sample extraction category of each text segmentation in the sample extraction text; for each sample extraction category, determine the sample identifier corresponding to the sample extraction category according to a preset correspondence; and determine each text segmentation in the sample extraction text and the sample identifier corresponding to the text segmentation as the label of the sample extraction text.

17. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the method according to any one of claims 1 to 8.

18. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 8 when executing the program.

Images