Information extraction method and device, electronic equipment and storage medium

Abstract

The application discloses an information extraction method and device, electronic equipment and storage medium, and belongs to the field of text information extraction, wherein the information extraction method comprises the following steps: determining a mapping relationship between a target entity type in a downstream extraction task and an entity type concept in a pre-established entity type tree to obtain an entity type mapping table; converting the target entity type into an entity type list compatible with an information extraction model corresponding to the downstream extraction task based on the entity type mapping table; performing information extraction on a target text based on the entity type list to obtain entity information corresponding to the entity type list; and performing inverse mapping on the entity information corresponding to the entity type list based on the entity type mapping table to obtain target entity information corresponding to the target entity type. The application can effectively improve the efficiency of entity information extraction and reduce development cost.

Description

Technical Field

[0001] This invention relates to the field of text information extraction technology, and in particular to an information extraction method, apparatus, electronic device and storage medium. Background Technology

[0002] Information extraction has always been a research hotspot in the field of natural language understanding. Its purpose is to extract specific types of target entities from common texts, playing an important role in syntactic analysis, question answering systems, and knowledge graph construction. However, the definition of entity type (i.e., schema) differs in different application scenarios, requiring the construction of different extraction models. Therefore, in practical information extraction applications, it is often necessary to maintain multiple extraction models simultaneously, leading to high complexity and cost.

[0003] In existing technologies, general information extraction models mainly fall into two categories: generative extraction models and pointer network extraction models. Existing generative extraction models, which use generative methods to obtain target entity text, struggle to guarantee the accuracy of long texts and suffer from poor controllability. Existing pointer network-based extraction models can only extract entities corresponding to a single entity type at a time. For scenarios with many entity types, the extraction process needs to be repeated, resulting in low computational efficiency. Furthermore, their decoding process is complex and difficult to optimize, making them unsuitable for handling situations with complex and variable semantic granularity. Summary of the Invention

[0004] This invention provides an information extraction method, apparatus, electronic device, and storage medium to solve or partially solve the problems of needing to construct different information extraction models for different entity types in different application scenarios, as well as the low accuracy, poor controllability, low computational efficiency, and complex decoding process of existing extraction models for long text generation.

[0005] In a first aspect, the present invention provides an information extraction method, comprising: determining the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established entity type tree, and obtaining an entity type mapping table;

[0006] Based on the entity type mapping table, the target entity type is converted into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task;

[0007] Based on the entity type list, information is extracted from the target text to obtain the entity information corresponding to the entity type list.

[0008] Based on the entity type mapping table, the entity information corresponding to the entity type list is reverse-mapped to obtain the target entity information corresponding to the target entity type.

[0009] In some embodiments, the step of extracting information from the target text based on the entity type list to obtain entity information corresponding to the entity type list includes:

[0010] Input the entity type list and the target text into the information extraction model to obtain the entity information corresponding to the entity type list;

[0011] The information extraction model is trained using entity type samples and text samples as training samples, and the entity information corresponding to the entity type samples as training labels.

[0012] The entity type sample is determined based on the entity type tree.

[0013] In some embodiments, the information extraction model includes: an entity type-side encoder, a text-side encoder, a feature interaction module, and a global pointer module, wherein,

[0014] The entity type side encoder is used to extract features from the entity types input to the information extraction model;

[0015] The text-side encoder is used to extract features from the text input to the information extraction model;

[0016] The feature interaction module is used to perform feature interaction between the output of the entity type-side encoder and the output of the text-side encoder.

[0017] The global pointer module is used to perform feature mapping on the output of the feature interaction module and output a multi-head prediction matrix.

[0018] In some embodiments, the step of inputting the entity type list and the target text into the information extraction model to obtain entity information corresponding to the entity type list includes:

[0019] Based on the entity type side encoder, feature extraction is performed on the entity type list to obtain the entity type features corresponding to each entity type in the entity type list. Based on the text side encoder, feature extraction is performed on the target file to obtain the text features corresponding to the target text.

[0020] Based on the feature interaction module, feature interaction is performed on the entity type feature and text feature to obtain the interacted entity type feature and the interacted text feature.

[0021] Based on the global pointer module, feature mapping is performed on the entity type features and text features after the interaction to obtain a multi-head prediction matrix;

[0022] The multi-head prediction matrix is ​​decoded to obtain the entity information corresponding to the input entity type.

[0023] In some embodiments, the step of performing feature interaction on the entity type features and text features based on the feature interaction module to obtain the interacted entity type features and interacted text features includes:

[0024] The entity type features and text features are mapped respectively;

[0025] Perform matrix multiplication on the mapped entity type features and text features to obtain the first attention matrix;

[0026] The first attention matrix is ​​normalized along the dimensions of the entity type features and the text features, respectively, to obtain the second attention matrix and the third attention matrix;

[0027] The mapped entity type features are multiplied by the second attention matrix to obtain the interactive text features, and the mapped text features are multiplied by the third attention matrix to obtain the interactive entity type features.

[0028] In some embodiments, the step of performing feature mapping on the interacted entity type features and the interacted text features based on the global pointer module to obtain a multi-head prediction matrix includes:

[0029] Obtain the feature vector of the first character in the entity type feature after the interaction, and use it as the first entity type feature;

[0030] Map the features of the first entity type;

[0031] The text features after the interaction are mapped four times. The outputs of two mappings are multiplied by matrix to obtain the entity prediction matrix. The outputs of the other two mappings are multiplied by the first entity type features after mapping to obtain the first matrix and the second matrix. The second matrix is ​​transposed and then summed with the first matrix to obtain the multi-head matrix.

[0032] The entity prediction matrix and the multi-head matrix are summed to obtain the multi-head prediction matrix.

[0033] In some embodiments, the training process of the information extraction model includes:

[0034] Acquire training data, and preprocess the training data based on the entity type tree to obtain the training samples and training labels;

[0035] The training samples are input into the initial information extraction model to perform a pre-training task, and the prediction data corresponding to the pre-training task is obtained.

[0036] Calculate the loss value between the predicted data corresponding to the pre-training task and the real data corresponding to the pre-training task, wherein the real data corresponding to the pre-training task is determined based on the training labels;

[0037] Based on the loss value, the initial information extraction model is iterated through parameters. After the iteration is completed, a general information extraction model is obtained.

[0038] The general information extraction model is fine-tuned based on the downstream extraction task to obtain the information extraction model.

[0039] In some embodiments, the pre-training tasks include: entity type identification task, fragment extraction task, sequence labeling prediction task, and fragment boundary prediction task.

[0040] In a second aspect, the present invention provides an information extraction device, comprising:

[0041] The mapping relationship determination unit is used to determine the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established entity type tree, and obtain the entity type mapping table.

[0042] The mapping unit is used to convert the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task, based on the entity type mapping table.

[0043] The information extraction unit is used to extract information from the target text based on the entity type list to obtain entity information corresponding to the entity type list.

[0044] The inverse mapping unit is used to perform inverse mapping on the entity information corresponding to the entity type list based on the entity type mapping table to obtain the target entity information corresponding to the target entity type.

[0045] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the information extraction method as described in any of the first aspects.

[0046] Fourthly, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the information extraction method as described in any of the first aspects.

[0047] The information extraction method, apparatus, electronic device, and storage medium provided by this invention establish an entity type mapping table and, based on the entity type mapping table, convert the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task. Then, based on the entity type list, information extraction is performed on the target text to obtain the entity information corresponding to the entity type list. Finally, based on the entity type mapping table, the entity information corresponding to the entity type list is reverse-mapped to obtain the target entity information corresponding to the target entity type. This enables entity information extraction based on a unified general information extraction model in different application scenarios, effectively improving the efficiency of entity information extraction and reducing development costs. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0049] Figure 1 An application environment diagram showing the operational capability of the information extraction method provided in one embodiment of the present invention;

[0050] Figure 2 A flowchart illustrating an information extraction method according to an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of the schema tree structure and its correspondence provided in one embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of an information extraction process based on a general information extraction model, provided as an embodiment of the present invention.

[0053] Figure 5 This is a schematic diagram of the structure of a general information extraction model provided in one embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram of the structure of a feature interaction module provided in one embodiment of the present invention;

[0055] Figure 7 This is a schematic diagram of the structure of a global pointer module provided in one embodiment of the present invention;

[0056] Figure 8 A schematic diagram illustrating the construction process of a general information extraction model based on a schema tree, provided in one embodiment of the present invention;

[0057] Figure 9An example diagram of the extraction results of the general information extraction model provided in one embodiment of the present invention on the CLUENER dataset;

[0058] Figure 10 This is a schematic diagram of the structure of an information extraction device provided in one embodiment of the present invention;

[0059] Figure 11 This is a schematic diagram of the physical structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0061] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0062] To address the shortcomings of existing technologies, such as the need to construct different information extraction models for different entity types in different application scenarios, and the low accuracy, poor controllability, low computational efficiency, and complex decoding process of existing general information extraction models for long text generation, this paper addresses these issues.

[0063] This invention provides an information extraction method, apparatus, electronic device, and storage medium, which are described below in conjunction with... Figures 1-11 The present invention will now be described.

[0064] The information extraction method provided by this invention can be applied to, for example... Figure 1 The application environment shown. Figure 1 This is a diagram illustrating the application environment in which the information extraction method provided in one embodiment of the present invention can operate. For example... Figure 1As shown, the application environment includes terminal 110 and server 120. Terminal 110 and server 120 communicate via a network, which can be a wireless communication network or a wired communication network. The number of terminals and servers is unlimited. The wireless communication network can include, but is not limited to, at least one of the following: WIFI (Wireless Fidelity) and Bluetooth. The wired communication network can include, but is not limited to, at least one of the following: wide area network (WAN), metropolitan area network (MAN), and local area network (LAN).

[0065] In some embodiments, terminal 110 (terminal device) includes various handheld devices, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem with wireless communication capabilities, such as mobile phones, tablets, desktop laptops, and smart devices capable of running applications, including the central console of a smart car. Specifically, it can refer to user equipment (UE), access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication equipment, user agent, or user device. Terminal devices can also be satellite phones, cellular phones, smartphones, wireless data cards, wireless modems, machine-type communication devices, cordless phones, session initiation protocol (SIP) phones, wireless local loop (WLL) stations, personal digital assistants (PDAs), handheld devices with wireless communication capabilities, computing devices or other processing devices connected to a wireless modem, in-vehicle devices or wearable devices, virtual reality (VR) terminal devices, augmented reality (AR) terminal devices, wireless terminals in industrial control, wireless terminals in self-driving, wireless terminals in remote medical care, wireless terminals in smart grids, wireless terminals in transportation safety, wireless terminals in smart cities, wireless terminals in smart homes, terminal devices in 5G networks or future communication networks, etc. Terminals can be battery-powered or attached to and powered by the power system of a vehicle or vessel. The power system of a vehicle or ship can also charge the terminal's battery to extend the terminal's communication time.

[0066] Server 120 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0067] It should be noted that the implementation of the method in this invention can be completed directly on the terminal 110, or directly on the server 120, or it can be completed on the server 120 and then sent to the terminal 110 by the server 120.

[0068] Terminal 110 or server 120 determines the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established entity type tree to obtain an entity type mapping table; based on the entity type mapping table, the target entity type is converted into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task; based on the entity type list, information extraction is performed on the target text to obtain the entity information corresponding to the entity type list; based on the entity type mapping table, the entity information corresponding to the entity type list is inversely mapped to obtain the target entity information corresponding to the target entity type. This enables entity information extraction based on a unified general information extraction model in different application scenarios, effectively improving the efficiency of entity information extraction and reducing development costs.

[0069] Figure 2 This is a flowchart illustrating an information extraction method according to an embodiment of the present invention. Figure 2 As shown, an information extraction method is provided, which is then applied to... Figure 1 The method is illustrated using a terminal as an example, including the following steps: step 200, step 201, step 202, and step 203. These steps are merely one possible implementation of the present invention.

[0070] Step 200: Determine the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established entity type tree to obtain the entity type mapping table;

[0071] Optionally, downstream extraction tasks refer to information extraction tasks in actual application scenarios.

[0072] The target entity type is the entity type that needs to be extracted in the downstream extraction task, based on different application scenarios.

[0073] The pre-built entity type tree is a schema tree, which is constructed by collecting a large amount of information to extract supervised data, organizing the schema tags contained in all the data, and then combining open source knowledge graphs and vocabularies to summarize the common schema tag concepts and the synonym relationships and inclusion relationships between concepts, and constructing a schema tree accordingly.

[0074] It should be noted that the "concept" here actually refers to the true meaning of the schema tags. We can summarize and record the relationships between the meanings of different types of schema tags, such as inclusion relationships and synonym relationships, and then construct a schema tree based on the recorded schema information.

[0075] A schema tree can be used to integrate datasets from different information extraction sources and augment supervised data. Based on the integrated and augmented supervised data, the constructed model is trained, and the resulting model can extract target entities of the types contained in the schema tree.

[0076] Figure 3 This is a schematic diagram of the schema tree structure and its correspondence provided in one embodiment of the present invention, as shown below. Figure 3 As shown, the concept of "name" is the collection of all people's names and titles; "singer" and "lead actor" refer to two different professional titles, therefore "name" includes "lead actor" and "singer," and the three constitute the parent-child and sibling nodes of the schema tree. In addition, synonyms are also recorded in the schema information.

[0077] In different application scenarios, the entity types contained in the schema tree can be mapped or combined according to the actual application requirements to obtain the required entity types. This can overcome the problem of incompatibility between schema definitions in different scenarios, and at the same time, it can avoid the text being truncated due to the introduction of too many schemas.

[0078] For example, in a real extraction task, if you want an entity corresponding to the type "birth and death time," this type may not exist in the schema tree constructed in this embodiment of the invention. However, this type concept is the union of the two types "birth time" and "death time" in the schema tree. Therefore, you can obtain the required schema type by mapping or combining the entity corresponding to "birth and death time" as the entity of "birth time" plus the entity of "death time." Different scenarios require different schemas, and different mapping or combination methods can solve the problem of incompatible schema definitions in different scenarios.

[0079] In this embodiment of the invention, by determining the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established schema tree, and recording the mapping relationship, a schema mapping table corresponding to the downstream extraction task can be established.

[0080] It can be understood that the schema mapping table includes at least one mapping relationship, which is the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established schema tree.

[0081] Figure 4 This is a schematic diagram of an information extraction process based on a general information extraction model, provided as an embodiment of the present invention. Figure 4 As shown in step one, the target entity types include target schema 1, target schema 2, ... The entity type vocabulary in the schema tree includes schema A, schema B, schema X, schema Y, ...

[0082] By analyzing the mapping relationship between the actual entity types to be extracted and the entity types in the schema tree, the entity types corresponding to the target schema 1 in the schema tree were determined, including schema A, schema B, ... Similarly, the entity types corresponding to the target schema 2 in the schema tree were determined, including schema X, schema Y, ..., thus obtaining the schema mapping table.

[0083] Step 201: Based on the entity type mapping table, convert the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task;

[0084] Optionally, the information extraction model corresponding to the downstream extraction task can be obtained based on a general information extraction model. For example, the general information extraction model can be fine-tuned based on the requirements of the downstream extraction task, or the general information extraction model can be directly adopted.

[0085] Among them, the general information extraction model has the ability to extract general schema entities.

[0086] The entity types compatible with the information extraction model corresponding to the downstream extraction task are the entity types in the aforementioned pre-established schema tree. Since the entity type data in the schema tree participates in the training of the general information extraction model, after converting the target entity type into the list of entity types compatible with the information extraction model corresponding to the downstream extraction task, information extraction can be performed to obtain better extraction results.

[0087] Continue to refer to Figure 4 Based on such Figure 4 The mapping table shown in step one can convert the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task. For example, Figure 4 In the context of target entity types, if the target entity types are target schema 1 and target schema 2, then the list of entity types corresponding to the target entity attendees is a list consisting of schema A, schema B, ..., schema X, schema Y, ...

[0088] It should be noted that the entity type list includes at least one entity type.

[0089] Step 202: Based on the entity type list, extract information from the target text to obtain the entity information corresponding to the entity type list;

[0090] Optionally, the entity information includes the entity type and the start and end position information of the entity corresponding to the entity type.

[0091] like Figure 4 As shown in step two, the entity type list composed of schema A, schema B, schema X, schema Y, etc., and the target text are input into the information extraction model corresponding to the downstream extraction task. Figure 4 The dual-tower general extraction model in the model extracts target text based on a list of entity types to obtain entity information corresponding to the list of entity types. The entity information corresponding to the list of entity types is "schema A": { "span a":[starting position 1, ending position 1]} ……"schema X": { "span x":[starting position n, ending position n]}…….

[0092] Step 203: Based on the entity type mapping table, perform inverse mapping on the entity information corresponding to the entity type list to obtain the target entity information corresponding to the target entity type.

[0093] It should be noted that since step 202 extracts information from the target text based on the entity type list, the result is the entity information corresponding to the entity type list. That is, the entity corresponding to the entity type in the schema tree of the target text is extracted, rather than the actual entity type to be extracted. Therefore, it is necessary to perform reverse mapping (or inverse mapping) according to the previously determined mapping table to obtain the entity information corresponding to the target entity type.

[0094] like Figure 4 As shown in step three, based on the established schema mapping table, the entity information corresponding to the entity type list, "schema A": { "span a":[starting position 1, ending position 1]} ……"schema X": { "spanx":[starting position n, ending position n]}……, is reverse-mapped to obtain "target schema 1": { "span a":[starting position 1, ending position 1]} ……"target schema 2": { "span x":[starting position n, ending position n]}……, which is to obtain the target entity information corresponding to the target entity type.

[0095] It should be noted that in some application scenarios, if the target entity type belongs to the entity type in the schema tree established in the embodiments of the present invention, then equivalent mapping can be directly performed. The target entity type can be used as an entity type list, and information extraction can be performed on the target text to directly obtain the entity information corresponding to the target entity type.

[0096] In this embodiment of the invention, by establishing an entity type mapping table and converting the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task, information extraction is performed on the target text based on the entity type list to obtain the entity information corresponding to the entity type list. Finally, based on the entity type mapping table, the entity information corresponding to the entity type list is reverse-mapped to obtain the target entity information corresponding to the target entity type. This enables entity information extraction based on a unified general information extraction model in different application scenarios, which can effectively improve the efficiency of entity information extraction and reduce development costs.

[0097] It should be noted that each embodiment of the present invention can be freely combined, rearranged, or executed individually, and does not need to rely on or depend on a fixed execution order.

[0098] In some embodiments, the step of extracting information from the target text based on the entity type list to obtain entity information corresponding to the entity type list includes:

[0099] Input the entity type list and the target text into the information extraction model to obtain the entity information corresponding to the entity type list;

[0100] The information extraction model is trained using entity type samples and text samples as training samples, and the entity information corresponding to the entity type samples as training labels.

[0101] The entity type sample is determined based on the entity type tree.

[0102] Optionally, Figure 5 This is a schematic diagram of the structure of a general information extraction model provided in one embodiment of the present invention, as shown below. Figure 5 As shown, the general information extraction model includes: an encoder, a feature interaction module, and a global pointer module.

[0103] The encoder includes a schema-side encoder and a text-side encoder.

[0104] The schema-side encoder takes a list of schemas to be extracted as input and outputs schema features. The schema-side encoder can extract features from all entity types simultaneously.

[0105] The text-side encoder takes the text to be extracted as input and outputs the text features corresponding to the text to be extracted.

[0106] Optionally, the encoder can be a Transformer-based encoder, such as the encoder of the pre-trained language model BART, or the encoder of the BERT pre-trained model. This embodiment of the invention does not specifically limit this.

[0107] It should be noted that, in order to facilitate subsequent feature interaction and fusion calculations, it is necessary to ensure that the feature encoding length of each character output by the text-side encoder is consistent with that of the schema-side encoder.

[0108] like Figure 5 As shown, the input to the feature interaction module is the schema features and text features output by the encoder mentioned above. The module interacts with each other through the cross attention involved, and the output is the schema feature vector and the text feature vector after interaction.

[0109] The two features obtained by the feature interaction module have the same dimensions as the input dimension. In order to reduce the module design cost and increase the model depth and enhance the feature extraction capability of the model, the feature interaction module can be repeatedly cascaded in this embodiment of the invention. Cascading means that the modules are repeatedly stacked one after the other. In addition to the initial module receiving the specified input, the input of the subsequent modules comes from the output of the previous module with the same structure.

[0110] Optionally, the feature interaction module can be used a total of 6 times in a cascade.

[0111] The input to the global pointer module is the interactive schema feature vector and the interactive text feature vector. After a series of feature mappings, the output is a multi-head prediction matrix. Each sub-prediction matrix in the multi-head prediction matrix encodes the extraction result of the corresponding schema.

[0112] As mentioned in the foregoing embodiments, the information extraction model corresponding to the downstream extraction task can be obtained based on a general information extraction model. For example, the general information extraction model can be fine-tuned based on the requirements of the downstream extraction task, or the general information extraction model can be directly adopted. It can be understood that the information extraction model or the general information extraction model is trained using entity type samples and text samples as training samples, and the entity information corresponding to the entity type samples as training labels.

[0113] The entity type samples are determined based on the schema tree. This can be understood as constructing entity type samples based on the entity types in the schema tree. Therefore, the general information extraction model has good extraction capabilities for entity types in the schema tree.

[0114] In this embodiment of the invention, by obtaining the entity information corresponding to the entity type list and the target text input information extraction model, entity information can be extracted based on a unified general information extraction model in different application scenarios. It can be applied to extraction scenarios with many entity types, without repeatedly executing the extraction process, reducing redundant calculations and effectively reducing development costs.

[0115] In some embodiments, the step of inputting the entity type list and the target text into the information extraction model to obtain entity information corresponding to the entity type list includes:

[0116] Based on the entity type side encoder, feature extraction is performed on the entity type list to obtain the entity type features corresponding to each entity type in the entity type list. Based on the text side encoder, feature extraction is performed on the target file to obtain the text features corresponding to the target text.

[0117] Based on the feature interaction module, feature interaction is performed on the entity type feature and text feature to obtain the interacted entity type feature and the interacted text feature.

[0118] Based on the global pointer module, feature mapping is performed on the entity type features and text features after the interaction to obtain a multi-head prediction matrix;

[0119] The multi-head prediction matrix is ​​decoded to obtain the entity information corresponding to the input entity type.

[0120] Optionally, feature extraction is performed on the entity type list based on the schema-side encoder in the information extraction model to obtain the schema features corresponding to each entity type in the entity type list.

[0121] Based on the text-side encoder in the information extraction model, features are extracted from the target text to obtain the text features corresponding to the target text.

[0122] Based on the feature interaction module in the information extraction model, the schema features and text features are interacted to obtain the interacted schema features and the interacted text features.

[0123] Based on the global pointer module in the information extraction model, feature mapping is performed on the interactive schema features and interactive text features to obtain a multi-head prediction matrix.

[0124] Specifically, the interactive schema features and interactive text features are input. Figure 5The global pointer module of the general information extraction model performs feature mapping on the interacted schema features and interacted text features to obtain a multi-head prediction matrix. The multi-head prediction matrix is ​​obtained by summing the entity prediction matrix and the multi-head matrix. The entity prediction matrix encodes all possible entities, and the multi-head matrix includes multiple sub-matrices, each sub-matrice representing the entity output result corresponding to an entity type. Rows and columns represent the start and end positions of the entity, respectively. Furthermore, the number of sub-prediction matrices in the multi-head prediction matrix is ​​consistent with the number of input entity types. Each sub-matrice in the multi-head prediction matrix encodes the extraction result of the corresponding schema.

[0125] Then, the multi-head prediction matrix is ​​decoded to obtain the entity information corresponding to the input entity type. The decoding process of the multi-head prediction matrix is ​​as follows: after thresholding each sub-prediction matrix in the multi-head prediction matrix, the row and column indices of the elements exceeding the threshold are obtained, and these indices are the start and end positions of the extracted entities.

[0126] In this embodiment of the invention, feature extraction is performed on the entity type list and the target text respectively to obtain the schema features corresponding to each entity type in the entity type list and the text features corresponding to the target text. Then, feature interaction is performed on the schema features and text features, and feature mapping is performed on the interacted schema features and interacted text features to obtain the multi-head prediction matrix. The multi-head prediction matrix is ​​decoded to obtain the entity information corresponding to the input entity type. This realizes the extraction of entity information based on a unified general information extraction model in different application scenarios. It can be applied to extraction scenarios with many entity types, without repeatedly executing the extraction process, reducing redundant calculations and effectively reducing development costs.

[0127] Figure 6 This is a schematic diagram of the structure of a feature interaction module provided in one embodiment of the present invention, as shown below. Figure 6 As shown, in some embodiments, the step of performing feature interaction on the entity type features and text features based on the feature interaction module to obtain the interacted entity type features and interacted text features includes:

[0128] The entity type features and text features are mapped respectively;

[0129] Perform matrix multiplication on the mapped entity type features and text features to obtain the first attention matrix;

[0130] The first attention matrix is ​​normalized along the dimensions of the entity type features and the text features, respectively, to obtain the second attention matrix and the third attention matrix;

[0131] The mapped entity type features are multiplied by the second attention matrix to obtain the interactive text features, and the mapped text features are multiplied by the third attention matrix to obtain the interactive entity type features.

[0132] Specifically, the schema features are mapped twice using two linear layers to obtain the mapped schema features, and the text features are mapped twice using two linear layers to obtain the mapped text features.

[0133] Linear layers are fully connected layers, such as Figure 6 The Linear layer in the middle.

[0134] Perform matrix multiplication on the mapped schema features from one mapping operation and the mapped text features from another mapping operation (e.g., Figure 6 The first attention matrix is ​​obtained by using the MatMul layer in the matrix.

[0135] Along the dimensions of the schema feature (e.g.) Figure 6 The first attention matrix is ​​normalized using the m-dimensional (m-th dimension) in the matrix, i.e., by... Figure 6 The softmax layer in the middle is used for normalization to obtain the second attention matrix. Along the dimensions of the text features (e.g., ... Figure 6 L in c The first attention matrix is ​​normalized using the dimension to obtain the third attention matrix.

[0136] Then, the mapped schema features output from the second mapping are multiplied by the matrix with the second attention, i.e., through... Figure 6 The MatMul layer performs matrix multiplication to obtain the interactive text features. The text features output from another mapping are then multiplied by the third attention layer, i.e., through... Figure 6 Matrix multiplication is performed on the MatMul layer to obtain the interactive schema features.

[0137] The dimensions of the text features after interaction are consistent with the dimensions of the input text features. Similarly, the dimensions of the schema features after interaction are consistent with the dimensions of the input schema features. During the feature interaction process, they can be repeatedly cascaded, with a total of 6 cascade times.

[0138] In this embodiment of the invention, by performing feature interaction between schema features and text features, interactive schema features and interactive text features are obtained. This allows full utilization of the contextual relationship between entities and text, thereby accurately extracting entity information from the text.

[0139] In some embodiments, the step of performing feature mapping on the interacted entity type features and the interacted text features based on the global pointer module to obtain a multi-head prediction matrix includes:

[0140] Obtain the feature vector of the first character in the entity type feature after the interaction, and use it as the first entity type feature;

[0141] Map the features of the first entity type;

[0142] The text features after the interaction are mapped four times. The outputs of two mappings are multiplied by matrix to obtain the entity prediction matrix. The outputs of the other two mappings are multiplied by the first entity type features after mapping to obtain the first matrix and the second matrix. The second matrix is ​​transposed and then summed with the first matrix to obtain the multi-head matrix.

[0143] The entity prediction matrix and the multi-head matrix are summed to obtain the multi-head prediction matrix.

[0144] Optionally, Figure 7 This is a schematic diagram of the structure of a global pointer module provided in one embodiment of the present invention, as shown below. Figure 7 As shown, the schema feature dimension is one dimension more than the text feature dimension. In this embodiment of the invention, the schema dimension is reduced to the same as the text feature dimension through pooling operations.

[0145] The feature vector of the first character of the schema feature after interaction is obtained as the first schema feature, which represents the entire schema feature.

[0146] To reduce the computational load and improve computational efficiency, linear layers (i.e., fully connected layers) are used to perform dimensionality reduction mapping on schema features and text features respectively.

[0147] A linear layer is used to map the features of the first schema. Four linear layers are then used to map the interacted text features four times. The outputs of two mappings are multiplied by a MatMul layer to obtain an entity prediction matrix, which encodes all possible entities. The outputs of the other two mappings are multiplied by the mapped features of the first schema by a MatMul layer to obtain a first matrix and a second matrix. Then, the second matrix is ​​transposed and summed with the first matrix to obtain a multi-head matrix.

[0148] The entity prediction matrix and the multi-head matrix are summed to obtain the multi-head prediction matrix.

[0149] In this embodiment of the invention, a multi-head prediction matrix is ​​obtained by performing feature mapping on the interactive schema features and interactive text features. Then, the entity information to be extracted can be obtained by decoding the multi-head prediction matrix, which can effectively reduce the amount of model computation and improve computational efficiency.

[0150] Figure 8 This is a schematic diagram of the training process of the information extraction model provided in the embodiments of the present invention, such as... Figure 8 As shown, in some embodiments, the training process of the information extraction model includes: steps 800, 801, 802, 803 and 804.

[0151] Step 800: Obtain training data, and preprocess the training data based on the entity type tree to obtain the training samples and training labels;

[0152] Specifically, a large amount of open-source supervised data is collected as training data, which consists of samples of labeled entities.

[0153] Preprocessing of training data using the schema tree described in the foregoing embodiments of the present invention includes:

[0154] For entities already labeled in the sample, according to the principle that the superordinate concept includes the subordinate concept, the labeled entities are expanded to include the superordinate concept and synonym concept of the corresponding schema.

[0155] Then, for each sample, several schemas without corresponding entities are randomly added as negative schema samples.

[0156] After preprocessing the training data, multiple training samples are obtained. Each training sample consists of entity type samples and text samples, and training labels corresponding to multiple training samples are obtained, that is, entity information corresponding to entity type samples.

[0157] This invention integrates different information extraction datasets using a schema tree and augments the supervised data. Based on the integrated and augmented supervised data, the constructed model is trained, and the resulting model can extract target entities of the types contained in the schema tree.

[0158] Taking the data “{'text': '《XXX》is a web novel serialized on XX Literature City, author is YYY','label':[[1,11,'novel work'], [12,17,'serialization website'], [28,31,'author'], [17,19,'novel progress'],[20,24,'novel type']]}” as an example, “[1,11,'novel work']” in the data indicates that characters 1 to 11 (excluding 11) in the text are entities of the type “novel work”. Since the superordinate concept of "novel works" in the schema tree is "books", and "books" has synonyms such as "book title" and "books", the annotations "[1,11,'books'],[1,11,'book title'],[1,11,'books']" are added. In addition, several schemas with no corresponding entities in the data are added, and the entities are marked as empty, such as "[0,0,'publisher'], [0,0,'time'], [0,0,'location']". The training data is preprocessed based on the schema tree to obtain the final preprocessed data as: "{'text': '《XXX》is a web novel serialized on XX Literature City, and the author is YYY','label':[[1,11,'novel work'], [12,17,'serialization website'], [28,31,'author'], [17,19,'novel progress'],[20,24,'novel type'],[1,11,'book'],[1,11,'book title'],[1,11,'book'],[0,0,'publisher'],[0,0,'time'], [0,0,'location']]}.

[0159] Step 801: Input the training samples into the initial information extraction model to perform a pre-training task and obtain the prediction data corresponding to the pre-training task;

[0160] First, an initial information extraction model is constructed, including an encoder, a feature interaction module, and a global pointer module.

[0161] The pre-trained samples are input into the initial information extraction model to perform the pre-training task and obtain the prediction data corresponding to the pre-training task.

[0162] Optionally, the pre-training tasks include: entity type identification task, fragment extraction task, sequence labeling prediction task, and fragment boundary prediction task.

[0163] It can be understood that the entity type identification task is a schema identification task.

[0164] In some embodiments, the initial information extraction model is trained using four different pre-training tasks: schema identification, fragment extraction, sequence labeling prediction, and fragment boundary prediction. The specific meanings of these four different pre-training tasks are as follows:

[0165] Schema identification task: For each training sample, the schema relationships contained in its annotations are divided into three categories: hierarchical relationships, synonym relationships, sibling relationships, and unrelated relationships. A relation matrix can be constructed, with elements assigned numbers 1, 2, 3, and 4 based on the pairwise relation types between schemas. To identify different schemas, the model performs additional matrix operations, multiplying the feature vectors of the interacting schemas pairwise to obtain the predicted relation matrix. The true relation matrix is ​​compared with the predicted matrix, and the loss value is calculated using the multi-label cross-entropy function.

[0166] Fragment Extraction Task: To ensure good model performance in downstream information extraction tasks, the extraction task is included as one of the pre-training tasks. A matrix is ​​created for each schema, and the elements in the corresponding rows and columns are set to 1 according to the start and end position indices of the corresponding entities, transforming the labeled information into a multi-head matrix. This matrix is ​​compared with the prediction matrix output by the model, and the loss value can be calculated using the multi-label cross-entropy function.

[0167] Sequence labeling prediction task: Construct label vectors based on labeled data. This involves creating a vector of the same length as the text for each schema, setting the elements from the start to the end of the corresponding entity to 1, and the rest to 0. For this prediction task, the model needs to use an additional linear layer to transform the interactive text feature vectors to prevent interference with other tasks. The transformed feature vectors of each character are then used in matrix operations with the interactive text feature vectors to obtain the predicted entity fragments for each schema. This result is compared with the previously constructed label vectors, and the loss value can be calculated using the multi-label cross-entropy function.

[0168] Span Boundary Prediction Task: To enhance the model's ability to extract and utilize semantic information, Span Boundary Objective (SBO) is used as a pre-training task. The process involves randomly selecting several character segments from the text and masking them (i.e., replacing them with the special character "[MASK]"). The contaminated text is then input into the model. Two additional linear layers are added to the model, taking the interaction features of the characters on either side of the masked segment and the relative position encoding features as input, and outputting the predicted character category. Each predicted character is compared with the actual character, and the loss value can be calculated using the cross-entropy function.

[0169] In this embodiment of the invention, the schema identification task assists the model in discovering the differences between schemas during the model pre-training process.

[0170] Fragment extraction, fragment boundary, and sequence labeling prediction tasks can guide the model to learn the relevance between the schema and the corresponding entity context.

[0171] In this embodiment of the invention, the initial information extraction model is pre-trained based on four different pre-training tasks, which enables the general information extraction model obtained after pre-training to have the ability to accurately and quickly extract general schema entities.

[0172] Step 802: Calculate the loss value between the predicted data corresponding to the pre-training task and the real data corresponding to the pre-training task, wherein the real data corresponding to the pre-training task is determined based on the training labels;

[0173] Optionally, the loss values ​​between the predicted data and the real data corresponding to the schema identification task, the loss values ​​between the predicted data and the real data corresponding to the fragment extraction task, the loss values ​​between the predicted data and the real data corresponding to the sequence labeling prediction task, and the loss values ​​between the predicted data and the real data corresponding to the fragment boundary prediction task are calculated.

[0174] Step 803: Based on the loss value, perform parameter iteration on the initial information extraction model. After the iteration is completed, a general information extraction model is obtained.

[0175] During training, the parameters of the initial information extraction model are iterated to continuously reduce the model loss until convergence, thus completing the pre-training process and finally obtaining the general information extraction model.

[0176] Step 804: Fine-tune the general information extraction model based on the downstream extraction task to obtain the information extraction model.

[0177] Optionally, during the process of fine-tuning the general information extraction model based on the downstream extraction task, the above-mentioned fragment extraction task is executed to obtain the information extraction model.

[0178] In this embodiment of the invention, a schema identification task is designed to assist the model in discovering the differences between schemas during the model pre-training process. At the same time, combined with fragment extraction tasks, fragment boundary prediction tasks, and sequence labeling prediction tasks, the model is guided to learn the correlation between schema and corresponding entity context. The resulting general information extraction model has good entity extraction capabilities. After pre-training, the general information extraction model is fine-tuned based on downstream extraction tasks, which can quickly obtain an information extraction model that meets the requirements of downstream extraction tasks and effectively reduce the cost of implementing downstream extraction tasks.

[0179] Figure 9 This is an example diagram showing the extraction results of a general information extraction model provided in one embodiment of the present invention on the CLUENER dataset. Some results are shown below. Figure 9 As shown.

[0180] This invention uses the open-source data CLUENER as an example to illustrate the application process of the general information extraction model. The schemas in CLUENER containing "name, location, company, game, movie, book title, government, organization, position, and attraction" are identical in concept to the schemas with the same name in the schema tree constructed in this invention, forming an equivalent mapping. These ten schemas can be directly input into the schema-side encoder of the dual-tower general information extraction model described in this invention, and the text from CLUENER is sequentially input into the model's text-side encoder. After the model outputs the multi-head prediction matrix, the extraction result for each text can be obtained through decoding.

[0181] In this embodiment of the invention, it is disclosed that the open-source data CLUENER and the schema tree constructed in this invention can form an equivalent mapping, the data preprocessing process is relatively simple, and the computational load of the model can be effectively reduced.

[0182] The information extraction device provided in the embodiments of the present invention will be described below. The information extraction device described below can be referred to in correspondence with the information extraction method described above.

[0183] Figure 10 This is a schematic diagram of the structure of an information extraction device provided in one embodiment of the present invention, as shown below. Figure 10 As shown, the device 1000 includes:

[0184] The mapping relationship determination unit 1010 is used to determine the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established entity type tree, and to obtain the entity type mapping table.

[0185] The mapping unit 1020 is used to convert the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task based on the entity type mapping table.

[0186] The information extraction unit 1030 is used to extract information from the target text based on the entity type list to obtain entity information corresponding to the entity type list.

[0187] The inverse mapping unit 1040 is used to perform inverse mapping on the entity information corresponding to the entity type list based on the entity type mapping table to obtain the target entity information corresponding to the target entity type.

[0188] In some embodiments, the step of extracting information from the target text based on the entity type list to obtain entity information corresponding to the entity type list includes:

[0189] Input the entity type list and the target text into the information extraction model to obtain the entity information corresponding to the entity type list;

[0190] The information extraction model is trained using entity type samples and text samples as training samples, and the entity information corresponding to the entity type samples as training labels.

[0191] The entity type sample is determined based on the entity type tree.

[0192] In some embodiments, the information extraction model includes: an entity type-side encoder, a text-side encoder, a feature interaction module, and a global pointer module, wherein,

[0193] The entity type side encoder is used to extract features from the entity types input to the information extraction model;

[0194] The text-side encoder is used to extract features from the text input to the information extraction model;

[0195] The feature interaction module is used to perform feature interaction between the output of the entity type-side encoder and the output of the text-side encoder.

[0196] In some embodiments, the step of inputting the entity type list and the target text into the information extraction model to obtain entity information corresponding to the entity type list includes:

[0197] Based on the entity type side encoder, feature extraction is performed on the entity type list to obtain the entity type features corresponding to each entity type in the entity type list. Based on the text side encoder, feature extraction is performed on the target file to obtain the text features corresponding to the target text.

[0198] Based on the feature interaction module, feature interaction is performed on the entity type feature and text feature to obtain the interacted entity type feature and the interacted text feature.

[0199] Based on the global pointer module, feature mapping is performed on the entity type features and text features after the interaction to obtain a multi-head prediction matrix;

[0200] The multi-head prediction matrix is ​​decoded to obtain the entity information corresponding to the input entity type.

[0201] In some embodiments, the step of performing feature interaction on the entity type features and text features based on the feature interaction module to obtain the interacted entity type features and interacted text features includes:

[0202] The entity type features and text features are mapped respectively;

[0203] Perform matrix multiplication on the mapped entity type features and text features to obtain the first attention matrix;

[0204] The first attention matrix is ​​normalized along the dimensions of the entity type features and the text features, respectively, to obtain the second attention matrix and the third attention matrix;

[0205] The mapped entity type features are multiplied by the second attention matrix to obtain the interactive text features, and the mapped text features are multiplied by the third attention matrix to obtain the interactive entity type features.

[0206] In some embodiments, the step of performing feature mapping on the interacted entity type features and the interacted text features based on the global pointer module to obtain a multi-head prediction matrix includes:

[0207] Obtain the feature vector of the first character in the entity type feature after the interaction, and use it as the first entity type feature;

[0208] Map the features of the first entity type;

[0209] The text features after the interaction are mapped four times. The outputs of two mappings are multiplied by matrix to obtain the entity prediction matrix. The outputs of the other two mappings are multiplied by the first entity type features after mapping to obtain the first matrix and the second matrix. The second matrix is ​​transposed and then summed with the first matrix to obtain the multi-head matrix.

[0210] The entity prediction matrix and the multi-head matrix are summed to obtain the multi-head prediction matrix.

[0211] In some embodiments, the training process of the information extraction model includes:

[0212] Acquire training data, and preprocess the training data based on the entity type tree to obtain the training samples and training labels;

[0213] The training samples are input into the initial information extraction model to perform a pre-training task, and the prediction data corresponding to the pre-training task is obtained.

[0214] Calculate the loss value between the predicted data corresponding to the pre-training task and the real data corresponding to the pre-training task, wherein the real data corresponding to the pre-training task is determined based on the training labels;

[0215] Based on the loss value, the initial information extraction model is iterated through parameters. After the iteration is completed, a general information extraction model is obtained.

[0216] The general information extraction model is fine-tuned based on the downstream extraction task to obtain the information extraction model.

[0217] In some embodiments, the pre-training tasks include: entity type identification task, fragment extraction task, sequence labeling prediction task, and fragment boundary prediction task.

[0218] It should be noted that the information extraction device provided in this embodiment of the invention can implement all the method steps implemented in the above-described information extraction method embodiment and can achieve the same technical effect. Therefore, the parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0219] Figure 11 This is a schematic diagram of the physical structure of an electronic device provided in one embodiment of the present invention, as shown below. Figure 11 As shown, the electronic device may include a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 communicate with each other via the communication bus 1140. The processor 1110 can call logical instructions in the memory 1130 to execute an information extraction method. This method includes: determining the mapping relationship between the target entity type in the downstream extraction task and the entity type concepts in a pre-established entity type tree, obtaining an entity type mapping table; based on the entity type mapping table, converting the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task; based on the entity type list, performing information extraction on the target text to obtain entity information corresponding to the entity type list; and based on the entity type mapping table, performing a reverse mapping on the entity information corresponding to the entity type list to obtain the target entity information corresponding to the target entity type.

[0220] Furthermore, the logical instructions in the aforementioned memory 1130 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0221] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program is implemented to perform the information extraction method provided in the above-described method embodiments. The method includes: determining the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in a pre-established entity type tree to obtain an entity type mapping table; based on the entity type mapping table, converting the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task; based on the entity type list, performing information extraction on the target text to obtain entity information corresponding to the entity type list; and based on the entity type mapping table, performing inverse mapping on the entity information corresponding to the entity type list to obtain target entity information corresponding to the target entity type.

[0222] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0223] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0224] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An information extraction method characterized by, include: Determine the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established entity type tree to obtain the entity type mapping table; Based on the entity type mapping table, the target entity type is converted into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task; Based on the entity type list, information is extracted from the target text to obtain the entity information corresponding to the entity type list. Based on the entity type mapping table, the entity information corresponding to the entity type list is reverse-mapped to obtain the target entity information corresponding to the target entity type. The step of extracting information from the target text based on the entity type list to obtain the entity information corresponding to the entity type list includes: The entity type list and the target text are input into the information extraction model to obtain entity information corresponding to the entity type list. The information extraction model includes a feature interaction module and a global pointer module. The feature interaction module is used to perform feature interaction between the entity type features corresponding to each entity type in the entity type list and the text features corresponding to the target text to obtain the interacted entity type features and the interacted text features. The global pointer module is used to perform feature mapping on the output of the feature interaction module to output a multi-head prediction matrix. The global pointer module is specifically used for: Obtain the feature vector of the first character in the entity type feature after the interaction, and use it as the first entity type feature; Map the features of the first entity type; The text features after the interaction are mapped four times. The outputs of two mappings are multiplied by matrix to obtain the entity prediction matrix. The outputs of the other two mappings are multiplied by the first entity type features after mapping to obtain the first matrix and the second matrix. The second matrix is ​​transposed and then summed with the first matrix to obtain the multi-head matrix. The entity prediction matrix and the multi-head matrix are summed to obtain the multi-head prediction matrix.

2. The information extraction method according to claim 1, characterized in that, The information extraction model is trained using entity type samples and text samples as training samples, and the entity information corresponding to the entity type samples as training labels. The entity type sample is determined based on the entity type tree.

3. The information extraction method according to claim 2, characterized in that, The information extraction model also includes an entity type-side encoder and a text-side encoder, wherein... The entity type side encoder is used to extract features from the entity types input to the information extraction model; The text-side encoder is used to extract features from the text input to the information extraction model; The feature interaction module is used to perform feature interaction on the output of the entity type side encoder and the output of the text side encoder.

4. The information extraction method according to claim 3, characterized in that, The step of inputting the entity type list and target text into the information extraction model to obtain entity information corresponding to the entity type list includes: Based on the entity type side encoder, feature extraction is performed on the entity type list to obtain the entity type features corresponding to each entity type in the entity type list. Based on the text side encoder, feature extraction is performed on the target text to obtain the text features corresponding to the target text. Based on the feature interaction module, feature interaction is performed on the entity type feature and text feature to obtain the interacted entity type feature and interacted text feature. Based on the global pointer module, feature mapping is performed on the entity type features and text features after the interaction to obtain a multi-head prediction matrix; The multi-head prediction matrix is ​​decoded to obtain the entity information corresponding to the entity type list.

5. The information extraction method according to claim 4, characterized in that, The step of performing feature interaction on the entity type features and text features based on the feature interaction module to obtain the interacted entity type features and text features includes: The entity type features and text features are mapped respectively; Perform matrix multiplication on the mapped entity type features and text features to obtain the first attention matrix; The first attention matrix is ​​normalized along the dimensions of the entity type features and the text features, respectively, to obtain the second attention matrix and the third attention matrix; The mapped entity type features are multiplied by the second attention matrix to obtain the interactive text features, and the mapped text features are multiplied by the third attention matrix to obtain the interactive entity type features.

6. The information extraction method according to claim 2 or 3, characterized in that, The training process of the information extraction model includes: Acquire training data, and preprocess the training data based on the entity type tree to obtain the training samples and training labels; The training samples are input into the initial information extraction model to perform a pre-training task, and the prediction data corresponding to the pre-training task is obtained. Calculate the loss value between the predicted data corresponding to the pre-training task and the real data corresponding to the pre-training task, wherein the real data corresponding to the pre-training task is determined based on the training labels; Based on the loss value, the initial information extraction model is iterated through parameters. After the iteration is completed, a general information extraction model is obtained. The general information extraction model is fine-tuned based on the downstream extraction task to obtain the information extraction model.

7. The information extraction method according to claim 6, characterized in that, The pre-training tasks include: entity type identification, fragment extraction, sequence labeling prediction, and fragment boundary prediction.

8. An information extraction device, characterized in that, include: The mapping relationship determination unit is used to determine the mapping relationship between the target entity type in the downstream extraction task and the entity type concept in the pre-established entity type tree, and obtain the entity type mapping table. The mapping unit is used to convert the target entity type into a list of entity types compatible with the information extraction model corresponding to the downstream extraction task, based on the entity type mapping table. The information extraction unit is used to extract information from the target text based on the entity type list to obtain entity information corresponding to the entity type list. The inverse mapping unit is used to perform inverse mapping on the entity information corresponding to the entity type list based on the entity type mapping table to obtain the target entity information corresponding to the target entity type. Specifically, the information extraction unit is used for: The entity type list and the target text are input into the information extraction model to obtain entity information corresponding to the entity type list. The information extraction model includes a feature interaction module and a global pointer module. The feature interaction module is used to perform feature interaction between the entity type features corresponding to each entity type in the entity type list and the text features corresponding to the target text to obtain the interacted entity type features and the interacted text features. The global pointer module is used to perform feature mapping on the output of the feature interaction module to output a multi-head prediction matrix. The global pointer module is specifically used for: Obtain the feature vector of the first character in the entity type feature after the interaction, and use it as the first entity type feature; Map the features of the first entity type; The text features after the interaction are mapped four times. The outputs of two mappings are multiplied by matrix to obtain the entity prediction matrix. The outputs of the other two mappings are multiplied by the first entity type features after mapping to obtain the first matrix and the second matrix. The second matrix is ​​transposed and then summed with the first matrix to obtain the multi-head matrix. The entity prediction matrix and the multi-head matrix are summed to obtain the multi-head prediction matrix.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the information extraction method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the information extraction method as described in any one of claims 1 to 7.

Images