Entity relationship joint extraction method and device based on lightweight self-attention mechanism

Abstract

The present invention discloses a method and device for entity-relationship joint extraction based on a lightweight self-attention mechanism. The method comprises obtaining target sentence data and inputting the target sentence data into an entity classification model; classifying all subsequences of the target sentence data using the entity classification model to obtain entity sequences and non-entity sequences; obtaining any pair of entity pairs in the entity sequence and inputting the entity pairs and the word combinations between the entity pairs into a relationship classification model; and generating entity-relationship joint classification results using the entity-relationship joint extraction model; wherein the entity-relationship joint extraction model includes a Bert encoder, an entity classification model, and a relationship classification model. The present invention has low complexity, can improve the performance of entity-relationship models, and can be widely applied in the field of artificial intelligence technology.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for jointly extracting entity relationships based on a lightweight self-attention mechanism. Background Art

[0002] Named Entity Recognition: Named Entity Recognition, abbreviated as NER. Also known as proper name recognition or named entities, it refers to the identification of entities with specific meanings in text, mainly including names of people, places, organizations, proper nouns, as well as text such as time, quantity, currency, and proportional values. It refers to things that can be identified by proper nouns (names). A named entity generally represents a unique individual object, including names of people and places. NER is a subtask of classifying and locating named entity sentiments in unstructured text. Its process is to generate named entity expressions with proper noun annotation information from unstructured text expressions.

[0003] Relation extraction: Relation extraction involves extracting triples like (subject, relation, object) from a text. In English, this is called (subject, relation, object). Therefore, relation extraction is sometimes also called triple extraction. As can be seen from the definition of relation extraction, relation extraction primarily performs two tasks: identifying the subject and object in a text (entity recognition) and determining the relationship between these two entities (relation classification).

[0004] Self-attention: A machine learning technique that naturally learns about properties. Also known as internal attention, it is an attention mechanism that relates different positions of a single sequence to compute a representation of the same sequence. It has proven useful in machine reading, abstractive summarization, or image description generation.

[0005] Entity recognition and relationship extraction are key technologies for building knowledge graphs from natural language text data sources. With the development of social networks, the volume and speed of natural language text data are increasing rapidly, and new knowledge is constantly being generated. At the same time, updating knowledge graphs requires the addition of new knowledge. For knowledge graphs used in fake news detection, the speed of knowledge update is particularly important. However, to accelerate the acquisition of knowledge from rapidly growing natural language text data, a method is needed that can rapidly identify entities and relationships from natural language text data residing on various online platforms. This will help improve the speed of knowledge graph updates. Current entity and relationship extraction tasks typically use multi-head self-attention models. However, the classic multi-head self-attention model includes three transformation matrices and has a large number of parameters, resulting in high complexity, low training efficiency, and the need for large training datasets. Natural language processing models based on pre-trained models have achieved promising results in entity and relationship extraction tasks. However, due to the high training cost and model complexity of pre-trained models, they are difficult to deploy and apply on edge devices, making them unsuitable for rapidly updating knowledge graphs using online natural language text. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides a low-complexity entity relationship joint extraction method and device based on a lightweight self-attention mechanism, and improves the performance of the entity relationship model.

[0007] A first aspect of the present invention provides a method for joint entity relationship extraction based on a lightweight self-attention mechanism, comprising:

[0008] Obtain target sentence data, and input the target sentence data into an entity classification model;

[0009] Classify all subsequences of the target sentence data using the entity classification model to obtain entity sequences and non-entity sequences;

[0010] Obtain any pair of entity pairs in the entity sequence, and combine the entity pairs and the words between the entity pairs and input them into the relational classification model;

[0011] Generate entity relationship joint classification results through entity relationship joint extraction model;

[0012] The entity-relationship joint extraction model includes a Bert encoder, an entity classification model and a relationship classification model.

[0013] Optionally, acquiring target sentence data and inputting the target sentence data into an entity classification model includes:

[0014] Input the target sentence into the Bert encoder model for encoding to obtain an encoding result;

[0015] Splitting the encoding result into multiple subsequences to construct the target sentence data;

[0016] The target sentence data is input into the entity classification model.

[0017] Optionally, the entity classification model and the relationship classification model are implemented by a single-layer perceptron.

[0018] Optionally, the Bert encoder is a light attention encoder, which includes a plurality of encoder blocks and a text encoder module composed of a fully connected network layer;

[0019] When the input of the light attention encoder is a text vector, sentence encoding is performed after synchronously obtaining the one-dimensional position encoding information of the text vector;

[0020] After the sentence is encoded, an additional one-dimensional vector is added as global information to supplement the overall information of the sentence;

[0021] The vector sequence after sentence encoding is input into the span classifier for entity detection, and all detected subsequences are divided into entity types and non-entity types;

[0022] Combine entity pairs and the span between them into a long vector;

[0023] The long vector is input into a relation classifier to extract the relation in the long vector.

[0024] Optionally, the span classifier includes a fully connected mapping layer; the relation classifier includes a fully connected mapping layer;

[0025] The encoder block in the light attention encoder includes two submodules, one of which adopts the light self-attention mechanism, performs residual connection and outputs, and normalizes the output; the other submodule is a projection layer module, which includes a three-layer fully connected layer, where the dimension of the first fully connected layer is consistent with the input dimension, the dimension of the second fully connected layer is four times the input dimension, and the dimension of the third fully connected layer is consistent with the input dimension.

[0026] Optionally, the number of encoder blocks in the light attention encoder is dynamically updated according to the memory size.

[0027] Optionally, the processing of the light attention encoder includes:

[0028] Perform dot multiplication of the input feature matrix and the two transformation matrices to obtain Query and Key, and use the input feature matrix itself as Value;

[0029] Then perform cosine similarity calculation on the Query value and the Key value to obtain the similarity score;

[0030] Perform dot multiplication on the similarity scores to convert them into a matrix, and then use a sotfmax normalization to convert the resulting diagonal matrix into an irregular matrix, so as to act on the value of Value;

[0031] The final output value is the output of the self-attention layer. The output is connected to a residual and the layer is normalized.

[0032] Another aspect of the present invention further provides an entity relationship joint extraction device based on a lightweight self-attention mechanism, comprising:

[0033] The first module is used to obtain target sentence data and input the target sentence data into an entity classification model;

[0034] The second module is used to classify all subsequences of the target sentence data using the entity classification model to obtain entity sequences and non-entity sequences;

[0035] The third module is used to obtain any pair of entity pairs in the entity sequence, and combine the entity pairs and the words between the entity pairs and input them into the relational classification model;

[0036] The fourth module is used to generate entity relationship joint classification results through the entity relationship joint extraction model;

[0037] The entity-relationship joint extraction model includes a Bert encoder, an entity classification model and a relationship classification model.

[0038] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0039] The memory is used to store programs;

[0040] The processor executes the program to implement the method described above.

[0041] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0042] The present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the above method.

[0043] An embodiment of the present invention obtains target sentence data and inputs the target sentence data into an entity classification model; classifies all subsequences of the target sentence data using the entity classification model to obtain entity sequences and non-entity sequences; obtains any pair of entity pairs in the entity sequence, and inputs the entity pairs and the word combinations between the entity pairs into a relational classification model; and generates entity-relationship joint classification results using an entity-relationship joint extraction model; wherein the entity-relationship joint extraction model includes a Bert encoder, an entity classification model, and a relation classification model. The present invention has low complexity and can improve the performance of the entity-relationship model. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1 An overall step flow chart provided for an embodiment of the present invention;

[0046] Figure 2 A schematic diagram of the structure of the entity relationship extraction model provided by an embodiment of the present invention;

[0047] Figure 3 Flowchart of the operation process of the light self-attention module provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of this application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0049] To address the problems in the prior art, an embodiment of the present invention provides a method for joint entity relationship extraction based on a lightweight self-attention mechanism, including:

[0050] Obtain target sentence data, and input the target sentence data into an entity classification model;

[0051] Classify all subsequences of the target sentence data using the entity classification model to obtain entity sequences and non-entity sequences;

[0052] Obtain any pair of entity pairs in the entity sequence, and combine the entity pairs and the words between the entity pairs and input them into the relational classification model;

[0053] Generate entity relationship joint classification results through entity relationship joint extraction model;

[0054] The entity-relationship joint extraction model includes a Bert encoder, an entity classification model and a relationship classification model.

[0055] Optionally, acquiring target sentence data and inputting the target sentence data into an entity classification model includes:

[0056] Input the target sentence into the Bert encoder model for encoding to obtain an encoding result;

[0057] Splitting the encoding result into multiple subsequences to construct the target sentence data;

[0058] The target sentence data is input into the entity classification model.

[0059] Optionally, the entity classification model and the relationship classification model are implemented by a single-layer perceptron.

[0060] Optionally, the Bert encoder is a light attention encoder, which includes a plurality of encoder blocks and a text encoder module composed of a fully connected network layer;

[0061] When the input of the light attention encoder is a text vector, sentence encoding is performed after synchronously obtaining the one-dimensional position encoding information of the text vector;

[0062] After the sentence is encoded, an additional one-dimensional vector is added as global information to supplement the overall information of the sentence;

[0063] The vector sequence after sentence encoding is input into the span classifier for entity detection, and all detected subsequences are divided into entity types and non-entity types;

[0064] Combine entity pairs and the span between them into a long vector;

[0065] The long vector is input into a relation classifier to extract the relation in the long vector.

[0066] Optionally, the span classifier includes a fully connected mapping layer; the relation classifier includes a fully connected mapping layer;

[0067] The encoder block in the light attention encoder includes two submodules, one of which adopts the light self-attention mechanism, performs residual connection and outputs, and normalizes the output; the other submodule is a projection layer module, which includes a three-layer fully connected layer, where the dimension of the first fully connected layer is consistent with the input dimension, the dimension of the second fully connected layer is four times the input dimension, and the dimension of the third fully connected layer is consistent with the input dimension.

[0068] Optionally, the number of encoder blocks in the light attention encoder is dynamically updated according to the memory size.

[0069] Optionally, the processing of the light attention encoder includes:

[0070] Perform dot multiplication of the input feature matrix and the two transformation matrices to obtain Query and Key, and use the input feature matrix itself as Value;

[0071] Then perform cosine similarity calculation on the Query value and the Key value to obtain the similarity score;

[0072] Perform dot multiplication on the similarity scores to convert them into a matrix, and then use a sotfmax normalization to convert the resulting diagonal matrix into an irregular matrix, so as to act on the value of Value;

[0073] The final output value is the output of the self-attention layer. The output is connected to a residual and the layer is normalized.

[0074] Another aspect of the present invention further provides an entity relationship joint extraction device based on a lightweight self-attention mechanism, comprising:

[0075] The first module is used to obtain target sentence data and input the target sentence data into an entity classification model;

[0076] The second module is used to classify all subsequences of the target sentence data using the entity classification model to obtain entity sequences and non-entity sequences;

[0077] The third module is used to obtain any pair of entity pairs in the entity sequence, and combine the entity pairs and the words between the entity pairs and input them into the relational classification model;

[0078] The fourth module is used to generate entity relationship joint classification results through the entity relationship joint extraction model;

[0079] The entity-relationship joint extraction model includes a Bert encoder, an entity classification model and a relationship classification model.

[0080] Another aspect of an embodiment of the present invention further provides an electronic device, including a processor and a memory;

[0081] The memory is used to store programs;

[0082] The processor executes the program to implement the method described above.

[0083] Another aspect of the embodiments of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a program, and the program is executed by a processor to implement the method described above.

[0084] The present invention also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium and execute the computer instructions, causing the computer device to perform the above method.

[0085] The specific implementation process of the present invention is described in detail below with reference to the accompanying drawings:

[0086] The current mainstream entity extraction model has a large number of model parameters and is conveniently deployed on edge devices to process some knowledge graph tasks that require rapid data collection and updating. In order to solve the problems existing in the prior art, the present invention proposes a novel span-based joint entity and relationship extraction method with a light attention encoder (the model name is SpELA). In order to reduce the complexity of the model, better apply the relevant model to small data sets, and deploy it on edge devices, this paper proposes a light self-attention mechanism based on a multi-head attention mechanism. The V transformation matrix is simplified above, and the K and Q transformation matrices obtain the information learned in the simplified transformation matrix by learning their respective parameters. The cosine similarity projection matrix is used to map the input to the feature space of K and Q. The present invention proposes a novel light attention encoder based on a span-based joint entity relationship method, which improves the performance of the entity relationship model while reducing the complexity of the model, so that it can be used on edge devices with smaller memory.

[0087] like Figure 1 As shown, the overall implementation steps of the present invention include: obtaining target sentence data and inputting the target sentence data into an entity classification model; classifying all subsequences of the target sentence data through the entity classification model to obtain entity sequences and non-entity sequences; obtaining any pair of entity pairs in the entity sequence, and combining the entity pairs and the words between the entity pairs into a relational classification model; generating a classification result of an entity-relationship joint through an entity-relationship joint extraction model; wherein the entity-relationship joint extraction model includes a Bert encoder, an entity classification model and a relational classification model.

[0088] It should be noted that the entity classification model and the relationship classification model of this embodiment are components of the entity-relationship joint extraction model, which is composed of the Bert encoder, the entity classification model, and the relationship classification model.

[0089] Among them, reference Figure 1 The whole process can be described as follows: first, a sentence is input into the Bert encoder for encoding. Then the model splits the encoding result of a sentence into multiple subsequences. Then, each subsequence is classified through an entity neural network classification model, and the category with low probability is classified as the none category. Then, all subsequences other than the none category are taken out, and the words between the two subsequences and the two subsequences are spliced together. Then, the sentence becomes a joint entity. Then, the sentence is classified by another relational neural network classification model to determine the relationship between them. If there is no relationship, it is classified as the none category. If there is a relationship, the relationship between the entities is output to determine the relationship between the entities.

[0090] The following takes the sentence "Qilixiang is Jay Chou's song" as an input example to explain the specific implementation process of the present invention in detail: First, there is a sentence "Qilixiang is Jay Chou's song" in which Qilixiang and Jay Chou are two entity classes that have been labeled in advance. Then the sentence is encoded through the Bert encoder. For ease of understanding, the encoded sentence is still: Qilixiang is Jay Chou's song. Then this sentence is divided into n subsequences, such as: Qilixiang, Qilixiang is, Jay Chou, Jay Chou's song... Then they are all taken out and classified through the entity classifier. The sequences with the classification results as entities are separately proposed, and those that are not are discarded, such as: Qilixiang is an entity, proposed, Jay Chou's song is not an entity, discarded, Jay Chou is an entity, proposed. Finally, the entity classification model obtains the two entities Qilixiang and Jay Chou, and then splices the words between them together to become, such as: Qilixiang is Jay Chou, then this sentence is a joint entity, and finally it is passed through the relationship classification model to output the relationship to which this sentence belongs, such as the author relationship. If there is no relationship, output none.

[0091] Among them, the entity neural network classification model and the relational neural network classification model are both implemented by single-layer perceptrons.

[0092] Figure 2This is our proposed span-based entity relation extraction model, in which the light-attention encoder is a text encoder module consisting of n encoder blocks and a fully connected network layer. When a text vector (as shown in the rectangle at the bottom) is input, one-dimensional positional encoding information is added to the group to ensure that the model can learn the context of the sentence. After the sentence is encoded, an additional one-dimensional vector CLS is added as global information to supplement the learned overall information of the sentence. The span classifier is a span-based entity relation classification module, which is a fully connected mapping layer. When an encoded vector sequence enters it, it performs entity detection on the vector subsequences. It then classifies all detected subsequences into entity types and filters out non-entities. After this, the remaining entity pairs and the spans between them are combined into a long vector. The long vector is input to the relation classifier, which extracts relations from the long vector using a fully connected mapping layer.

[0093] The encoder block in the light-attention encoder consists of two modules. The first module is a variant of the self-attention mechanism, called the light self-attention mechanism. Its output is normalized and residual connections are performed to form the output. The second module is a projection layer module. Its specific structure is a three-layer fully connected layer. The first layer has the same dimension as the input, the second layer has a dimension four times that of the input layer, and the final layer is projected back to the input layer dimension. The output is then normalized and residual connections are performed to form the output. These two modules form an encoder block, and the number of blocks can be dynamically selected based on the device's video memory size.

[0094] like Figure 3 As shown, when the feature matrix is input, it will first be multiplied by the two transformation matrices to obtain the query and key, and the feature matrix itself will be used as the value. The formula is as follows:

[0095] Q=W q (I), K=W k (I), V=I

[0096] Among them: Q stands for Query, K stands for Key, V stands for Value, and I stands for Input.

[0097] Then, the cosine similarity calculation is performed on the Q value and the K value to obtain the similarity score. The formula is as follows:

[0098]

[0099] Where: α i Represents the similarity score.

[0100] Since the similarity score is a column vector and the V value is a matrix, if you want to apply the similarity score to the matrix, you need to perform a dot product on the similarity score to convert it into a matrix, and then use a sotfmax normalization to convert the obtained diagonal matrix into an irregular matrix, so as to apply it to the V value. The formula is as follows:

[0101] P=α·α T , attn=softmax(P), output=attn·V

[0102] Among them: output represents the output matrix

[0103] The final output value V is the output of the self-attention layer. In order to reduce problems such as gradient diffusion, the output is connected to a residual and the layer is normalized. The formula is as follows:

[0104] X=Norm(output+I)

[0105] Where: X represents the input matrix of the next layer module

[0106] Application scenarios of the invention:

[0107] When applying knowledge graphs to fake news detection and public opinion detection, the speed of knowledge updates is particularly important. However, to accelerate the acquisition of knowledge from rapidly growing natural language text data, a method is needed that can quickly identify entities and relationships within natural language text data residing on various online platforms. This will help increase the speed of knowledge graph updates. Furthermore, public opinion on the internet ferments rapidly today, necessitating a model that can quickly model public opinion. However, existing models have excessively large parameters. Our proposed model reduces these parameters and can quickly model public opinion, thereby guiding public opinion in the right direction.

[0108] In summary, the present invention can significantly reduce the number of model parameters, allowing the model to be deployed on edge devices, and changes the similarity calculation method, making it more suitable for language data tasks, making the model performance more outstanding.

[0109] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0110] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0111] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0112] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0113] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0114] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0115] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0116] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0117] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. Entity relationship joint extraction method based on lightweight self-attention mechanism, characterized by: include: Obtain target sentence data, and input the target sentence data into an entity classification model; Classify all subsequences of the target sentence data using the entity classification model to obtain entity sequences and non-entity sequences; Obtain any pair of entity pairs in the entity sequence, and combine the entity pairs and the words between the entity pairs and input them into the relational classification model; Generate entity relationship joint classification results through entity relationship joint extraction model; The entity-relationship joint extraction model includes a Bert encoder, an entity classification model and a relationship classification model; The Bert encoder is a light attention encoder, which includes multiple encoder blocks and a text encoder module composed of a fully connected network layer; When the input of the light attention encoder is a text vector, sentence encoding is performed after synchronously obtaining the one-dimensional position encoding information of the text vector; After the sentence is encoded, an additional one-dimensional vector is added as global information to supplement the overall information of the sentence; The vector sequence after sentence encoding is input into the span classifier for entity detection, and all detected subsequences are divided into entity types and non-entity types; Combine entity pairs and the span between them into a long vector; The long vector is input into a relation classifier to extract the relation in the long vector.

2. The entity relationship joint extraction method based on lightweight self-attention mechanism according to claim 1 is characterized in that: The acquiring target sentence data and inputting the target sentence data into the entity classification model includes: Input the target sentence into the Bert encoder model for encoding to obtain an encoding result; Splitting the encoding result into multiple subsequences to construct the target sentence data; The target sentence data is input into the entity classification model.

3. The entity relationship joint extraction method based on lightweight self-attention mechanism according to claim 1 is characterized in that: The entity classification model and the relationship classification model are implemented by a single-layer perceptron.

4. The entity relationship joint extraction method based on lightweight self-attention mechanism according to claim 1 is characterized in that: The span classifier includes a fully connected mapping layer; the relation classifier includes a fully connected mapping layer; The encoder block in the light attention encoder includes two submodules, one of which adopts the light self-attention mechanism, performs residual connection and outputs, and normalizes the output; the other submodule is a projection layer module, which includes a three-layer fully connected layer, where the dimension of the first fully connected layer is consistent with the input dimension, the dimension of the second fully connected layer is four times the input dimension, and the dimension of the third fully connected layer is consistent with the input dimension.

5. The entity relationship joint extraction method based on lightweight self-attention mechanism according to claim 4 is characterized in that: The number of encoder blocks in the described light attention encoder is dynamically updated according to the memory size.

6. The entity relationship joint extraction method based on lightweight self-attention mechanism according to claim 5 is characterized in that: The processing of the light attention encoder includes: Perform dot multiplication of the input feature matrix and the two transformation matrices to obtain Query and Key, and use the input feature matrix itself as Value; Then perform cosine similarity calculation on the Query value and the Key value to obtain the similarity score; Perform dot multiplication on the similarity scores to convert them into a matrix, and then use a sotfmax normalization to convert the resulting diagonal matrix into an irregular matrix, so as to act on the value of Value; The final output value is the output of the self-attention layer. The output is connected to a residual and the layer is normalized.

7. Entity relationship joint extraction device based on lightweight self-attention mechanism, characterized by: include: The first module is used to obtain target sentence data and input the target sentence data into an entity classification model; The second module is used to classify all subsequences of the target sentence data using the entity classification model to obtain entity sequences and non-entity sequences; The third module is used to obtain any pair of entity pairs in the entity sequence, and combine the entity pairs and the words between the entity pairs and input them into the relational classification model; The fourth module is used to generate entity relationship joint classification results through the entity relationship joint extraction model; The entity-relationship joint extraction model includes a Bert encoder, an entity classification model and a relationship classification model; The Bert encoder is a light attention encoder, which includes multiple encoder blocks and a text encoder module composed of a fully connected network layer; When the input of the light attention encoder is a text vector, sentence encoding is performed after synchronously obtaining the one-dimensional position encoding information of the text vector; After the sentence is encoded, an additional one-dimensional vector is added as global information to supplement the overall information of the sentence; The vector sequence after sentence encoding is input into the span classifier for entity detection, and all detected subsequences are divided into entity types and non-entity types; Combine entity pairs and the span between them into a long vector; The long vector is input into a relation classifier to extract the relation in the long vector.

8. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 6.

Images