Entity relationship identification method, device and readable storage medium
By constructing a loss function loop training entity relationship recognition model, the poor recognition effect caused by unbalanced data in multi-label classification in the prior art is solved, the recognition ability of fewer data categories is improved, and the overall recognition effect is enhanced.
Patent Information
- Application Number
- CN202210042332.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-14
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-01-14
AI Technical Summary
When the existing multi-label classification method processes unbalanced data, the model has poor recognition effect on categories with fewer training data, affecting the overall entity relationship recognition effect.
By constructing a loss function loop training entity relationship recognition model, using the preset weight matrix to weight sum the entity vector, combining the activation function to obtain the prediction probability of the relationship category, and implementing iterative training of the model by adjusting the loss function value until the loss function The value converges to the preset range.
The model's ability to identify categories with fewer data in multi-label classification is improved, and the accuracy and balance of overall entity relationship recognition is enhanced, ensuring that the model can effectively learn and identify relationship categories with a small sample size.
Smart Images

Figure CN114385817B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and computer-readable storage medium for identifying entity relationships. Background Art
[0002] Knowledge graph is an important area in the field of NLP, which aims to realize a smarter search engine. With the development of technology, it can be applied to scenarios such as intelligent search, intelligent question and answer, and personalized recommendation. Therefore, how to build knowledge graph has become a hot topic in the field of NLP.
[0003] The first step in building a knowledge graph is to extract information from the text. A key technology for information extraction is relationship extraction. After identifying discrete entities in the text, it is necessary to identify the relationships between entities. Since there may be multiple relationships between entities, this is a multi-label classification problem. The current method for extracting relationships at the chapter level is to use a dynamic threshold method to transform the loss function. For each sample, there will be a threshold. After the model gives the probability that this entity pair belongs to each relationship, a dynamic threshold will be given. The relationship categories above this threshold are all categories of this entity pair.
[0004] However, one problem that the above methods do not address is that the current multi-label classification problem is imbalanced. Imbalanced data will cause the model to perform poorly when identifying categories with less training data, resulting in poor overall recognition results.
[0005] In order to solve the above problems, the present invention urgently needs to provide a method for identifying entity relationships. Summary of the invention
[0006] The present invention provides an entity relationship recognition method, device, electronic device and computer-readable storage medium, the main purpose of which is to solve the problem of poor recognition effect in the existing entity recognition process by cyclically training an entity relationship recognition model through a constructed loss function.
[0007] To achieve the above object, the present invention provides a method for identifying entity relationships, which is applied to electronic devices. The method includes:
[0008] Preprocessing the training sample using a pre-built entity relationship recognition model to obtain a word vector for each word in the training sample;
[0009] According to the obtained word vectors, the entity pairs of the training sample are processed to obtain the entity vectors of each entity in all entity pairs of the training sample, wherein the entity pairs are composed of any two entities of the training sample;
[0010] Performing weighted summation processing on the word vectors of each entity of the entity pair using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair;
[0011] Processing each entity vector of the entity pair and the additional feature vector through an activation function to obtain a predicted probability of a relationship category of the entity pair;
[0012] The predicted probability of the entity-to-relationship category is processed by using a pre-constructed loss function to obtain a relationship category loss function value;
[0013] Circularly acquiring the relationship category loss function value until the relationship category loss function value converges to a preset range, so as to complete the iterative training of the entity relationship recognition model;
[0014] Entity relationship recognition is performed on the text to be recognized through the trained entity relationship recognition model.
[0015] Optionally, the preprocessing of the training sample using the pre-built entity relationship recognition model to obtain the word vector of each word in the training sample includes:
[0016] Obtaining a target word in the training sample and determining an initial word vector corresponding to the target word;
[0017] Determine the image feature vector, the root feature vector, and the pinyin feature vector corresponding to the target character according to the initial character vector;
[0018] A word vector corresponding to the target word is generated according to the target initial word vector, the image feature vector, the root feature vector, the pinyin feature vector and the preset weight matrix.
[0019] Optionally,
[0020] The step of processing the entity pairs of the training sample according to the obtained word vectors to obtain the entity vector of each entity in all entity pairs of the training sample includes:
[0021] According to the obtained word vector, obtain the word vector of each word in the training sample;
[0022] According to the obtained word vector, the entity vector in the training sample is obtained.
[0023] Optionally,
[0024] The step of performing weighted summation processing on the word vectors of each entity of the entity pair by using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair includes:
[0025] Processing the entity vector of the head entity in the entity pair by using a preset weight matrix to obtain the weight of the head entity in the entity pair;
[0026] Processing the entity vector of the tail entity in the entity pair by using a preset weight matrix to obtain the weight of the tail entity in the entity pair;
[0027] According to the weight of the head entity in the entity pair and the weight of the tail entity in the entity pair, an additional feature vector of each entity in the entity pair is obtained.
[0028] Optionally, the processing each entity vector of the entity pair and the additional feature vector by using an activation function to obtain a predicted probability of a relationship category of the entity pair includes:
[0029] Processing each entity vector and the additional feature vector of the entity pair respectively through a tanh activation function to obtain a tanh result value of each entity respectively;
[0030] The obtained tanh result value of each entity is multiplied and then input into the sigmoid function to obtain the predicted probability of the entity to the relationship category.
[0031] Optionally, the looping to obtain the relationship category loss function value, when the relationship category loss function value converges to a preset range, stops looping to obtain the relationship category loss function value to complete iterative training of the constructed entity relationship recognition model, including:
[0032] When the relationship type loss function value is lower than or equal to the preset category value, continue to loop to obtain the relationship type loss function value;
[0033] When the relationship type loss function value is higher than the preset category value, the acquisition cycle is stopped to obtain the relationship type loss function value, and the training of the entity relationship recognition model is completed. Optionally, the constructed loss function formula is: L = L1 + L2
[0034]
[0035]
[0036] Among them, L represents the loss function; L1 represents the loss function of one entity in the entity pair; L2 represents the loss function of the other entity in the entity pair; P represents the set of categories to which the entity pair belongs; N represents the category other than the entity pair; TH represents the preset category value; σ represents the sigmoid function; r represents a certain relationship; r' represents another relationship, and K represents a parameter.
[0037] In order to solve the above problem, the present invention further provides a device for identifying a base entity relationship, the device comprising:
[0038] A word vector acquisition module, used to pre-process the training sample using a pre-built entity relationship recognition model to obtain a word vector for each word in the training sample;
[0039] An entity vector acquisition module, used to process the entity pairs of the training sample according to the acquired word vectors, and acquire the entity vector of each entity in all entity pairs of the training sample, wherein the entity pair is composed of any two entities of the training sample;
[0040] An additional feature vector acquisition module, used to perform weighted summation processing on the word vectors of each entity of the entity pair through a preset weight matrix to obtain an additional feature vector of each entity of the entity pair;
[0041] A prediction vector acquisition module, used to process each entity vector of the entity pair and the additional feature vector through an activation function to obtain a prediction probability of the relationship category of the entity pair;
[0042] A relationship category loss function value acquisition module, used to process the predicted probability of the entity to the relationship category through a pre-built loss function acquisition to obtain a relationship category loss function value;
[0043] A model training completion module, used for cyclically obtaining the relationship category loss function value until the relationship category loss function value converges to a preset range, so as to complete the iterative training of the entity relationship recognition model;
[0044] The relationship category recognition module is used to perform entity relationship recognition on the text to be recognized through the trained entity relationship recognition model.
[0045] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:
[0046] at least one processor; and,
[0047] a memory communicatively connected to the at least one processor; wherein,
[0048] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the above-mentioned entity relationship identification method.
[0049] In order to solve the above problem, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned entity relationship identification method.
[0050] The embodiment of the present invention uses a pre-constructed entity relationship recognition model to pre-process the training sample, and obtain the word vector of each word in the training sample; according to the obtained word vector, the entity pair of the training sample is processed to obtain the entity vector of each entity in all entity pairs of the training sample, wherein the entity pair is composed of any two entities of the training sample; the word vector of each entity of the entity pair is weighted and summed by a preset weight matrix to obtain an additional feature vector of each entity of the entity pair; each entity vector of the entity pair and the additional feature vector are processed by an activation function to obtain the predicted probability of the relationship category of the entity pair; the predicted probability of the relationship category of the entity pair is processed by a pre-constructed loss function to obtain the relationship category loss function value; the relationship category loss function value is cyclically obtained until the relationship category loss function value converges to a preset range to complete the iterative training of the entity relationship recognition model; the entity relationship recognition of the to-be-recognized text is performed by the trained entity relationship recognition model. The main purpose of the present invention is to cyclically train the entity relationship recognition model through the constructed loss function, so as to solve the problem of poor recognition effect in the existing entity recognition process. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Figure 1 A schematic diagram of a flow chart of a method for identifying entity relationships provided by an embodiment of the present invention;
[0052] Figure 2 A schematic diagram of a module of an entity relationship recognition device provided by an embodiment of the present invention;
[0053] Figure 3 A schematic diagram of the internal structure of an electronic device for implementing a method for identifying entity relationships provided by an embodiment of the present invention.
[0054] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0055] In the following description, many specific details are set forth. However, it should be understood that embodiments of the present invention can be implemented without these specific details. In other instances, well-known methods, structures, and techniques are not shown in detail to avoid blurring the understanding of this specification. References to "one embodiment," "embodiment," "exemplary embodiments," "various embodiments," etc. indicate that the embodiment described in the present invention may include specific features, structures, or characteristics, but not every embodiment must include the specific features, structures, or characteristics. In addition, the repeated use of the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may be.
[0056] As used herein, unless otherwise indicated, the use of ordinal adjectives "first," "second," "third," etc. to describe common objects merely indicates that different instances of similar objects are being referenced, and is not intended to imply that the objects so described must be in a given order, whether in time, space, sequence, or any other manner.
[0057] As used herein, unless otherwise indicated, the terms "mobile device" and / or "device" generally refer to wireless communication devices, and more specifically refer to one or more of the following: a portable electronic device, a telephone (e.g., a cellular telephone, a smartphone), a computer (e.g., a laptop, a tablet computer), a portable media player, a personal digital assistant (PDA), or any other electronic device with networking capabilities.
[0058] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.
[0059] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0060] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.
[0061] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.
[0062] Machine Learning (ML) is a multi-disciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.
[0063] The present invention provides a method for identifying entity relationships. Figure 1 FIG. 1 is a flow chart of a method for identifying entity relationships provided by an embodiment of the present invention. The method can be executed by a device, and the device can be implemented by software and / or hardware.
[0064] In this embodiment, the entity relationship identification method includes:
[0065] S1: preprocessing the training sample using a pre-built entity relationship recognition model to obtain a word vector for each word in the training sample;
[0066] S2: Processing the entity pairs of the training sample according to the obtained word vectors to obtain the entity vectors of each entity in all entity pairs of the training sample, wherein the entity pairs are composed of any two entities of the training sample;
[0067] S3: performing weighted summation processing on the word vectors of each entity of the entity pair using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair;
[0068] S4: Processing each entity vector of the entity pair and the additional feature vector through an activation function to obtain a predicted probability of the relationship category of the entity pair;
[0069] S5: Processing the predicted probability of the entity-to-relationship category through a pre-constructed loss function to obtain a relationship category loss function value;
[0070] S6: cyclically obtaining the relationship category loss function value until the relationship category loss function value converges to a preset range, so as to complete the iterative training of the entity relationship recognition model;
[0071] S7: Perform entity relationship recognition on the text to be recognized through the trained entity relationship recognition model.
[0072] In an embodiment of the present invention, the entity relationship recognition model is cyclically trained through a constructed loss function, thereby solving the problem of poor recognition effect in the existing entity recognition process.
[0073] In the embodiment of the present invention, it is assumed that a document Among them, l represents the length of the document, t represents the tth character, and for each entity in the document, the special symbol * is inserted at the beginning and end of the entity.
[0074] In step S1, the pre-constructed entity relationship recognition model is used to pre-process the training sample to obtain the word vector of each word in the training sample, including:
[0075] Obtaining a target word in the training sample and determining an initial word vector corresponding to the target word;
[0076] Determine the image feature vector, the root feature vector, and the pinyin feature vector corresponding to the target character according to the initial character vector;
[0077] A word vector corresponding to the target word is generated according to the target initial word vector, the image feature vector, the root feature vector, the pinyin feature vector and the preset weight matrix.
[0078] The constructed entity relationship recognition model is used to preprocess the training sample to obtain the word vector of each word in the training sample. The formula used to obtain the word vector of each word in the training sample is:
[0079] [h1,h2,…,h l ]=BERT(|x1,x2,…,x l |)
[0080] Among them, h1,h2,…,h l Represents the word vector of each word in the training sample; x1,x2,…,xl represents each word in the training sample; l represents the length of the training sample.
[0081] In an embodiment of the present invention, BERT is a model for obtaining word vectors trained with massive data. By inputting each word in a sentence into the BERT model, the vector of each word in the last hidden layer of the BERT model can be obtained, and this vector can be used as the vector of each word. Among them, for the vector representation of the entity, when inputting it into the entity relationship recognition model, "*" is inserted before the token of the entity, and finally the vector corresponding to "*" is used as the vector representation of the entity. Since an entity may appear multiple times in the text, multiple vector representations of this entity can be obtained accordingly. For the multiple vector representations of this entity, the formula of step S2 is used to calculate, and the result of this formula is used as the final vector representation of this entity.
[0082] In step S2, the entity pairs of the training sample are processed according to the obtained word vectors to obtain the entity vectors of each entity in all entity pairs of the training sample, including:
[0083] According to the obtained word vector, obtain the word vector of each word in the training sample;
[0084] According to the obtained word vector, the entity vector in the training sample is obtained.
[0085] For any entity e i , assuming that the entity has N sub-symbols, then the final vector of the entity can be expressed as the following formula, that is: the formula used to obtain the entity vector of each entity in all entity pairs of the training sample is:
[0086]
[0087] in, Represents the entity vector at the jth position appearing in the training sample;
[0088] Indicates that the entity appears N times in the training sample.
[0089] In step S3, the word vectors of each entity of the entity pair are weighted and summed using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair, including:
[0090] Processing the entity vector of the head entity in the entity pair by using a preset weight matrix to obtain the weight of the head entity in the entity pair;
[0091] Processing the entity vector of the tail entity in the entity pair by using a preset weight matrix to obtain the weight of the tail entity in the entity pair;
[0092] Obtain the additional feature vector of each entity of the entity pair according to the weight of the head entity in the entity pair and the weight of the tail entity in the entity pair.
[0093] Among them, the formula for obtaining the additional feature vector of each entity of the entity pair is:
[0094] A (s,o) = A s ·A o
[0095]
[0096] Among them, A s represents the weight of the head entity S in the entity pair with respect to other entities;
[0097] A o represents the weight of the tail entity O in the entity pair with respect to other entities;
[0098] A (s,o) represents the weight of the entity pair with respect to all other tokens;
[0099] H represents the number of head entities;
[0100] q (s,o) represents the weight of the entity pair of H head entities with respect to other tokens.
[0101] In the embodiment of the present invention, according to the entity relationship recognition model, the weight matrix A ijk of the last layer of tokens and tokens can be obtained, where 0 < i <= H, H represents the number of multi_heads in Bert, and j and k are both 0 < j, k <= l. Through the weight matrix, the weight of the head entity S in the entity pair with respect to other entities can be obtained as A s , and the weight of the tail entity in the entity pair with respect to other entities is A o . Combining the two entities, the weight of this entity pair with respect to all other tokens is:
[0102] A (s,o) = A s ·A o
[0103] Since the entity pair has H heads, the final weight of the entity pair with respect to other tokens is:
[0104]
[0105] Furthermore, according to q (s,o)The topk most important words for this entity pair are taken from large to small, and then weighted summed with the corresponding vectors of these topk words to obtain an additional feature vector c (s,o) .
[0106] In step S4, the activation function is used to process each entity vector of the entity pair and the additional feature vector to obtain the predicted probability of the relationship category of the entity pair, including:
[0107] S41: processing each entity vector and the additional feature vector of the entity pair respectively through a tanh activation function to obtain a tanh result value of each entity respectively;
[0108] S42: The obtained tanh result value of each entity is multiplied, and then input into the sigmoid function to obtain the predicted probability of the entity to the relationship category.
[0109] For an entity pair (e s ,e o ), according to the obtained entity vector Finally, the predicted probability of the entity pair with respect to a certain relationship can be expressed as the following formula, namely: the formula used to obtain the predicted probability of the entity pair relationship category is:
[0110]
[0111]
[0112]
[0113] Among them, Z s Indicates e s The entity vector is the vector after passing through the fully connected layer and then the activation function;
[0114] W s represents the parameter matrix; Z o Indicates e o The entity vector is the vector after passing through the fully connected layer and then the activation function; b r represents the bias term; σ represents the sigmoid function; Represents Z s The transposed matrix of r represents a parameter matrix; r represents a certain relationship.
[0115] In step S5, the loss function formula constructed is: L = L1 + L2
[0116]
[0117]
[0118] Among them, L represents the loss function; L1 represents the loss function of one entity in the entity pair; L2 represents the loss function of the other entity in the entity pair; P represents the set of categories to which the entity pair belongs; N represents the category other than the entity pair; TH represents the preset category value; σ represents the sigmoid function; r represents a certain relationship; K represents a parameter.
[0119] In step S6, the loop obtains the relationship category loss function value, and when the relationship category loss function value converges to a preset range, the loop is stopped to obtain the relationship category loss function value to complete the iterative training of the constructed entity relationship recognition model, including:
[0120] S61: When the relationship type loss function value is lower than or equal to the preset category value, continue to loop to obtain the relationship type loss function value;
[0121] S62: When the relationship type loss function value is higher than a preset category value, stop looping to obtain the relationship type loss function value, and complete the training of the entity relationship recognition model.
[0122] In an embodiment of the present invention, based on the loss function L, the basis for judging whether an entity pair belongs to the relationship category is whether the score of the entity pair in this category is higher than the score of the threshold category. According to this loss function, if the score of this entity pair in this category is much higher than the threshold category, it means that the entity relationship recognition model has learned how to judge this category, then the loss function value of this category is lower, and the entity relationship recognition model will then pay more attention to other categories that have not been learned well.
[0123] If the score of the entity pair in this category is very close to the threshold, it means that the entity relationship recognition model has not yet learned how to judge this category, so the loss function value is higher, so that the entity relationship recognition model pays more attention to this category in the subsequent learning. That is, through the loss function σ(K(TH-logit r )) and σ(K(logit r -TH)) are used to control the two items. For positive samples, the score logit of relationship r r , the score of the preset category value is TH, and the sigmoid function is a monotonically increasing function, so TH-logit r The smaller the logit r The larger the value is, the smaller the TH is, and the loss function under this category will be reduced. Finally, based on the improvement of this loss function, the trained entity relationship recognition model can learn how to distinguish relationship categories with a large number of samples, as well as how to distinguish relationship categories with a small number of samples.
[0124] In an embodiment of the present invention, under the loss function L=L1+L2, when the entity relationship recognition model is trained, if the predicted probability that the sample belongs to a certain category is very different from the probability of a preset category, the proportion of the loss function of this category in the total loss function is reduced. In this way, the proportion of difficult-to-learn samples in the loss function of the model will increase. Since the optimization goal of model training is to minimize the loss function, this will allow the model to learn more features of difficult-to-distinguish samples, thereby increasing the model's recognition ability of these difficult-to-distinguish samples.
[0125] At the same time, in the extraction of chapter relations, the relationship between two related entities is mainly reflected in certain words in the article, which can be used as evidence that the two entities belong to this relationship. Therefore, if evidence can be found that the two entities belong to this relationship, it will be of great help to the final prediction. Since the current language model has better semantic representation capabilities through training on a large amount of text, introducing the language model into downstream tasks will improve the recognition results of downstream tasks. Therefore, current NLP tasks will use the language model as the bottom-level structure to encode the text. It is found that an intermediate result in the language model is the correlation between each character and other characters. Then, through this intermediate result, we can get which characters are related to the two entities, and take out the TOPK characters according to the correlation as evidence for distinguishing the two entities.
[0126] In step S7, the trained entity relationship recognition model is used to perform entity relationship recognition on the text to be recognized, and the relationship category of the entity pair is obtained. That is, after the model is trained based on the loss function, for a new article and entity pair, the relationship category of the entity pair can be predicted according to the above steps.
[0127] The embodiment of the present invention uses the constructed entity relationship recognition model to preprocess the training sample, and obtain the word vector of each word of the training sample; according to the obtained word vector, the entity pair of the training sample is processed to obtain the entity vector of each entity in all entity pairs of the training sample; the word vector of each entity of the entity pair is weighted and summed by the weight matrix to obtain the additional feature vector of each entity of the entity pair; each entity vector of the entity pair and the additional feature vector are processed by the activation function to obtain the predicted probability of the relationship category of the entity pair; the predicted probability of the relationship category of the entity pair is processed by the constructed loss function to obtain the relationship category loss function value; the relationship category loss function value is obtained cyclically, and when the relationship category loss function value converges to a preset range, the relationship category loss function value is stopped to complete the iterative training of the constructed entity relationship recognition model; the entity relationship recognition of the to-be-recognized text is performed by the trained entity relationship recognition model to obtain the relationship category of the entity pair. The main purpose of the present invention is to cyclically train the entity relationship recognition model by the constructed loss function, so as to solve the problem of poor recognition effect in the existing entity recognition process.
[0128] like Figure 2 As shown, it is a functional module diagram of the entity relationship recognition device of the present invention. The entity relationship recognition device 100 of the present invention can be installed in an electronic device. According to the implemented functions, the entity relationship recognition device 100 may include: a word vector acquisition module 101, an entity vector acquisition module 102, an additional feature vector acquisition module 103, a prediction vector acquisition module 104, a relationship category loss function value acquisition module 105, a model training completion module 106 and a relationship category recognition module 107. The module of the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.
[0129] In this embodiment, the functions of each module / unit are as follows:
[0130] A word vector acquisition module 101 is used to pre-process the training sample using a pre-built entity relationship recognition model to obtain a word vector for each word in the training sample;
[0131] An entity vector acquisition module 102, configured to process the entity pairs of the training sample according to the acquired word vectors, and acquire the entity vector of each entity in all entity pairs of the training sample, wherein the entity pair is composed of any two entities of the training sample;
[0132] An additional feature vector acquisition module 103 is used to perform weighted summation processing on the word vectors of each entity of the entity pair through a preset weight matrix to obtain an additional feature vector of each entity of the entity pair;
[0133] A prediction vector acquisition module 104 is used to process each entity vector of the entity pair and the additional feature vector through an activation function to obtain a prediction probability of a relationship category of the entity pair;
[0134] A relationship category loss function value acquisition module 105 is used to process the predicted probability of the entity-to-relation category through a pre-built loss function acquisition to acquire a relationship category loss function value;
[0135] A model training completion module 106 is used to cyclically obtain the relationship category loss function value until the relationship category loss function value converges to a preset range to complete the iterative training of the entity relationship recognition model;
[0136] The relationship category recognition module 107 is used to perform entity relationship recognition on the text to be recognized by using the trained entity relationship recognition model.
[0137] The embodiment of the present invention uses the constructed entity relationship recognition model to preprocess the training sample, and obtain the word vector of each word of the training sample; according to the obtained word vector, the entity pair of the training sample is processed to obtain the entity vector of each entity in all entity pairs of the training sample; the word vector of each entity of the entity pair is weighted and summed by the weight matrix to obtain the additional feature vector of each entity of the entity pair; each entity vector of the entity pair and the additional feature vector are processed by the activation function to obtain the predicted probability of the relationship category of the entity pair; the predicted probability of the relationship category of the entity pair is processed by the constructed loss function to obtain the relationship category loss function value; the relationship category loss function value is obtained cyclically, and when the relationship category loss function value converges to a preset range, the relationship category loss function value is stopped to complete the iterative training of the constructed entity relationship recognition model; the entity relationship recognition of the to-be-recognized text is performed by the trained entity relationship recognition model to obtain the relationship category of the entity pair. The main purpose of the present invention is to cyclically train the entity relationship recognition model by the constructed loss function, so as to solve the problem of poor recognition effect in the existing entity recognition process.
[0138] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing the entity relationship recognition method of the present invention.
[0139] The electronic device 1 may include a processor 10 , a memory 11 , and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10 , such as an entity relationship recognition program 12 .
[0140] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the data audit program, etc., but also be used to temporarily store data that has been output or is to be output. The memory may store content, which may be displayed by the electronic device or sent to other devices (e.g., headphones) to be displayed or played by other devices. The memory may also store content received from other devices. The content from the other device may be displayed, played, or used by the electronic device to perform any necessary tasks or operations that may be performed by a computer processor or other components in the electronic device and / or wireless access point.
[0141] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and a combination of various control chips. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect the various components of the entire electronic device. It executes various functions of the electronic device 1 and processes data by running or executing programs or modules (such as data auditing programs, etc.) stored in the memory 11, and calling data stored in the memory 11. The electronic device may also include a chipset (not shown) for controlling communication between one or more processors and one or more of the other components of the user device. In a specific embodiment, the electronic device may be based on Architecture or architecture, and processors and chipsets can come from Processor and Chipset Families. The one or more processors 104 may also include one or more application specific integrated circuits (ASICs) or application specific standard products (ASSPs) that are configured to handle specific data processing functions or tasks.
[0142] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.
[0143] In addition, the network and I / O interface may include one or more communication interfaces or network interface devices to provide data transmission between the electronic device and other devices (e.g., a network server) via a network (not shown). The communication interface may include, but is not limited to: a human area network (BAN), a personal area network (PAN), a wired local area network (LAN), a wireless local area network (WLAN), a wireless wide area network (WWAN), and the like. The user device can be coupled to the network via a wired connection. However, the wireless system interface may include hardware or software to broadcast and receive messages using the Wi-Fi Direct standard and / or the IEEE 802.11 wireless standard, the Bluetooth standard, the Bluetooth low energy standard, the Wi-Gig standard, and / or any other wireless standard and / or a combination thereof.
[0144] Wireless systems may include transmitters and receivers or transceivers capable of operating over a wide range of operating frequencies governed by the IEEE 802.11 wireless standard. Communication interfaces may utilize acoustic, radio frequency, optical, or other signals to exchange data between electronic devices and other devices such as access points, hosts, servers, routers, reading devices, and the like. Networks may include, but are not limited to, the Internet, private networks, virtual private networks, wireless wide area networks, local area networks, metropolitan area networks, telephone networks, and the like.
[0145] The display may include, but is not limited to, a liquid crystal display, a light emitting diode display, or an E-Ink™ display manufactured by E Ink Corp. of Cambridge, Massachusetts. The display may be used to display content to a user in the form of text, images, or video. In certain instances, the display may also operate as a touch screen display, which may enable a user to initiate commands or operations by touching the screen using certain fingers or gestures.
[0146] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 3 The structure shown does not constitute a limitation on the electronic device 3, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0147] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
[0148] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.
[0149] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.
[0150] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.
[0151] The entity relationship recognition program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:
[0152] Preprocessing the training sample using a pre-built entity relationship recognition model to obtain a word vector for each word in the training sample;
[0153] According to the obtained word vectors, the entity pairs of the training sample are processed to obtain the entity vectors of each entity in all entity pairs of the training sample, wherein the entity pairs are composed of any two entities of the training sample;
[0154] Performing weighted summation processing on the word vectors of each entity of the entity pair using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair;
[0155] Processing each entity vector of the entity pair and the additional feature vector through an activation function to obtain a predicted probability of a relationship category of the entity pair;
[0156] The predicted probability of the entity-to-relationship category is processed by using a pre-constructed loss function to obtain a relationship category loss function value;
[0157] Circularly acquiring the relationship category loss function value until the relationship category loss function value converges to a preset range, so as to complete the iterative training of the entity relationship recognition model;
[0158] Entity relationship recognition is performed on the text to be recognized through the trained entity relationship recognition model.
[0159] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiment is not repeated here. It should be emphasized that in order to further ensure the privacy and security of the similarity semantic processing results of each group of semantic units to be processed, the similarity semantic processing of each group of semantic units to be processed can also be stored in a node of a blockchain.
[0160] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).
[0161] In an embodiment of the present invention, a computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the steps of the above-mentioned entity relationship identification method. The specific method is as follows:
[0162] Preprocessing the training sample using a pre-built entity relationship recognition model to obtain a word vector for each word in the training sample;
[0163] According to the obtained word vectors, the entity pairs of the training sample are processed to obtain the entity vectors of each entity in all entity pairs of the training sample, wherein the entity pairs are composed of any two entities of the training sample;
[0164] Performing weighted summation processing on the word vectors of each entity of the entity pair using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair;
[0165] Processing each entity vector of the entity pair and the additional feature vector through an activation function to obtain a predicted probability of a relationship category of the entity pair;
[0166] The predicted probability of the entity-to-relationship category is processed by using a pre-constructed loss function to obtain a relationship category loss function value;
[0167] Circularly acquiring the relationship category loss function value until the relationship category loss function value converges to a preset range, so as to complete the iterative training of the entity relationship recognition model;
[0168] Entity relationship recognition is performed on the text to be recognized through the trained entity relationship recognition model.
[0169] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0170] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0171] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0172] The above describes some embodiments of the present invention with reference to the block diagram and flow chart of the system and method and / or computer program product according to exemplary embodiments of the present invention. It should be understood that one or more square blocks in the block diagram and flow chart, and the combination of the square blocks in the block diagram and flow chart, can be implemented by computer executable program instructions respectively. Similarly, according to some embodiments of the present invention, some square blocks in the block diagram and flow chart may not need to be executed in the order presented, or may not even need to be executed at all.
[0173] These computer executable program instructions can be loaded onto a general-purpose computer, a special-purpose computer, a processor, or other programmable data processing device to produce a specific machine, so that the instructions executed on the computer, processor, or other programmable data processing device create a component for realizing one or more functions specified in the flowchart box or multiple boxes. These computer program products can also be stored in a computer-readable memory, which can instruct the computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product, which includes an instruction component that realizes one or more functions specified in the flowchart box or multiple boxes. For example, an embodiment of the present invention can provide a computer program product, which includes a computer-usable medium containing a computer-readable program code or program instructions, and the computer-readable program code is suitable for being executed to realize one or more functions specified in the flowchart box or multiple boxes. The computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating elements or steps are executed on a computer or other programmable device to easily produce a computer-implemented program, so that the instructions executed on a computer or other programmable device provide elements or steps for realizing the functions specified in the flowchart box or multiple boxes.
[0174] Accordingly, the blocks in the block diagram or flow chart support a combination of components for performing a specified function, a combination of elements or steps for performing a specified function and a program instruction component for performing a specified function. It should also be understood that each block in the block diagram and flow chart and the combination of blocks in the block diagram and flow chart can be implemented by a dedicated hardware-based computer system that performs the specified function, element or step, or by a combination of dedicated hardware or computer instructions.
[0175] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0176] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0177] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.
[0178] Although certain embodiments of the present invention have been described in conjunction with what are currently considered to be the most practical and various embodiments, it should be understood that the present invention is not limited to the disclosed embodiments, but is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims. Although specific terms are employed herein, they are used only in a general and descriptive sense and not for the purpose of limitation.
[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A method for identifying entity relationships, applied to electronic devices, characterized in that: The method comprises: Preprocessing the training sample using a pre-built entity relationship recognition model to obtain a word vector for each word in the training sample; According to the obtained word vectors, the entity pairs of the training sample are processed to obtain the entity vectors of each entity in all entity pairs of the training sample, wherein the entity pairs are composed of any two entities of the training sample; Performing weighted summation processing on the word vectors of each entity of the entity pair using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair; Processing each entity vector of the entity pair and the additional feature vector through an activation function to obtain a predicted probability of a relationship category of the entity pair; The predicted probability of the entity-to-relationship category is processed by using a pre-constructed loss function to obtain a relationship category loss function value; Circularly acquiring the relationship category loss function value until the relationship category loss function value converges to a preset range, so as to complete the iterative training of the entity relationship recognition model; Perform entity relationship recognition on the text to be recognized through the trained entity relationship recognition model; The step of performing weighted summation processing on the word vectors of each entity of the entity pair by using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair includes: Processing the entity vector of the head entity in the entity pair by using a preset weight matrix to obtain the weight of the head entity in the entity pair; Processing the entity vector of the tail entity in the entity pair by using a preset weight matrix to obtain the weight of the tail entity in the entity pair; According to the weight of the head entity in the entity pair and the weight of the tail entity in the entity pair, an additional feature vector of each entity in the entity pair is obtained.
2. The entity relationship identification method according to claim 1, characterized in that: The preprocessing of the training sample by using the pre-built entity relationship recognition model to obtain the word vector of each word in the training sample includes: Obtaining a target word in the training sample and determining an initial word vector corresponding to the target word; Determine the image feature vector, the root feature vector, and the pinyin feature vector corresponding to the target character according to the initial character vector; A word vector corresponding to the target word is generated according to the initial word vector, the image feature vector, the root feature vector, the pinyin feature vector and a preset weight matrix.
3. The entity relationship identification method according to claim 1, characterized in that: The step of processing the entity pairs of the training sample according to the obtained word vectors to obtain the entity vector of each entity in all entity pairs of the training sample includes: According to the obtained word vector, obtain the word vector of each word in the training sample; According to the obtained word vector, the entity vector in the training sample is obtained.
4. The entity relationship identification method according to claim 1, characterized in that: The step of processing each entity vector of the entity pair and the additional feature vector by using an activation function to obtain a predicted probability of a relationship category of the entity pair includes: Processing each entity vector and the additional feature vector of the entity pair respectively through a tanh activation function to obtain a tanh result value of each entity respectively; The obtained tanh result value of each entity is multiplied and then input into the sigmoid function to obtain the predicted probability of the entity to the relationship category.
5. The entity relationship identification method according to claim 1, characterized in that: The loop obtains the relationship category loss function value, and when the relationship category loss function value converges to a preset range, stops the loop to obtain the relationship category loss function value to complete the iterative training of the constructed entity relationship recognition model, including: When the relationship category loss function value is lower than or equal to the preset category value, continue to loop to obtain the relationship category loss function value; When the relationship category loss function value is higher than a preset category value, the acquisition loop to acquire the relationship category loss function value is stopped to complete the training of the entity relationship recognition model.
6. An entity relationship recognition device, characterized in that: The device comprises: A word vector acquisition module, used to pre-process the training sample using a pre-built entity relationship recognition model to obtain a word vector for each word in the training sample; An entity vector acquisition module, used to process the entity pairs of the training sample according to the acquired word vectors, and acquire the entity vector of each entity in all entity pairs of the training sample, wherein the entity pair is composed of any two entities of the training sample; An additional feature vector acquisition module, used to perform weighted summation processing on the word vectors of each entity of the entity pair through a preset weight matrix to obtain an additional feature vector of each entity of the entity pair; A prediction vector acquisition module, used to process each entity vector of the entity pair and the additional feature vector through an activation function to obtain a prediction probability of the relationship category of the entity pair; A relationship category loss function value acquisition module, used to process the predicted probability of the entity to the relationship category through a pre-built loss function acquisition to obtain a relationship category loss function value; A model training completion module, used for cyclically obtaining the relationship category loss function value until the relationship category loss function value converges to a preset range, so as to complete the iterative training of the entity relationship recognition model; The relationship category recognition module is used to perform entity relationship recognition on the text to be recognized through the trained entity relationship recognition model; The step of performing weighted summation processing on the word vectors of each entity of the entity pair by using a preset weight matrix to obtain an additional feature vector of each entity of the entity pair includes: Processing the entity vector of the head entity in the entity pair by using a preset weight matrix to obtain the weight of the head entity in the entity pair; Processing the entity vector of the tail entity in the entity pair by using a preset weight matrix to obtain the weight of the tail entity in the entity pair; According to the weight of the head entity in the entity pair and the weight of the tail entity in the entity pair, an additional feature vector of each entity in the entity pair is obtained.
7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the entity relationship identification method as described in any one of claims 1 to 5.
8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for identifying entity relationships as described in any one of claims 1 to 5 is implemented.
