Small sample relationship extraction method and device for content big data

Abstract

The present invention discloses a method and device for extracting small sample relations for content big data. The present invention firstly uses a pre-trained language model and a character-level LSTM to vectorize sentences; secondly, extracts text structure features through a dual affine mechanism and a graph neural network; simultaneously, uses a word-level LSTM and the [CLS] word vector representing semantic information in BERT to integrate context information into the word vector of an entity pair, and constructs entity pair features containing context information; finally, extracts the similarity and difference of sentence features through a similarity-difference relationship network to perform small sample relation extraction. The present invention applies small sample learning in a content big data scenario, fully extracts text features, and characterizes the differences between entity relations of different categories, thereby improving the accuracy of relation extraction.

Description

Technical Field

[0001] The present invention relates to a method and device for extracting small sample relations for content big data, belonging to the technical field of Internet and big data. Background Art

[0002] With the continuous development of the Internet industry, the text data on the Internet is growing at an exponential rate. The text data on the Internet is heterogeneous, fragmented, and multi-source, and contains rich knowledge and information. The entity relationship extraction method can extract structured information representation from unstructured text data. The structured information representation plays a fundamental role in the construction of applications such as knowledge graphs, search engines, and intelligent question-answering systems. In addition, the extracted structured information must be organized and managed reasonably and efficiently to more fully explore and utilize the interrelated information in the text, thereby realizing content governance under Internet big data. The entity relationship extraction method usually needs to be based on entity annotation, but entity annotation is often time-consuming, labor-intensive, and costly. Therefore, it is of great practical significance to study the entity relationship extraction technology for small sample scenarios.

[0003] In recent years, small sample relation extraction based on neural network models has become a mainstream method. However, existing small sample relation extraction methods consider fewer dimensions when extracting sentence features, and it is difficult to efficiently extract multi-dimensional features of text. At the same time, when using sentence features for relation extraction, existing small sample relation extraction methods often only focus on the similarities between sentence features, while ignoring the differences between different types of entity relations. Summary of the invention

[0004] In view of the problems and shortcomings of existing small sample relationship extraction methods, the present invention proposes a small sample relationship extraction method and device for content big data. The present invention uses a dual affine mechanism and a graph convolutional neural network to extract text structure information and entity pair information from sentences containing entity pairs as sentence features, thereby improving the adequacy of feature extraction. At the same time, the present invention uses a similarity-difference relationship network to extract the similarity and difference of relationship categories, and then implements small sample relationship extraction based on the similarity and difference, thereby increasing attention to features that can distinguish relationships of different categories.

[0005] In order to achieve the above-mentioned purpose, the present invention designs a method and device for extracting small sample relations for content big data. First, a pre-trained language model BERT or a word-level bidirectional LSTM is used to vectorize the sentence, and the character-level semantic information of the word is extracted using the bidirectional LSTM, and then spliced ​​to the word vector pre-trained by BERT. Secondly, in view of the important role of the structural characteristics of sentence dependencies in the extraction of inter-entity relations, the present invention uses a dual affine mechanism to construct a dependency graph, and then uses a graph convolutional neural network to extract text structure features. At the same time, as a bridge connecting the subject and object entities, the semantic information of the entity pair plays a decisive role in relation extraction. The semantic information of the entity pair is easily affected by the context. In order to reflect this influence on the word vector of the entity pair, the present invention uses the long-term memory unit of the bidirectional LSTM network and the [CLS] word vector pre-trained by BERT to attach it to the word vector in the entity pair through projection. Then, the present invention combines the text structure features with the entity pair features as the feature vector of the sentence. Finally, in order to avoid ignoring the differences between sentence features of different entity relationships when using sentence feature vectors for relationship extraction, which leads to a single method of measuring text features, the present invention proposes to use convolutional neural networks to measure similarities and differences respectively, and then predict entity relationship categories based on similarities and differences.

[0006] A small sample relationship extraction method for content big data, which mainly includes four steps, as follows:

[0007] Step 1, vectorize the sentence by integrating character information. Before extracting the relational dependency structure features of the sentence and the entity pair features of the integrated context, the words in the text need to be vectorized. The present invention uses the pre-trained language model BERT to vectorize the text. Considering that under the Episode training strategy for small sample learning, the text input of the support set Support and the query set Query needs to be vectorized at the same time, the requirements for memory and graphics card resources are high, so the present invention adds an affine layer after the pre-trained word vector of the BERT basic version to reduce the dimension of the word vector, reduce the number of parameters of the model, reduce the occupation of resources and speed up the convergence of the model. The BERT pre-trained language model randomly covers the words, and then predicts the covered words through the context, and the lexical information of the words cannot be fully obtained. According to the study of grammatical structure characteristics in language morphology, the word form change of English words is the main means of expressing grammatical relations, and the word form change of English words is mainly realized by the transformation of English characters, so the characters that make up each word also play an important role in understanding the semantics of words. The present invention uses a character-level bidirectional LSTM to obtain word vectors that have been embedded with character information.

[0008] Step 2, extracting the relational dependency structure features of the sentence. The effect of the relational extraction neural network model can be further improved by means of dependency analysis of the sentence. The present invention uses a dual affine mechanism to realize the construction of an undirected dependency graph on the text sequence word vector input. Before performing the dual affine transformation, two multi-layer perceptrons are used to reduce the dimension of the word vector, thereby avoiding overfitting of the model. At the same time, the dimensionality reduction operation of the word vector also helps to remove redundant semantic information, because the original word vector may already contain information about the predicted dependency arc. The adjacency matrix of an undirected graph is a real symmetric matrix, which represents the node adjacency relationship of a graph data structure. The use of a multi-layer graph convolutional network can aggregate other node information in the neighborhood of the central node in the graph data structure, and each convolutional layer only processes the first-order neighborhood information. By superimposing several convolutional layers, information transmission of multi-order neighborhoods can be realized. The present invention constructs a dependency graph of the input text sentence based on the dependency matrix constructed by the dual affine mechanism, and then uses a graph convolutional neural network to extract text structure features from the dependency graph.

[0009] Step 3, extracting entity pair features that integrate context. The features of entity pairs in text sentences are mainly affected by the semantic features of the subject and object entities and the context information of the entity pairs. The context information can also help the model correctly understand the relationship between entities when polysemous entities appear. In order to maintain the long-distance dependency of the text, the present invention uses a bidirectional LSTM network structure to encode the word-level context features of entity pairs in text sentences and the classification word vector used in BERT to represent the context of the sentence, and then integrates the context features into the word vector by projection, and then uses a segmented pooling neural network to fuse the word vectors in the subject and object entities to obtain the entity pair features of the sentence.

[0010] Step 4, based on the similarity-difference relationship network, small sample relationship extraction is realized. The main reason why the neural network model is difficult to show the effect in the small sample learning scenario is that a small amount of sample data cannot reflect the real data distribution, which leads to the fact that the empirical risk cannot simulate the expected risk well under the learning strategy guided by the empirical risk minimization principle. The similarity-difference relationship network in the present invention includes two parts: a similarity relationship extraction module and a difference relationship extraction module. The similarity relationship extraction module can measure the similarity between entity relationship text instances in multiple dimensions, and the difference relationship extraction module measures the difference degree of semantic features in different dimensions. The network structure of the similarity measurement module and the difference measurement module is the same, but the parameters are independent of each other. The present invention trains the model based on the meta-task training strategy of Episode. In each round of Episode training, the features of the sentences are first "broadcasted", and then the spliced ​​feature vectors are input into the similarity-difference relationship network to extract similarity and difference, and finally the small sample relationship extraction is realized based on similarity and difference. At the same time, the KL divergence of the similarity distribution of the same type of relationship and the cross entropy loss of the prediction result are used as the model loss function to train the model parameters.

[0011] The present invention also provides a small sample relationship extraction device for content big data, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the small sample relationship extraction method for content big data is implemented.

[0012] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0013] 1. The present invention can extract features that have an improvement effect on relationship extraction from multiple dimensions such as word form information, text structure, entity pairs, etc., and fully characterize the differences between different types of entity relationships, thereby improving the accuracy of small sample relationship extraction.

[0014] 2. The present invention effectively incorporates word form information into word vectors by integrating sentence vectorization with character-level information, which helps to improve the representation of word morphological features by word vectors and improves the richness of semantic information represented by word vectors.

[0015] 3. The present invention extracts the dependency information of sentences through a dual affine mechanism and a graph convolutional neural network, avoiding the problems of external errors and difficulty in adapting to new entities and relationships caused by using external natural language tools to extract sentence dependencies.

[0016] 4. The present invention incorporates the contextual information of the sentence into the word vector within the entity pair, thereby enhancing the role of the sentence context containing entity relations in promoting the extraction of relations between entities.

[0017] 5. The present invention uses a similarity-difference relationship network to extract the similarities and differences of sentence features, focusing not only on the similarities of the same type of relationships, but also on the differences of different types of relationships, making the use of the significant features of the sentences more comprehensive and full. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a model diagram of a small sample relationship extraction method for content big data according to an embodiment of the present invention.

[0019] Figure 2 This is a character-level bidirectional LSTM neural network diagram of an embodiment of the present invention.

[0020] Figure 3 Schematic diagram of a dual affine mechanism according to an embodiment of the present invention.

[0021] Figure 4 This is a convolutional neural network diagram of an embodiment of the present invention.

[0022] Figure 5 This is a similarity-difference relationship network diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The technical solution provided by the present invention will be described in detail below in conjunction with specific embodiments. It should be understood that the following specific implementation methods are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0024] The embodiment of the present invention discloses a method for extracting small sample relations from content big data, and its implementation model is as follows: Figure 1 As shown, the specific implementation steps are as follows:

[0025] Step 1: vectorize the sentence by integrating character information. This embodiment uses the basic version of the pre-trained language model BERT to vectorize the sentence. The calculation is shown in formula (1), where [CLS] represents the classification feature vector that can represent the context of the sentence, [SEP] represents the sentence word vector, sentenct = [w1, w2, ..., w L ] represents the word list of the sentence, and L represents the length of the text sentence. Then, the affine layer is used to reduce the dimension of the pre-trained word vector, and the calculation is shown in formula (2), where represents the splicing operation, U (bert) Represents a matrix that concatenates the parameter matrix and the bias vector, making it easier to pass the bias through the parameter matrix U (bert) Conduct training.

[0026] H bert =BERT base ([CLS]; sentenct; [SEP]) (1)

[0027]

[0028] This example uses a bidirectional LSTM to obtain the word vector h that embeds character information. chars ,like Figure 2 As shown, the final calculated word vector input h word is the pre-trained word vector h after dimensionality reduction pre-train And the word vector h embedded with character information chars Splicing to get h chars and h word The calculations are shown in equations (3) and (4) respectively.

[0029]

[0030] h word =[h pre-train ;h chars ] (4)

[0031] c w For each character in a word.

[0032] Step 2: Extract the relational dependency structure features of the sentence. This embodiment uses a bi-affine mechanism to implement undirected dependency graph construction on the text sequence word vector input, such as Figure 3 As shown. In this embodiment, two multi-layer perceptrons MLP are used before performing the double affine transformation. (arc-head) and MLP (arc-dep) Change the dimension of the word vector from D emb Reduce to k, reduce the dimension of the neural network, and thus avoid overfitting of the model. emb is the original word vector dimension. In addition, the dimensionality reduction operation on the word vector dimension also helps to remove redundant semantic information, because the original word vector h i may already contain information about predicted dependency arcs.

[0033] The calculation of multi-layer perceptron dimensionality reduction is shown in equations (5) and (6).

[0034]

[0035]

[0036] The input sentence contains L words, including the root node ROOT added in this embodiment, there are a total of d = L + 1 tokens, each of which requires a score vector To describe whether the dependency relationship between the token and other tokens exists, all the score vectors in the sentence form a score matrix The probability calculation of whether the dependency arc of each Token exists is shown in formula (7), where It is the feature vector after the features of d tokens are reduced in dimension by a multi-layer perceptron. composition, and is the weight parameter that needs to be learned through training. After combining the two weight parameters in formula (7), the score matrix S of the dependency arc is established (arc-score) The calculation of is as shown in formula (8), where byU (1) and u (2) structure,

[0037]

[0038]

[0039] Score matrix S (arc-score) It represents the probability that there is a dependency relationship between two words. When the probability of the dependency relationship exceeds the threshold α, it is considered that the dependency relationship does exist and promotes the extraction of entity relationships. In this way, the dependency matrix S of the text sentence is constructed. (arc) , the calculation is shown in formula (9).

[0040] S (arc) =ReLU(S (arc-score) -α) (9)

[0041] In this embodiment, the dependency matrix S (arc) Based on constructing the dependency graph of the input text sentence, a graph convolutional neural network is used to extract text structure features from the dependency graph, such as Figure 4 As shown, through S (arc) The adjacency matrix is ​​constructed as shown in formula (10). According to the relevant knowledge of graph convolutional neural network, the graph convolution propagation formula used in this embodiment can be derived as shown in formula (11).

[0042]

[0043]

[0044] in, is the feature vector of each node after l layers of graph convolution, σ is the nonlinear activation function, and d WE is the word vector dimension, is the degree matrix of the dependency graph, It is a simplified l-layer parameter matrix using the recursive property of Chebyshev polynomials. Using a graph convolutional neural network to perform convolution operations on the dependency graph can extract enough semantic information about the dependency relationship between core words and dependent words, realize the extraction of structural features of the input text sentence, and thus help improve the entity relationship extraction effect. Since the length of the text sentence is not fixed, this embodiment performs segmented maximum pooling fusion on the output vectors of the main entity, the object entity, and the text sentence, and then splices them to obtain the text structure feature vector h of the input text sentence. structure .

[0045] Step 3: Extract entity pair features that integrate the context. In order to maintain the long-distance dependencies of the text, this embodiment uses a bidirectional LSTM network structure to encode the context features of entity pairs in the text sentence. And the classification word vector h used in BERT after text encoding CLS Both capture the contextual information of the entity pair and can improve the effect of entity pair features in relation extraction. LSTM and h CLS The projection on the entity pair's word vector is integrated into the entity pair's word vector as context information. The fused subject and object entity word vectors and The calculations are shown in equations (12) and (13).

[0046]

[0047]

[0048] In this embodiment, the subject and object entity word vector features that incorporate context information are also fused using segmented maximum pooling and then concatenated to obtain the entity pair feature vector h of the input text sentence. entitypair .

[0049] Step 4: Extract small sample relationships based on the similarity-difference relationship network. This embodiment uses an Episode-based meta-task training strategy to train the model. In each Episode training round, K instances of N categories are extracted from the training set for training. In this embodiment, the similarity measurement module and the difference measurement module have the same network structure, but the parameters are independent of each other. Figure 5 As shown. The inputs of the two modules mentioned above are the same, input T fea The construction method is shown in formula (14), where: Represents the text structure feature vector h of the i-th sample structure , Represents the entity pair feature vector h of the i-th sample entitypair, use the concatenation vector of the text structure features and entity pair features of each text in the support set and query set to perform a "broadcast" connection with the features of K instance text sentences of N types of relations in the support set.

[0050]

[0051] The two metric modules use four neural network blocks to extract multidimensional similarity and multidimensional difference. Each neural network block consists of a 1×3 convolution layer, a batch normalization layer, a ReLU nonlinear activation layer, and a 1×1 convolution layer. Finally, the similarity T is output through the SoftMax activation layer. sim and difference T dif , as shown in equations (15) and (16).

[0052]

[0053]

[0054] and are the network parameters of the neural networks of the similarity measurement module and the difference measurement module respectively.

[0055] In this embodiment, the similarity tensor and difference tensor of the text sentence in the support set and query set and the N*K instances in the support set are used as the similarity and difference between the text sentence in the query set and the instance relationship category prototype in the support set, and the calculation is shown in formula (17) and formula (18). Finally, the similarity between the text and the entity relationship category is calculated as T sim and difference T dif Based on , the predicted value of the entity relationship category of the text sentence in the query set is calculated as shown in formula (19).

[0056]

[0057]

[0058]

[0059] The model loss function of the method in this embodiment is composed of KL loss Loss KL and cross entropy loss Loss CE The entity relationship instances in the support set and query set should be similar in distribution to the prototypes of the class in the support set, so the KL divergence (Kullback-Leibler divergence) is used to measure the consistency of this multi-dimensional similarity distribution and as part of the loss function. KL The calculation of is shown in formula (20), where x∈[1,2,…,D sim ], Dkl is the KL divergence calculation symbol, The prototype represents the similarity of the n-th entity relationship instance. The cross entropy loss between the model prediction label and the true label Loss CE The calculation is shown in formula (21), where Q represents the number of instances in the query set, and y real (i) represents the true label of the i-th instance.

[0060]

[0061]

[0062] In summary, the present invention firstly uses a pre-trained language model and character-level LSTM to vectorize sentences; secondly, text structure features are extracted through a dual affine mechanism and a graph neural network; at the same time, word-level LSTM and the [CLS] word vector representing semantic information in BERT are used to integrate contextual information into the word vector of entity pairs, and entity pair features containing contextual information are constructed; finally, the similarity and difference of sentence features are extracted through a similarity-difference relationship network to perform small sample relationship extraction. While fully extracting the multi-dimensional features of sentences, the present invention also focuses on the significant features that can distinguish different types of relationships, improves the accuracy of small sample relationship extraction, and provides a novel and effective solution for the relationship extraction technology of content big data in a small number of sample annotation scenarios.

[0063] Based on the same inventive concept, the present invention also provides a small sample relationship extraction device for content big data, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the small sample relationship extraction method for content big data is implemented. Specifically, the small sample relationship extraction device for content big data includes a character information fusion vectorization module, a relationship dependency structure extraction module, an entity pair feature extraction module, and a small sample relationship extraction module, wherein the character information fusion vectorization module is used to use a pre-trained language model and a character-level LSTM to vectorize sentences, specifically implementing the content of step 1 in the small sample relationship extraction method for content big data; the relationship dependency structure extraction module is used to extract text structure features through a dual affine mechanism and a graph neural network, specifically implementing the content of step 2 in the small sample relationship extraction method for content big data; the entity pair feature extraction module is used to use word-level LSTM and the [CLS] word vector representing semantic information in BERT to integrate contextual information into the word vector of the entity pair, construct entity pair features containing contextual information, and specifically implement the content of step 3 in the small sample relationship extraction method for content big data; the small sample relationship extraction module is used to extract the similarity and difference of sentence features through a similarity-difference relationship network to perform small sample relationship extraction, specifically implementing the content of step 4 in the small sample relationship extraction method for content big data.

[0064] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A small sample relationship extraction method for content big data, characterized by: The steps include: Step 1: Vectorize the sentence by integrating character information Use the pre-trained language model BERT to vectorize the text. Add an affine layer to the pre-trained word vector of the basic version of BERT to reduce the dimension of the word vector and reduce the number of model parameters. Use character-level bidirectional LSTM to obtain word vectors that have embedded character information. The specific process includes the following: The sentence is vectorized using the basic version of the pre-trained language model BERT. The calculation is shown in formula (1), where [CLS] represents the classification feature vector that can represent the sentence context, [SEP] represents the sentence word vector, sentence = [w1, w2, ..., w L ] represents the word list of the sentence, and L represents the length of the text sentence: H bert =BERT base ([CLS];sentence;[SEP])=[h CLS ,h1,h2,…,h L ,h SEP ] (1) Then use the affine layer to reduce the dimension of the pre-trained word vector, and the calculation is shown in formula (2), where Represents the splicing operation, which facilitates the bias to be passed through the parameter matrix U (bert) To train: Use bidirectional LSTM to obtain word vector h with embedded character information chars , and finally the calculated word vector is input into h word is the pre-trained word vector h after dimensionality reduction pre-train And the word vector h embedded with character information chars Splicing to get h chars and h word The calculations are shown in formula (3) and formula (4) respectively: h word =[h pre-train ;h chars ] (4) Step 2: Extract the sentence’s relational dependency structure features The bi-affine mechanism is used to implement undirected dependency graph construction on the text sequence word vector input. Two multi-layer perceptrons are used to reduce the dimension of the word vector before the bi-affine transformation. The dependency graph of the input text sentence is constructed based on the dependency matrix constructed by the bi-affine mechanism, and then the graph convolutional neural network is used to extract text structure features from the dependency graph. The specific process includes the following: Use two multi-layer perceptrons (MLPs) before performing the double affine transformation. (arc-head) and MLP (arc-dep) Change the dimension of the word vector from D emb Reduced to k, the dimension of the neural network is reduced. The calculation of dimensionality reduction of the multilayer perceptron is shown in equations (5) and (6): The input sentence contains L words, including the added root node ROOT, a total of d = L + 1 tokens, each token needs a score vector To describe whether the dependency relationship between the token and other tokens exists, all the score vectors in the sentence form a score matrix The probability calculation of whether the dependency arc of each Token exists is shown in formula (7), where It is the feature vector after the features of d tokens are reduced in dimension by a multi-layer perceptron. composition, and is the weight parameter that needs to be learned through training; after combining the two weight parameters in formula (7), the score matrix S of the dependency arc is established (arc-score) The calculation of is as shown in formula (8), where byU (1) and u (2) structure: Score matrix S (arc-score) It represents the probability that there is a dependency relationship between two words. When the probability of the dependency relationship exceeds the threshold α, it is considered that the dependency relationship does exist and promotes the extraction of entity relationships. In this way, the dependency matrix S of the text sentence is constructed. (arc) , calculated as shown in formula (9): S (arc) =ReLU(S (arc-score) -α) (9) The dependency matrix S (arc) Based on constructing the dependency graph of the input text sentence, a graph convolutional neural network is used to extract text structure features from the dependency graph. (arc) The adjacency matrix is ​​constructed as shown in formula (10). According to the relevant knowledge of graph convolutional neural network, the graph convolution propagation formula is derived as shown in formula (11): in, is the feature vector of each node after l layers of graph convolution, σ is the nonlinear activation function, is the degree matrix of the dependency graph, It is a simplified l-layer parameter matrix using the recursive property of Chebyshev polynomials; the output vectors of the main entity, the object entity, and the text sentence are fused by segmented maximum pooling and then spliced ​​to obtain the text structure feature vector h of the input text sentence structure ; Step 3: Extract entity pair features that integrate context The bidirectional LSTM network structure is used to encode the word-level context features of the entity pairs in the text sentence and the classification word vector used in BERT to represent the context of the sentence. The context features are then integrated into the word vector by projection. The segmented pooling neural network is then used to fuse the word vectors in the subject and object entities to obtain the entity pair features of the sentence. The specific process includes the following: Use a bidirectional LSTM network structure to encode the contextual features of entity pairs in text sentences, and bidirectional long-term memory of bidirectional LSTM output And the classification word vector h used in BERT after text encoding CLS Both capture the contextual information of the entity pair, with the vector h containing the contextual information LSTM and h CLS The projection on the entity pair's word vector is integrated into the entity pair's word vector as context information. The fused subject and object entity word vectors and The calculation is shown in equations (12) and (13): The subject and object entity word vector features that incorporate context information are also fused using segmented maximum pooling and then concatenated to obtain the entity pair feature vector h of the input text sentence. entitypair ; Step 4: Extract small sample relationships based on similarity-difference relationship network The similarity-difference relationship network includes two parts: a similarity relationship extraction module and a difference relationship extraction module; the similarity relationship extraction module can measure the similarity between text instances containing entity relationships in multiple dimensions, and the difference relationship extraction module is used to measure the difference in semantic features in different dimensions; the similarity measurement module and the difference measurement module have the same network structure, but the parameters are independent of each other; the model is trained based on the meta-task training strategy of Episode, and in each round of Episode training, the features of the sentences are first "broadcasted" connected, and then the connected feature vectors are input into the similarity-difference relationship network to extract similarities and differences, and finally small sample relationship extraction is realized based on similarities and differences; at the same time, the KL divergence of the similarity distribution of the same type of relationship and the cross entropy loss of the prediction result are used as the model loss function to train the model parameters; the specific process includes the following: The model is trained using the Episode-based meta-task training strategy. In each Episode training, K instances of N categories are extracted from the training set for training. The similarity measurement module and the difference measurement module have the same network structure, but the parameters are independent of each other. The inputs of the two modules are the same, and the input T fea The construction method is shown in formula (14), where: Represents the text structure feature vector h of the i-th sample structure , Represents the entity pair feature vector h of the i-th sample entitypair , use the text structure features and entity pair features of each text in the support set and query set to perform a "broadcast" connection with the features of K instance text sentences of N types of relations in the support set: The two metric modules use four neural network blocks to extract multidimensional similarity and multidimensional difference. Each neural network block consists of a 1×3 convolution layer, a batch normalization layer, a ReLU nonlinear activation layer, and a 1×1 convolution layer. Finally, the similarity T is output through the SoftMax activation layer. sim and difference T dif , as shown in formula (15) and formula (16): The similarity and difference tensors between the text sentences in the support set and the query set and the N*K instances in the support set are taken as the similarity and difference between the prototypes of the relationship categories between the text sentences in the query set and the instances in the support set, and are calculated as shown in formulas (17) and (18): Finally, the similarity T between the text and the entity relationship category is used sim and difference T dif Based on , the predicted value of entity relationship category of text sentences in the query set is calculated as shown in formula (19): The model loss function is KL loss Loss KL and cross entropy loss Loss CE Composition, Loss KL The calculation of is shown in formula (20), where x∈[1, 2, …, D sim ], D kl is the KL divergence calculation symbol, The prototype representing the similarity of the n-th entity relationship instance: Cross entropy loss between model prediction labels and true labels Loss CE The calculation is shown in formula (21): Where Q represents the number of instances in the query set, and y real (i) represents the true label of the i-th instance.

2. A small sample relationship extraction device for content big data, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, it implements the small sample relationship extraction method for content big data as described in claim 1.

Images