A deep learning-based method for biomedical entity recognition and relationship prediction

Through the joint prediction and pointer network technology of deep learning models, the interdependence and overlap of entities in biomedical entity recognition and relationship prediction are solved, efficient entity and relationship recognition is achieved, and prediction accuracy and completeness are improved.

CN114021569BActive Publication Date: 2024-12-03ANHUI UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111298745.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-04
Publication Date
2024-12-03
Estimated Expiration
2041-11-04

AI Technical Summary

Technical Problem

The prior art ignores the underlying interdependence and error propagation between entities in biomedical entity recognition and relationship prediction, and fails to effectively deal with the problem of overlapping relationship triplets, resulting in high demand for computing resources and low prediction accuracy.

Method used

Using an end-to-end model based on deep learning, the bioBERT encoding layer, batch normalization layer and entity boundary decoding layer are combined with a pointer network to realize joint prediction of entities and relationships. First identify the head entity and then predict the tail entity according to the relationship type. Solved entity overlap problem.

Benefits of technology

It effectively reduces the demand for computing resources, improves the prediction accuracy, avoids error propagation, and makes the prediction results more accurate and complete.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114021569B_ABST
    Figure CN114021569B_ABST
Patent Text Reader

Abstract

The present invention discloses a method for biomedical entity recognition and relationship prediction based on deep learning, and the steps include: 1. Obtaining neural network training data, i.e. biomedical literature, performing sentence segmentation and word segmentation operations on it, and processing unstructured text into structured text; 2. Annotating structured text, manually annotating entities in sentences and relationships between entities; 3. Building a deep neural network model; 4. Training the constructed model with annotated data to obtain a deep learning model with optimal experimental results; 5. Inputting unannotated data into the optimal model to predict entity relationships in the literature. The present invention can simultaneously identify entities in biomedical literature and the relationships between entities, thereby effectively improving the accuracy of recognition and reducing the demand for computing resources.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of biomedical text information mining, and specifically is a method for biomedical entity recognition and relationship prediction based on deep learning. Background Art

[0002] Named Entity Recognition (NER), also known as "proper name recognition", refers to the identification of entities with specific meanings in text, mainly including names of people, places, institutions, proper nouns, etc. Simply put, it is to identify the boundaries and categories of entity references in natural text. In the biomedical field, entities can be divided into cell entities, gene entities, protein entities, drug entities, etc.; entities are the main members of constructing knowledge graphs.

[0003] The task of relation prediction is to predict the relationship between entities in the text. Relationship refers to the connection between different entities. Entities are not independent of each other, and there is often a certain association between them. The ultimate goal is to find the triple of (entity 1, relationship, entity 2) from unstructured text. It is very important for the construction of knowledge base and question-answering tasks. It is the core issue of information extraction and has a driving role in the construction of biomedical knowledge graphs and databases.

[0004] Early entity recognition and relation prediction divided the task into pipeline subtasks: first detect the entity, then classify the type of relationship between the optional entity pairs. Such a framework makes the task easy to perform, but it ignores the underlying interdependence and error propagation between the two subtasks. Later, a joint prediction model was proposed, that is, entity recognition and relation extraction are performed simultaneously. Unlike the pipeline method, the joint learning framework uses a single model to extract entities and relations. It can effectively integrate the information of entities and relations, and it has been shown to achieve better results in this task.

[0005] In recent years, with the success of deep learning on many natural language processing tasks, it has also been applied to entity recognition and relation prediction. Although joint models have been well studied, most previous works ignore the problem of overlapping relation triplets, i.e., multiple triplets share one or two entities, thus losing a lot of key information. Summary of the invention

[0006] The present invention aims to address the deficiencies of the above-mentioned prior art and proposes a method for biomedical entity recognition and relationship prediction based on deep learning, in order to simultaneously identify entities in biomedical literature and the relationships between entities, thereby effectively improving the accuracy of prediction while reducing the demand for computing resources.

[0007] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:

[0008] The method of biomedical entity recognition and relationship prediction based on deep learning of the present invention is characterized by being performed in the following steps:

[0009] Step 1: Get the training data set:

[0010] Step 1.1: Obtain unstructured text of biomedical literature;

[0011] Step 1.2: The unstructured text is preprocessed by sentence segmentation and noise data removal to obtain a structured unlabeled text training data set, denoted as X = {x1, x2, ..., x i ,...,x M}; where M represents the number of sentences in a batch; x i represents the i-th sentence sequence, and has Where N represents the number of words in the sentence sequence; Represents the i-th sentence sequence x i The jth word in ;

[0012] Step 2: Manually label the structured unlabeled text training dataset X:

[0013] Step 2.1: Annotation of biomedical named entities:

[0014] The biomedical named entities in the sentence sequence are manually annotated, and the entity type is recorded as: L = {L1, L2, ..., L k ,...,L n}, where n represents the total number of named entity types; L k represents the kth entity type;

[0015] Step 2.2: Label entity relationships;

[0016] The relationships between entities in the sentence sequence are manually annotated, and the relationship type is recorded as R = {r1, r2, ..., r l ,...,r m}, where m represents the total number of relationship types between entities; r l Indicates the lth relationship type;

[0017] Get the i-th sentence sequence x i The triples of m entity relations in are recorded as in, Represents the i-th sentence sequence x i The lth triple in ; represents the lth head entity, represents the lth relationship type, represents the t-th tail entity, So as to obtain the data label set

[0018] Step 3: Build a deep neural network model consisting of the bioBERT encoding layer, batch normalization layer, and entity boundary decoding layer:

[0019] Step 3.1. Processing of bioBERT encoding layer:

[0020] The text training dataset X is input into the bioBERT encoding layer to obtain the feature information of each word in all sentence sequences, denoted as H = {H1, H2, ..., H i ,...,H M}, where H i Represents the i-th sentence sequence x i The eigenvector of , and: H i ={H i,1 ,H i,2 ,...,H i,j ,...,H i,N}, H i,j Represents the i-th sentence sequence x i The jth word in characteristic information;

[0021] Step 3.2, batch normalization layer processing:

[0022] The feature information H is input into the batch normalization layer for normalization, and then scaled and translated to obtain the feature distribution H′={H′1,H′2,...,H′ i ,...,H′ M}, where H′ i Represents the i-th sentence sequence x i The feature vector after the normalization layer has: H′ i ′={h i,1 ,h i,2 ,...,h i,j ,...,h i,N}, h i,j Represents the i-th sentence sequence x i The jth word in feature information, N represents the number of words in the sentence sequence;

[0023] Step 3.3: Build a header entity boundary decoding layer based on a pointer network, wherein the header entity boundary decoding layer consists of two fully connected layers;

[0024] Step 3.3.1, initialize i=1;

[0025] Step 3.3.2, initialize j=1;

[0026] Step 3.3.3, initialize l=1;

[0027] Step 3.3.4: The first fully connected layer uses formula (1) to predict the starting position of the head entity:

[0028]

[0029] In formula (1), Represents the i-th sentence sequence x i The jth word is the probability of the starting position of the head entity, W subject_start ,b subject_start is the parameter of the first fully connected layer, Sigmoid represents the sigmoid activation function;

[0030] Step 3.3.2: When , it means the i-th sentence sequence x i The jth word is the starting position of the head entity, and the jth word The characteristic information is recorded as And execute step 3.3.3, where threshold is the threshold;

[0031] Step 3.3.3: The second fully connected layer uses formula (2) to predict the end position of the head entity:

[0032]

[0033] In formula (2), Represents the i-th sentence sequence x i The jth word is the probability of the end position of the head entity, W subject_end ,b subject_end is the parameter of the second fully connected layer;

[0034] Step 3.3.4: When , it means the i-th sentence sequence x i The jth word The end position of the head entity;

[0035] Step 3.3.5: According to the starting position and ending position of the head entity, get the i-th sentence sequence x i The lth head entity Vector information

[0036] Step 3.4: Build a tail entity boundary decoding layer based on the pointer network:

[0037] According to the number of relation types m, build m tail entity boundary decoding layers, and each tail entity boundary decoding layer consists of two fully connected layers;

[0038] Step 3.4.1: The first fully connected layer in the l-th tail entity boundary decoding layer uses formula (3) to predict the current l-th relationship type r l Lower entity The corresponding tail entity starting position information:

[0039]

[0040] In formula (3), Represents the i-th sentence sequence x i The jth word is the probability of the starting position of the tail entity, is the lth relationship type r l Parameters of the first fully connected layer in the next l-th tail entity boundary decoding layer;

[0041] Step 3.4.2: When , it means the i-th sentence sequence x i The jth word For the lth relationship type r l The starting position of the tail entity under the j-th word The characteristic information is recorded as And execute step 3.4.3;

[0042] Step 3.4.3: The second fully connected layer in the l-th tail entity boundary decoding layer uses formula (4) to predict the current l-th relationship type r l Lower entity The corresponding tail entity end position information:

[0043]

[0044] In formula (4), Represents the jth word is the probability of the end position of the tail entity, are the parameters of the second fully connected layer in the l-th tail entity boundary decoding layer;

[0045] Step 3.4.4: When , it means the i-th sentence sequence x i The jth word The end position of the tail entity;

[0046] Step 3.4.5: According to the starting position and ending position of the tail entity, get the i-th sentence sequence x i Relationship type r l i and the corresponding lth tail entity Thus, the lth predicted triplet of the i-th sentence sequence is

[0047] Step 3.4.6, assign l+1 to l, and return to step 3.4.4 and execute sequentially until l>m;

[0048] Step 3.4.7, assign j+1 to j, and return to step 3.4.3 to execute sequentially until j>N; then get the prediction result

[0049] Step 3.4.7, assign i+1 to i, and return to step 3.4.3 to execute sequentially until i>M; then get the prediction result y={y1,y2,...y i ,...,y M};

[0050] Step 4: Use formula (5) to establish the binary cross entropy loss function L:

[0051]

[0052] The deep neural network model is trained using the Adam optimizer, and the binary cross entropy loss function L is calculated. The error is back-propagated to the deep neural network model to update the model parameters until the loss L does not continue to decrease or the total number of iterations reaches the maximum number of iterations epoch max Training stops; thus, an optimal deep learning model is obtained for predicting the relationships between entities in unlabeled data.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. Different from the traditional entity recognition and relationship prediction methods, the present invention implements an end-to-end deep learning model, which can effectively learn the feature information of words, thus avoiding the manual design of dictionaries, rules and other feature extraction work;

[0055] 2. The present invention adopts a joint prediction model, in which entity recognition and relationship prediction are performed simultaneously, which can effectively avoid the problem of error propagation and make full use of the interactive information between the two subtasks;

[0056] 3. The present invention predicts the head entity first, and then predicts the tail entity according to the relationship type, which can effectively solve the problem of triple entity overlap, and the prediction result is more accurate and complete. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0058] In this embodiment, a method for biomedical entity recognition and relationship prediction based on deep learning is provided. The specific method flow chart is shown in Figure 1, and the method is performed in the following steps:

[0059] Step 1: Supervised learning methods require a large amount of labeled training data sets. The first step is to obtain and label the required training data sets for this article.

[0060] Step 1.1: Obtain unstructured text of biomedical literature;

[0061] Step 1.2: Preprocess the unstructured text by segmenting sentences and words and removing noise data to obtain a structured unlabeled text training data set, denoted as X = {x1, x2, ..., x i ,...,x M}; where M represents the number of sentences in a batch; x i represents the i-th sentence sequence, and has Where N represents the number of words in the sentence sequence; Represents the i-th sentence sequence x i The jth word in ;

[0062] Step 2: Manually label the structured unlabeled text training dataset X:

[0063] Step 2.1: Annotation of biomedical named entities:

[0064] The biomedical named entities in the sentence sequence are manually annotated, and the entity type is recorded as: L = {L1, L2, ..., L k ,...,L n}, where n represents the total number of named entity types; L k represents the kth entity type;

[0065] Step 2.2: Label entity relationships;

[0066] The relationships between entities in the sentence sequence are manually annotated, and the relationship type is recorded as R = {r1, r2, ..., r l ,...,r m}, where m represents the total number of relationship types between entities; r l Indicates the lth relationship type;

[0067] Get the i-th sentence sequence x i The triples of m entity relations in are recorded as in, Represents the i-th sentence sequence x i The lth triple in ; represents the lth head entity, represents the lth relationship type, represents the t-th tail entity, So as to obtain the data label set

[0068] Step 3: Build a deep neural network model consisting of the bioBERT encoding layer, batch normalization layer, and entity boundary decoding layer. The main idea of ​​the model is to first identify the head entity s, then find the corresponding tail entity o according to the relationship type r, and finally get the entity relationship triple (s, r, o):

[0069] Step 3.1. Processing of bioBERT encoding layer:

[0070] The main function of the first encoding layer is to encode text data into a vector form that can be processed by machines. For each word and symbol, a vector can be generated based on the sentence. These vectors contain the correlation and similarity between words and symbols. In the encoding layer, we use the pre-trained bioBERT model, which is designed to learn deep representations by jointly acting on the left and right contexts of each word;

[0071] The text training dataset X is input into the bioBERT encoding layer to obtain the feature information of each word in all sentence sequences, denoted as H = {H1, H2, ..., H i ,...,H M}, where H i Represents the i-th sentence sequence x i The eigenvector of , and: H i ={H i,1 ,H i,2 ,...,H i,j ,...,H i,N}, H i,j Represents the i-th sentence sequence x i The jth word in characteristic information;

[0072] Step 3.2, batch normalization layer processing:

[0073] The second layer is the BatchNormalization layer. The batch normalization layer is similar to ordinary data standardization. It is a way to unify scattered data and a way to optimize neural networks. Data with uniform specifications can make it easier for machine learning to learn the rules in the data.

[0074] After the feature information H is input into the batch normalization layer for normalization, it is scaled and translated to obtain the feature distribution H′={H′1,H′2,...,H′ i ,...,H′ M}, where H′ i Represents the i-th sentence sequence x i The feature vector after the normalization layer has: H′ i ={h i,1 ,h i,2 ,...,h i,j ,...,h i,N}, h i,j Represents the i-th sentence sequence x i The jth word in feature information, N represents the number of words in the sentence sequence;

[0075] Step 3.3: Build a head entity boundary decoding layer based on the pointer network. The head entity boundary decoding layer consists of two fully connected layers.

[0076] Step 3.3.1, initialize i = 1; start from the first sentence sequence and execute the following steps in sequence until i <M;

[0077] Step 3.3.2, initialize j = 1; for the i-th sentence sequence x i Perform the following steps from the first word to the last word until j <N;

[0078] Step 3.3.3, initialize l = 1; for each head entity, perform the following steps starting from the first relationship type until l <n;

[0079] Step 3.3.4: The first fully connected layer uses formula (1) to predict the starting position of the head entity:

[0080]

[0081] In formula (1), Represents the i-th sentence sequence x i The jth word is the probability of the starting position of the head entity, W subject_start ,b subject_start is the parameter of the first fully connected layer, Sigmoid represents the sigmoid activation function;

[0082] Step 3.3.2: When , it means the i-th sentence sequence x i The jth word is the starting position of the head entity, and the jth word The characteristic information is recorded as And execute step 3.3.3, where threshold is the threshold;

[0083] Step 3.3.3: The second fully connected layer uses formula (2) to predict the end position of the head entity:

[0084]

[0085] In formula (2), Represents the i-th sentence sequence x i The jth word is the probability of the end position of the head entity, W subject_end ,b subject_end is the parameter of the second fully connected layer;

[0086] Step 3.3.4: When , it means the i-th sentence sequence x i The jth word The end position of the head entity;

[0087] Step 3.3.5: According to the starting position and ending position of the head entity, get the i-th sentence sequence x i The lth head entity Vector information At this point, the header entity information has been identified;

[0088] Step 3.4: Build a tail entity boundary decoding layer based on the pointer network:

[0089] According to the number of relation types m, build m tail entity boundary decoding layers, and each tail entity boundary decoding layer consists of two fully connected layers;

[0090] Step 3.4.1: The first fully connected layer in the l-th tail entity boundary decoding layer uses formula (3) to predict the current l-th relationship type r l Lower entity The corresponding tail entity starting position information:

[0091]

[0092] In formula (3), Represents the i-th sentence sequence x i The jth word is the probability of the starting position of the tail entity, is the lth relationship type r l Parameters of the first fully connected layer in the next l-th tail entity boundary decoding layer;

[0093] Step 3.4.2: When , it means the i-th sentence sequence xi The jth word For the lth relationship type r l The starting position of the tail entity under the jth word The characteristic information is recorded as And execute step 3.4.3;

[0094] Step 3.4.3: The second fully connected layer in the l-th tail entity boundary decoding layer uses formula (4) to predict the current l-th relationship type r l Lower entity The corresponding tail entity end position information:

[0095]

[0096] In formula (4), Represents the jth word is the probability of the end position of the tail entity, are the parameters of the second fully connected layer in the l-th tail entity boundary decoding layer;

[0097] Step 3.4.4: When , it means the i-th sentence sequence x i The jth word The end position of the tail entity;

[0098] Step 3.4.5: According to the starting position and ending position of the tail entity, get the i-th sentence sequence x i Relationship type r l i and the corresponding lth tail entity Thus, the lth predicted triplet of the i-th sentence sequence is At this point, the head entity, relationship type, and tail entity information are all predicted;

[0099] Step 3.4.6, assign l+1 to l, and return to step 3.4.4 and execute sequentially until l>m;

[0100] Step 3.4.7, assign j+1 to j, and return to step 3.4.3 to execute sequentially until j>N; then get the prediction result

[0101] Step 3.4.7, assign i+1 to i, and return to step 3.4.3 to execute sequentially until i>M; then get the prediction result y={y1,y2,...y i ,...,y M};

[0102] Step 4: Model training uses the early stopping strategy, and the number of iterations currently started is epoch now , the initial value is set to 1, and the maximum number of iterations is epoch max , set the fixed value to 100, and use formula (5) to establish the binary cross entropy loss function L; the number of iterations for which the model loss L does not continue to decrease is epoch noimprove , set to a fixed value of 7:

[0103]

[0104] The deep neural network model is trained using the Adam optimizer, and the binary cross entropy loss function L is calculated. The error is back-propagated to the deep neural network model to update the model parameters until the number of iterations where L does not continue to decrease reaches epoch noimprove , or the total number of iterations reaches epoch max Training stops; thus, an optimal deep learning model is obtained for predicting the relationships between entities in unlabeled data.

[0105] The present invention adopts the method of finding the head entity first and then finding the tail entity according to the relationship type, which can effectively avoid the problem of triple entity overlap, improve the accuracy of the prediction results, and make the information more complete.

Claims

1. A method for biomedical entity recognition and relationship prediction based on deep learning, characterized by Proceed as follows: Step 1: Get the training data set: Step 1.1: Obtain unstructured text of biomedical literature; Step 1.2: The unstructured text is preprocessed by sentence segmentation and noise data removal to obtain a structured unlabeled text training data set, denoted as X = {x1, x2, ..., x i ,...,x M }; where M represents the number of sentences in a batch; x i represents the i-th sentence sequence, and has Where N represents the number of words in the sentence sequence; Represents the i-th sentence sequence x i The jth word in ; Step 2: Manually label the structured unlabeled text training dataset X: Step 2.1: Annotation of biomedical named entities: The biomedical named entities in the sentence sequence are manually annotated, and the entity type is recorded as: L = {L1, L2, ..., L k ,...,L n }, where n represents the total number of named entity types; L k represents the kth entity type; Step 2.2: Label entity relationships; The relationships between entities in the sentence sequence are manually annotated, and the relationship type is recorded as R = {r1, r2, ..., r l ,...,r m }, where m represents the total number of relationship types between entities; r l Indicates the lth relationship type; Get the i-th sentence sequence x i The triples of m entity relations in are recorded as in, Represents the i-th sentence sequence x i The lth triple in ; represents the lth head entity, represents the lth relationship type, represents the t-th tail entity, So as to obtain the data label set Step 3: Build a deep neural network model consisting of the bioBERT encoding layer, batch normalization layer, and entity boundary decoding layer: Step 3.

1. Processing of bioBERT encoding layer: The text training dataset X is input into the bioBERT encoding layer to obtain the feature information of each word in all sentence sequences, denoted as H = {H1, H2, ..., H i ,...,H M }, where H i Represents the i-th sentence sequence x i The eigenvector of , and: H i ={H i,1 ,H i,2 ,...,H i,j ,...,H i,N }, H i,j Represents the i-th sentence sequence x i The jth word in characteristic information; Step 3.2, batch normalization layer processing: The feature information H is input into the batch normalization layer for normalization, and then scaled and translated to obtain the feature distribution H′={H′1,H′2,...,H′ i ,...,H′ M }, where H′ i Represents the i-th sentence sequence x i The feature vector after the normalization layer has: H′ i ={h i,1 ,h i,2 ,...,h i,j ,...,h i,N }, h i,j Represents the i-th sentence sequence x i The jth word in feature information, N represents the number of words in the sentence sequence; Step 3.3: Build a header entity boundary decoding layer based on a pointer network, wherein the header entity boundary decoding layer consists of two fully connected layers; Step 3.3.1, initialize i=1; Step 3.3.2, initialize j=1; Step 3.3.3, initialize l=1; Step 3.3.4: The first fully connected layer uses formula (1) to predict the starting position of the head entity: In formula (1), Represents the i-th sentence sequence x i The jth word is the probability of the starting position of the head entity, W subject_start ,b subject_start is the parameter of the first fully connected layer, Sigmoid represents the sigmoid activation function; Step 3.3.2: When , it means the i-th sentence sequence x i The jth word is the starting position of the head entity, and the jth word The characteristic information is recorded as And execute step 3.3.3, where threshold is the threshold; Step 3.3.3: The second fully connected layer uses formula (2) to predict the end position of the head entity: In formula (2), Represents the i-th sentence sequence x i The jth word is the probability of the end position of the head entity, W subject _end ,b subject_end is the parameter of the second fully connected layer; Step 3.3.4: When , it means the i-th sentence sequence x i The jth word The end position of the head entity; Step 3.3.5: According to the starting position and ending position of the head entity, get the i-th sentence sequence x i The lth head entity Vector information Step 3.4: Build a tail entity boundary decoding layer based on the pointer network: According to the number of relation types m, build m tail entity boundary decoding layers, and each tail entity boundary decoding layer consists of two fully connected layers; Step 3.4.1: The first fully connected layer in the l-th tail entity boundary decoding layer uses formula (3) to predict the current l-th relationship type r l Lower entity The corresponding tail entity starting position information: In formula (3), Represents the i-th sentence sequence x i The jth word is the probability of the starting position of the tail entity, is the lth relationship type r l Parameters of the first fully connected layer in the next l-th tail entity boundary decoding layer; Step 3.4.2: When , it means the i-th sentence sequence x i The jth word For the lth relationship type r l The starting position of the tail entity under the j-th word The characteristic information is recorded as And execute step 3.4.3; Step 3.4.3: The second fully connected layer in the l-th tail entity boundary decoding layer uses formula (4) to predict the current l-th relationship type r l Lower entity The corresponding tail entity end position information: In formula (4), Represents the jth word is the probability of the end position of the tail entity, are the parameters of the second fully connected layer in the l-th tail entity boundary decoding layer; Step 3.4.4: When , it means the i-th sentence sequence x i The jth word The end position of the tail entity; Step 3.4.5: According to the starting position and ending position of the tail entity, get the i-th sentence sequence x i Relationship type r l i and the corresponding lth tail entity Thus, the lth predicted triplet of the i-th sentence sequence is Step 3.4.6, assign l+1 to l, and return to step 3.4.4 and execute sequentially until l>m; Step 3.4.7, assign j+1 to j, and return to step 3.4.3 to execute sequentially until j>N; then get the prediction result Step 3.4.7, assign i+1 to i, and return to step 3.4.3 to execute sequentially until i>M; then get the prediction result y={y1,y2,...y i ,...,y M }; Step 4: Use formula (5) to establish the binary cross entropy loss function L: The deep neural network model is trained using the Adam optimizer, and the binary cross entropy loss function L is calculated. The error is back-propagated to the deep neural network model to update the model parameters until the loss L does not continue to decrease or the total number of iterations reaches the maximum number of iterations epoch max Training stops; thus, an optimal deep learning model is obtained for predicting the relationships between entities in unlabeled data.