A knowledge graph embedding method based on deep learning network training

Abstract

The present invention provides a knowledge graph embedding method based on deep learning network training, which belongs to the technical field of deep learning and knowledge graphs. The method includes: obtaining multiple triplets, vectorizing and normalizing each triplet, and combining them into a training set; constructing a deep learning network, and initializing the parameters of the deep learning network; according to the training set, obtaining positive samples, and constructing negative samples according to preset probabilities; inputting positive and negative samples into the deep learning network for training, and using classification and scoring subnetworks to obtain classification prediction probabilities and sample scores respectively; according to the loss function, using the error backpropagation gradient descent training method, updating the parameters and input signals of the deep learning network; if the updated input signal vector meets the preset conditions, the knowledge graph embedding is completed, otherwise, the input signal vector is updated again; the present invention solves the problems that the existing knowledge graph embedding is difficult to extract the potential relationship between vectors and the training efficiency is low.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and knowledge graph technology, and in particular to a knowledge graph embedding method based on deep learning network training. Background Art

[0002] Knowledge graphs represent different things in the real world and their relationships in the form of triples, and are widely used in artificial intelligence, search engine optimization, intelligent question answering, system recommendation, information retrieval, knowledge reasoning, disease classification and other medical information. Triples include head entities, relationships and tail entities. Head entities and tail entities belong to the same entity set, and relationships belong to different sets of relationships. Each triple indicates that there is a correct relationship between a head entity and a tail entity. Due to the wide range of sources of knowledge graphs, the set of triples is extremely large, and entities and relationships are abstract concepts that are difficult to quantify, so it is necessary to use knowledge graph embedding methods. Knowledge graph embedding refers to the process of mapping entities and relationships in knowledge graphs to low-dimensional vector spaces, that is, using finite-length data vectors to represent entities and relationships, thereby converting abstract entities and relationships into corresponding numerical vectors.

[0003] The vigorous development of artificial intelligence has made people want to introduce deep learning networks into knowledge graph embedding, but due to the low training efficiency, there are few embedding methods based on convolutional neural network models and the network structure is simple; most models are based on a criterion: for real triples, the model output is low, also known as low score, and for false triples, the model scores are high. This type of algorithm trains such models by minimizing the scoring loss function, and completes the knowledge graph embedding after the model converges.

[0004] However, the current knowledge image embedding technology has the following problems: First, common knowledge graph embedding techniques are relatively simple and difficult to extract and utilize the potential relationships between vectors; Second: Due to the huge number of knowledge graphs, the training efficiency of models with more parameters is low, which hinders the research and application of deep learning models in knowledge graph embedding. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a knowledge graph embedding method based on deep learning network training, which solves the problems of difficulty in extracting potential relationships between vectors and low training probability in the existing knowledge graph embedding technology.

[0006] In order to achieve the above objectives, the technical solution adopted by the present invention is: a knowledge graph embedding method based on deep learning network training, comprising the following steps: S1. Represent each entity and relationship in the entity set and relationship set with a randomly generated vector that obeys a uniform distribution and has a preset dimension, normalize each vector, extract the corresponding relationship and entity vectors to form a triple vector and combine them into a training set; S2. Build a deep learning network and initialize the parameters of the deep learning network using a randomization method; S3. According to the training set, positive samples are obtained by random sampling, and negative samples are constructed according to the preset probability by using the nearest neighbor method and the common method; S4, inputting positive samples and negative samples into the deep learning network as input signals for training to obtain low-level feature signals, and inputting the low-level feature signals into the classification sub-network and the scoring sub-network to obtain the classification prediction probability and sample score respectively; S5. Update the parameters and input signals of the deep learning network according to the loss function, classification prediction probability and sample score by using the error backpropagation gradient descent training method, and normalize the updated input signals; S6. If the normalized updated input signal vector meets the preset conditions, the knowledge graph embedding is completed; otherwise, return to step S3.

[0007] The beneficial effects of the present invention are as follows: the present invention utilizes the nearest neighbor negative sample construction method to focus the learning of the deep learning network on the triple vector, thereby accelerating the convergence speed of the deep learning network and improving the learning performance of the deep learning network; constructing the deep learning network, the present invention has the ability to extract potential information between triplets and reduces the impact of some samples on the output of the deep learning network; utilizing two different loss functions, classification and scoring, to accelerate the convergence speed of the deep learning network and increase the feature extraction capability of the deep learning network; and solving the problem of difficulty in extracting potential relationships between vectors and low training efficiency in existing knowledge graph embedding technology.

[0008] Furthermore, the S1 comprises the following steps: S101. According to the entity set and the relationship set, each entity and relationship in the entity set and the relationship set is represented by a vector that is randomly generated, uniformly distributed, and has a preset dimension, and each vector is normalized to obtain an entity vector set and a relationship vector set; S102. According to the entity vector set and the relationship vector set, corresponding vectors in the triples are extracted to form triple vectors, and the triple vectors are combined into a training set.

[0009] The beneficial effect of the above further scheme is: the present invention vectorizes each relationship and entity in the entity set and relationship set and normalizes them into triple vectors, combines them to obtain a training set, improves the processing speed of the method of the present invention, and makes the potential relationship of the triple vector easier to extract.

[0010] Furthermore, the S3 comprises the following steps: S301, according to the training set, obtain random triples by random sampling, and use the random triples as positive samples; S302, using a random number generator to generate random numbers uniformly distributed in a preset interval. If the random number is greater than a preset threshold, a nearest neighbor method is used to construct a negative sample, otherwise a common method is used to construct a negative sample.

[0011] Furthermore, the specific steps of constructing negative samples by the nearest neighbor method in S302 are as follows: According to the three components in the positive sample, a component is randomly selected as the replacement component, and according to the replacement component, multiple negative sample components are randomly selected using the entity set, and the sample distance between each negative sample and the corresponding positive sample component is calculated. The sample distances are sorted, and the negative sample component corresponding to the minimum sample distance is selected. The negative sample component replaces the replacement component in the positive sample to construct a negative sample.

[0012] The beneficial effect of the above further scheme is: the present invention uses the nearest negative sample construction method to focus the network training on input signals with higher value, thereby improving the accuracy of the method of the present invention and enhancing the knowledge graph embedding effect.

[0013] Furthermore, the S4 comprises the following steps: S401, constructing a high-dimensional matrix using positive samples and corresponding negative samples, and inputting the high-dimensional matrix into a deep learning network as an input signal for training; S402, using a convolution layer and a pooling layer to process the input signal to obtain a low-level feature signal; S403, inputting the low-level feature signal into the classification sub-network and the scoring sub-network; S404, using the flattening layer in the classification subnetwork to flatten the low-level feature signal into a one-dimensional signal, after being connected through the fully connected layer, the classification prediction probability is calculated using a normalized exponential function; S405. Use the convolution module in the scoring subnetwork to extract high-level features of the low-level feature signal, use the maximum pooling layer to reduce the size of the high-level features, and calculate the sample score output by the scoring function.

[0014] The beneficial effects of the above further scheme are as follows: the present invention accelerates the convergence speed of the deep learning network and improves the ability of the deep learning network to utilize deep information by placing the classification network at the lower layer of the backbone network and the scoring network at the higher layer of the backbone network; and adopts multi-layer convolution to improve the ability of the deep learning network to extract and utilize deep features between vectors, and can process features within the same vector.

[0015] Furthermore, the S5 comprises the following steps: S501, calculating a first loss function according to the classification prediction probability and the true label, calculating a second loss function according to the sample score, and obtaining a total loss function by using the first loss function and the second loss function; S502, according to the second loss function, obtain the gradient matrix expression of the beater sub-network, and according to the output layer gradient matrix of the beater sub-network, use the error backpropagation gradient descent method to calculate the gradient of each layer except the output layer in turn, and calculate the gradient of each layer of the beater sub-network for the second loss function; S503, according to the first loss function, obtain the gradient matrix expression of the classification sub-network, and according to the output layer gradient matrix of the classification sub-network, use the error backpropagation gradient descent method to calculate the gradient of each layer except the output layer in turn, and calculate the gradient of each layer of the classification sub-network for the first loss function; S504, according to the gradient of the beating sub-network and the gradient of the classification sub-network, calculate the gradient of the junction of the beating sub-network and the classification sub-network, and calculate the gradient of the lower-layer network; S505. Use the gradient of each layer of the classification subnetwork, the gradient of each layer of the scoring subnetwork, the gradient at the junction of the scoring subnetwork and the classification subnetwork, and the gradient of the lower-layer network to update the coefficients and input signal of the deep learning network, and normalize the updated input signal vector.

[0016] Furthermore, the expression of the total loss function is as follows: ; ; ; in, represents the total loss function, represents the first loss function, represents the second loss function, represents the regularization coefficient, Represents the model coefficient The second-order norm of represents the total number of positive samples, represents the true label, Indicates j The classification prediction probability, represents the judgment coefficient, Indicates that the current input sample is a positive sample, Indicates that the current input sample is a negative sample. Indicates j Sample scores.

[0017] Furthermore, the expression of the gradient matrix of the subnetwork is as follows: ; ; ; in, represents the gradient matrix of the molecular network, Indicates j Positive sample gradients, Indicates j negative sample gradients, Indicates j The sample fraction of positive samples, Indicates j The sample fraction of negative samples.

[0018] Furthermore, the expression of the gradient matrix of the classification subnetwork is as follows: ; in, Represents the classification subnetwork k The gradient matrix, , Indicates the second i The classification prediction probability of positive samples is Indicates the second i +1 classification prediction probability of negative samples, Indicates indivual, .

[0019] The beneficial effects of the above further scheme are as follows: the present invention improves the convergence speed of the deep learning network and the accuracy of the output data, broadens the application breadth of the present invention and saves computing costs by using classification loss function and scoring loss function, and combining error feedback gradient descent training; and achieves better embedding effect by extracting nonlinear, high-level potential information between entities and relationships. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The present invention is a flow chart of the method.

[0021] Figure 2 Schematic diagram of the structure of the deep learning network constructed in this embodiment.

[0022] Figure 3 Schematic diagram of the structure of the residual module in this embodiment.

[0023] Figure 4 Schematic diagram of the construction principle of the high-dimensional matrix in this embodiment.

[0024] Figure 5 The figure is a flow chart of the method applied to the field of artificial intelligence. DETAILED DESCRIPTION

[0025] The specific implementation modes of the present invention are described below to facilitate those skilled in the art to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0026] Example 1 In this embodiment, one of the main reasons that hinder deep learning from being embedded in the knowledge graph is that the model training is slow, and the reason for this result is that the training set is large, the deep learning network has many parameters, and the convergence speed is slow. In response to these two problems, the present invention provides a method for fast training using a deep learning network for knowledge graph embedding. First, although there are too many training data, the data that affect the convergence speed of the network are mainly data with close sample distances, while data with large differences between samples have little effect on training performance and speed. Therefore, the present invention provides a nearest negative sample construction method, so that negative samples that are close to positive samples are more and more frequently input into the network for training, while negative samples that are far away have a lower training frequency; secondly, since the convergence process of the deep learning network often starts from the low layer and slowly expands to the high layer, the present invention uses both binary classification and scoring functions as outputs in the deep learning network construction, and places the classification sub-network at the lower layer of the backbone network, while the scoring sub-network is at a higher layer. In addition to being closer to the lower layers, since the classification sub-network only judges positive and negative samples and only requires rough features, it converges faster, helps stabilize the output features of the lower-layer networks, and makes it easier for the higher-level classification sub-networks to converge.

[0027] In this embodiment, the main contents of the present invention are: designing a nearest negative sample construction method to speed up the convergence speed; taking the triplet vector as input, using a deep learning network to extract the potential information between the triplets; using two different loss functions, classification and scoring, corresponding to two different output paths, which are respectively connected to different positions of the backbone network, with the classification output in front and the scoring subnetwork in the back, so that the network converges faster and the extracted feature energy is stronger.

[0028] like Figure 1 As shown, the present invention provides a knowledge graph embedding method based on deep learning network training, and its implementation method is as follows: S1. Represent each entity and relationship in the entity set and relationship set with a randomly generated vector that obeys a uniform distribution and has a preset dimension, normalize each vector, extract the corresponding relationship and entity vectors to form a triple vector and combine them into a training set. The specific steps are as follows: S101. According to the entity set and the relationship set, each entity and relationship in the entity set and the relationship set is represented by a vector that is randomly generated, uniformly distributed, and has a preset dimension, and each vector is normalized to obtain an entity vector set and a relationship vector set; S102. According to the entity vector set and the relationship vector set, corresponding vectors in the triples are extracted to form triple vectors, and the triple vectors are combined into a training set.

[0029] In this embodiment, each entity in the entity set and each relationship in the relationship set are randomly generated, uniformly distributed, and have a preset dimension. In this embodiment, a suitable integer is selected N Make , H Represents an integer, suitable N Convenient for constructing input high-dimensional matrices; Each vector in the entity set and the relationship set is normalized to obtain an entity vector set and a relationship vector set, and the corresponding vectors in the triples are extracted from the entity vector set and the relationship vector set to form a triple vector, which is combined into a training set. The expression of the vector normalization is as follows: ; ; ; in, represents normalization, vectors representing entities and relations, Represents the components contained in each vector , represents the length of the vector, Representation vector The mold length.

[0030] S2. Build a deep learning network and initialize the parameters of the deep learning network using a randomization method.

[0031] In this embodiment, construct Figure 2 The deep learning network shown in the figure comprises: a plurality of cascaded residual modules, a first maximum pooling layer, a classification sub-network, and a scoring sub-network; The plurality of cascaded first residual modules are connected to the first maximum pooling layer; The first maximum pooling layer is connected to the classification sub-network and the scoring sub-network respectively; The classification subnetwork includes: a first flattening layer, a first fully connected layer, and a probability calculation layer connected in sequence; The sub-network includes: a second residual module, a second maximum pooling layer, a second flattening layer and a second fully connected layer connected in sequence; like Figure 3 As shown, the residual module includes: a first convolution layer, a second convolution layer, a first activation function, a third convolution layer and a second activation function; The second convolution layer is connected to the first activation function; the first activation function is connected to the third convolution layer; the first convolution layer is respectively connected to the third convolution layer and the second activation function; the signal input to the residual module is transmitted to the first convolution layer and the second convolution layer respectively; Initialize the parameters of the deep learning network using randomization.

[0032] In this embodiment, taking into account that the lower layers of the deep learning network converge faster than the higher layers, the classification network is placed in the lower layers of the backbone network, thereby accelerating the convergence speed of the deep learning network. Through the rapid convergence of the classification task, the feature output of the lower layer network is stabilized to a certain extent, and the convergence speed of the high-level scoring function is increased, thereby improving the convergence speed of the overall deep learning network. Placing the scoring network in the higher layers of the backbone network improves the ability of the deep learning network to utilize deep information, so that the deep learning network can quickly extract rough features with classification in the input signal in the early stage of training, and will pay more attention to fine features in the later stage of training so that the output score can better reflect the difference between positive and negative samples.

[0033] S3. According to the training set, positive samples are obtained by random sampling, and negative samples are constructed according to the preset probability by using the nearest neighbor method and the common method. The specific steps are as follows: S301, according to the training set, obtain random triples by random sampling, and use the random triples as positive samples; S302, using a random number generator to generate random numbers uniformly distributed in a preset interval. If the random number is greater than a preset threshold, a nearest neighbor method is used to construct a negative sample, otherwise a common method is used to construct a negative sample.

[0034] In this embodiment, based on the training set, a triplet in the training set is randomly selected. As a positive sample; Using a random number generator, Generate a uniformly distributed random number. If the random number is greater than the preset threshold, the nearest neighbor method is used to construct the negative sample set. Otherwise, the common method is used to construct the negative sample. The nearest neighbor method constructs negative samples as follows: , randomly select a component h or r or t As an alternative component, assume that the alternative component is h , and according to the substitution component h , using entity sets to randomly select multiple negative sample components , , m Represents the number of randomly selected negative sample components, and calculates the sample distance between each negative sample and the corresponding positive sample component , for each sample distance Sort and select the minimum sample distance The corresponding negative sample component , the negative sample component Replace the replacement component in the positive sample h , construct negative samples ; The expression of the sample distance between the negative sample and the corresponding positive sample component is as follows: ; in, Indicates q The sample distance between the negative sample and the positive sample component; The negative samples are constructed in a common way as follows: , randomly select a component as the replacement component, assuming that the replacement component is h , and according to the substitution component h , using the entity set to randomly select a negative sample component , the negative sample component Replace the replacement component in the positive sample h , which constitutes a negative sample .

[0035] S4. Input the positive samples and negative samples into the deep learning network as input signals for training to obtain low-level feature signals, and input the low-level feature signals into the classification sub-network and the scoring sub-network to obtain the classification prediction probability and sample score respectively. The specific steps are as follows: S401, constructing a high-dimensional matrix using positive samples and corresponding negative samples, and inputting the high-dimensional matrix into a deep learning network as an input signal for training; S402, using a convolution layer and a pooling layer to process the input signal to obtain a low-level feature signal; S403, inputting the low-level feature signal into the classification sub-network and the scoring sub-network; S404, using the flattening layer in the classification subnetwork to flatten the low-level feature signal into a one-dimensional signal, after being connected through the fully connected layer, the classification prediction probability is calculated using a normalized exponential function; S405. Use the convolution module in the scoring subnetwork to extract high-level features of the low-level feature signal, use the maximum pooling layer to reduce the size of the high-level features, and calculate the sample score output by the scoring function.

[0036] In this embodiment, the positive samples and negative samples are input into the deep learning network as input signals for training, such as Figure 4 As shown, first the positive samples and the corresponding negative samples are The dimensional input signal is converted into a high-dimensional matrix signal. The dimension of the high-dimensional matrix signal is ,in, ,and ; The constructed high-dimensional matrix can provide the internal information of each vector and the intrinsic connection between vectors by using convolution; In the deep learning network, multiple cascaded residual modules are used to extract the features of high-dimensional matrix signals, and the extracted features are reduced in size using the maximum pooling layer to obtain low-level feature signals; The low-level feature signal is input into the classification sub-network and the scoring sub-network; in the classification sub-network, the input low-level feature signal is flattened into a one-dimensional signal through the flattening layer, and after being connected through the fully connected layer, the classification prediction probability is calculated using the normalization function to obtain the classification prediction probability output by the classification sub-network; the expression of the normalization function for calculating the classification prediction probability is as follows: ; in, represents the classification prediction probability, and Represents the output signal, thinking that the output signal has only two categories, positive class, then i and k Take 0; negative class, then i and k Take 1; In the scoring subnetwork, the convolution module in the scoring subnetwork is used to extract the high-level features of the low-level feature signals, and the maximum pooling layer is used to reduce the size of the high-level features to calculate the sample scores output by the scoring function; the present invention uses a linear function for scoring, and the expression of the scoring function is as follows: ; in, represents the output coefficient of the fully connected layer, Represents the dot product.

[0037] S5. Based on the loss function, classification prediction probability and sample score, the parameters and input signals of the deep learning network are updated using the error backpropagation gradient descent training method, and the updated input signals are normalized. The specific steps are as follows: S501, calculating a first loss function according to the classification prediction probability and the true label, calculating a second loss function according to the sample score, and obtaining a total loss function by using the first loss function and the second loss function; S502, according to the second loss function, obtain the gradient matrix expression of the beater sub-network, and according to the output layer gradient matrix of the beater sub-network, use the error backpropagation gradient descent method to calculate the gradient of each layer except the output layer in turn, and calculate the gradient of each layer of the beater sub-network for the second loss function; S503, according to the first loss function, obtain the gradient matrix expression of the classification sub-network, and according to the output layer gradient matrix of the classification sub-network, use the error backpropagation gradient descent method to calculate the gradient of each layer except the output layer in turn, and calculate the gradient of each layer of the classification sub-network for the first loss function; S504, according to the gradient of the beating sub-network and the gradient of the classification sub-network, calculate the gradient of the junction of the beating sub-network and the classification sub-network, and calculate the gradient of the lower-layer network; S505. Use the gradient of each layer of the classification subnetwork, the gradient of each layer of the scoring subnetwork, the gradient at the junction of the scoring subnetwork and the classification subnetwork, and the gradient of the lower-layer network to update the coefficients and input signal of the deep learning network, and normalize the updated input signal vector.

[0038] In this embodiment, according to the loss function, the parameters and input signals of the network are updated in the manner of error backpropagation gradient descent training, and then the input signal is normalized; the first loss function is calculated according to the classification prediction probability and the true label, the second loss function is calculated according to the sample score, and the first loss function and the second loss function are combined with the model coefficient to obtain the total loss function; the expression of the total loss function is as follows: ; ; ; in, represents the total loss function, represents the first loss function, represents the second loss function, represents the regularization coefficient, Represents the model coefficient The second-order norm of is used to reduce overfitting, Represents the total number of positive samples, which is the number of positive samples input at one time, plus the same number of negative samples. Total number of samples, represents the true label, Indicates j The classification prediction probability, represents the judgment coefficient, Indicates that the current input sample is a positive sample, Indicates that the current input sample is a negative sample. Indicates j Sample scores; Calculate the gradient of each layer of the second loss function for the beater sub-network: According to the second loss function, the gradient expression of the beater sub-network is obtained, and according to the gradient of the output layer of the beater sub-network, the gradient of each remaining layer is calculated in turn by the error backpropagation gradient descent method to obtain the gradient of each layer of the second loss function for the beater sub-network; the expression of the gradient matrix of the beater sub-network is as follows: ; ; ; in, represents the gradient matrix of the molecular network, Indicates j Positive sample gradients, Indicates j negative sample gradients, Indicates j The sample fraction of positive samples, Indicates j The sample fraction of negative samples; Calculate the gradient of each layer of the classification subnetwork for the first loss function: Since a pair of positive samples and negative samples are input during training, it is necessary to calculate the gradient matrices of the positive samples and negative samples at the same time; According to the first loss function, the gradient matrix expression of the classification subnetwork is obtained, and according to the output layer gradient matrix of the classification subnetwork, the gradient matrices of each remaining layer are calculated in turn in the error backpropagation gradient descent method to obtain the gradient of each layer of the classification subnetwork for the first loss function; The expression of the gradient matrix of the classification subnetwork is as follows: ; in, Represents the classification subnetwork k The gradient matrix, , Indicates the second i The classification prediction probability of positive samples is Indicates the second i +1 classification prediction probability of negative samples, Indicates indivual, ; By calculating the gradient at the junction of the scoring subnetwork and the classification subnetwork, the gradient of the lower-level shared network is calculated, including the gradient passed to the input signal; using the required gradient, the coefficients and input signal of the deep learning network are updated, and the updated input signal vector is calculated. Normalize and generate an updated triplet vector.

[0039] S6. If the normalized updated input signal vector meets the preset conditions, the knowledge graph embedding is completed; otherwise, return to step S3.

[0040] In this embodiment, according to the preset stopping condition: the classification accuracy is greater than the preset classification probability, and the total loss function Loss is less than a preset small positive number; if the stopping condition is met, it means that the knowledge graph embedding is completed, otherwise, return to the step of selecting positive and negative samples according to the updated triple vector, and update the triple vector again.

[0041] Example 2 The present invention is further described below. The knowledge graph embedding method of the present invention can be widely used in the field of artificial intelligence. For optimizing search engines, the accuracy and relevance of search results can be improved by embedding entities and relationships; for recommendation systems, the system can better understand the similarities between users and items to provide more accurate recommendations; in information retrieval, entities can be quickly located and matched; for intelligent question answering, queries about entities and relationships can be answered by finding similar relationships in the embedding space; in knowledge reasoning, similar entities or relationships can be found by comparing the distances between embedded vectors, thereby inferring new relationships or facts that may exist, and predicting relationships between entities; in the medical field, a large amount of medical data can be integrated and analyzed through knowledge graphs to help doctors make diagnoses, develop drugs, etc.

[0042] In this embodiment, Figure 5 As shown, the present invention is implemented in any field of artificial intelligence as follows: A1. Represent each entity and relationship in the entity set and relationship set related to artificial intelligence with a randomly generated vector that obeys uniform distribution and has a preset dimension, normalize each vector, extract the corresponding relationship and entity vectors to form a triple vector and combine them into an artificial intelligence training set. The specific steps are as follows: A101. According to the entity set and relationship set related to artificial intelligence, use a randomly generated vector that obeys uniform distribution and has a preset dimension to represent each entity and relationship in the entity set and relationship set, and normalize each vector to obtain an entity vector set and a relationship vector set related to artificial intelligence; A102. According to the entity vector set and relationship vector set related to artificial intelligence, the corresponding vectors in the triples are extracted to form triple vectors, and the triple vectors are combined into an artificial intelligence training set.

[0043] In this embodiment, each entity in the entity set related to artificial intelligence and each relationship in the relationship set related to artificial intelligence are randomly generated, uniformly distributed, and have a dimension of a preset dimension. In this embodiment, a suitable integer is selected N Make , H Represents an integer, suitable N Convenient for constructing input high-dimensional matrices; Each vector in the entity set and relationship set related to artificial intelligence is normalized to obtain the entity vector set and relationship vector set related to artificial intelligence, and the corresponding vectors in the triples are extracted from the entity vector set and relationship vector set related to artificial intelligence to form triple vectors, which are combined into an artificial intelligence training set. The expression of the vector normalization is as follows: ; ; ; in, represents normalization, vectors representing AI entities and AI relationships, Represents the components contained in each vector , represents the length of the vector, Representation vector The mold length.

[0044] In this embodiment, each relationship and entity in the entity set and relationship set related to artificial intelligence is represented by a vector and normalized into a triple vector, which is combined to obtain an artificial intelligence training set, thereby improving the processing speed of the method of the present invention and making it easier to extract the potential relationship of the triple vector.

[0045] A2. Build a deep learning network and initialize the parameters of the deep learning network using a randomization method.

[0046] In this embodiment, a deep learning network is constructed, and the deep learning network includes: a plurality of cascaded residual modules, a first maximum pooling layer, a classification sub-network, and a scoring sub-network; The plurality of cascaded first residual modules are connected to the first maximum pooling layer; The first maximum pooling layer is connected to the classification sub-network and the scoring sub-network respectively; The classification subnetwork includes: a first flattening layer, a first fully connected layer, and a probability calculation layer connected in sequence; The sub-network includes: a second residual module, a second maximum pooling layer, a second flattening layer and a second fully connected layer connected in sequence; The residual module includes: a first convolution layer, a second convolution layer, a first activation function, a third convolution layer and a second activation function; The second convolutional layer is connected to the first activation function; the first activation function is connected to the third convolutional layer; the first convolutional layer is connected to the third convolutional layer and the second activation function respectively; the signal input to the residual module is transmitted to the first convolutional layer and the second convolutional layer respectively; and the parameters of the deep learning network are initialized in a randomized manner.

[0047] A3. According to the artificial intelligence training set, positive samples are obtained by random sampling, and negative samples are constructed according to the preset probability using the nearest neighbor method and the common method. The specific steps are as follows: A301. According to the artificial intelligence training set, a random triplet is obtained by random extraction, and the random triplet is used as a positive sample; A302. Use a random number generator to generate random numbers uniformly distributed in a preset interval. If the random number is greater than a preset threshold, use the nearest neighbor method to construct a negative sample. Otherwise, use the common method to construct a negative sample.

[0048] In this embodiment, according to the artificial intelligence training set, a triplet in the artificial intelligence training set is randomly extracted. As a positive sample; Using a random number generator, Generate a uniformly distributed random number. If the random number is greater than the preset threshold, the nearest neighbor method is used to construct the negative sample set. Otherwise, the common method is used to construct the negative sample. The nearest neighbor method constructs negative samples as follows: , randomly select a component h or r or t As an alternative component, assume that the alternative component is h , and according to the substitution component h , using entity sets to randomly select multiple negative sample components , , m Represents the number of randomly selected negative sample components, and calculates the sample distance between each negative sample and the corresponding positive sample component , for each sample distance Sort and select the minimum sample distance The corresponding negative sample component , the negative sample component Replace the replacement component in the positive sample h , construct negative samples ; The expression of the sample distance between the negative sample and the corresponding positive sample component is as follows: ; in, Indicates q The sample distance between the negative sample and the positive sample component; The negative samples are constructed in a common way as follows: , randomly select a component as the replacement component, assuming that the replacement component is h , and according to the substitution component h , using the entity set to randomly select a negative sample component , the negative sample component Replace the replacement component in the positive sample h , which constitutes a negative sample .

[0049] In this embodiment, the nearest negative sample construction method is used to focus network training on artificial intelligence input signals with higher value, thereby improving the accuracy of the method of the present invention and enhancing the artificial intelligence knowledge graph embedding effect.

[0050] A4. Input the positive samples and negative samples into the deep learning network as input signals for training to obtain low-level feature signals, and input the low-level feature signals into the classification sub-network and scoring sub-network to obtain the classification prediction probability and sample score respectively. The specific steps are as follows: A401. Use positive samples and corresponding negative samples to construct a high-dimensional matrix, and input the high-dimensional matrix into the deep learning network as an input signal for training; A402, using convolutional layer and pooling layer to process the input signal to obtain low-level feature signals; A403, input the low-level feature signal into the classification sub-network and the scoring sub-network; A404, using the flattening layer in the classification subnetwork, flatten the low-level feature signal into a one-dimensional signal, and after connecting it through the fully connected layer, use the normalized exponential function to calculate the classification prediction probability; A405. Use the convolution module in the scoring subnetwork to extract high-level features of low-level feature signals, use the maximum pooling layer to reduce the size of the high-level features, and calculate the sample score output by the scoring function.

[0051] In this embodiment, positive samples and negative samples are input into the deep learning network as input signals for training. The dimensional input signal is converted into a high-dimensional matrix signal. The dimension of the high-dimensional matrix signal is ,in, ,and ; The constructed high-dimensional matrix can provide the internal information of each vector and the intrinsic connection between vectors by using convolution; In the deep learning network, multiple cascaded residual modules are used to extract the features of high-dimensional matrix signals, and the extracted features are reduced in size using the maximum pooling layer to obtain low-level feature signals; The low-level feature signal is input into the classification sub-network and the scoring sub-network; in the classification sub-network, the input low-level feature signal is flattened into a one-dimensional signal through the flattening layer, and after being connected through the fully connected layer, the classification prediction probability is calculated using the normalization function to obtain the classification prediction probability output by the classification sub-network; the expression of the normalization function for calculating the classification prediction probability is as follows: ; in, represents the classification prediction probability, and Represents the output signal, thinking that the output signal has only two categories, positive class, then i and k Take 0; negative class, then i and k Take 1; In the scoring subnetwork, the convolution module in the scoring subnetwork is used to extract the high-level features of the low-level feature signals, and the maximum pooling layer is used to reduce the size of the high-level features to calculate the sample scores output by the scoring function; the present invention uses a linear function for scoring, and the expression of the scoring function is as follows: ; in, represents the output coefficient of the fully connected layer, Represents the dot product.

[0052] In this embodiment, by placing the classification network at the lower layer of the backbone network and the scoring network at the higher layer of the backbone network, the convergence speed of the deep learning network is accelerated and the ability of the deep learning network to utilize deep information is improved; and by using multi-layer convolution, the ability of the deep learning network to extract and utilize deep features between vectors is improved, and the features within the same vector can be processed.

[0053] A5. Based on the loss function, classification prediction probability and sample score, the parameters and input signals of the deep learning network are updated using the error backpropagation gradient descent training method, and the updated input signals are normalized. The specific steps are as follows: A501, calculate a first loss function according to the classification prediction probability and the true label, calculate a second loss function according to the sample score, and use the first loss function and the second loss function to obtain a total loss function; A502. According to the second loss function, the gradient matrix expression of the beater sub-network is obtained, and according to the output layer gradient matrix of the beater sub-network, the gradient of each layer except the output layer is calculated in turn by using the error backpropagation gradient descent method, and the gradient of each layer of the beater sub-network corresponding to the second loss function is calculated; A503. According to the first loss function, a gradient matrix expression of the classification subnetwork is obtained, and according to the output layer gradient matrix of the classification subnetwork, the gradient of each layer except the output layer is calculated in turn by using the error backpropagation gradient descent method, so as to obtain the gradient of each layer of the classification subnetwork with respect to the first loss function; A504. According to the gradient of the beating subnetwork and the gradient of the classification subnetwork, the gradient of the junction of the beating subnetwork and the classification subnetwork is calculated, and the gradient of the lower-layer network is calculated; A505. Use the gradient of each layer of the classification subnetwork, the gradient of each layer of the scoring subnetwork, the gradient at the junction of the scoring subnetwork and the classification subnetwork, and the gradient of the lower-layer network to update the coefficients and input signal of the deep learning network, and normalize the updated input signal vector.

[0054] In this embodiment, according to the loss function, the parameters and input signals of the network are updated in the manner of error backpropagation gradient descent training, and then the input signal is normalized; the first loss function is calculated according to the classification prediction probability and the true label, the second loss function is calculated according to the sample score, and the first loss function and the second loss function are combined with the model coefficient to obtain the total loss function; the expression of the total loss function is as follows: ; ; ; in, represents the total loss function, represents the first loss function, represents the second loss function, represents the regularization coefficient, Represents the model coefficient The second-order norm of is used to reduce overfitting, Represents the total number of positive samples, which is the number of positive samples input at one time, plus the same number of negative samples. Total number of samples, represents the true label, Indicates j The classification prediction probability, represents the judgment coefficient, Indicates that the current input sample is a positive sample, Indicates that the current input sample is a negative sample. Indicates j Sample scores; Calculate the gradient of each layer of the second loss function for the beater sub-network: According to the second loss function, the gradient expression of the beater sub-network is obtained, and according to the gradient of the output layer of the beater sub-network, the gradient of each remaining layer is calculated in turn by the error backpropagation gradient descent method to obtain the gradient of each layer of the second loss function for the beater sub-network; the expression of the gradient matrix of the beater sub-network is as follows: ; ; ; in, represents the gradient matrix of the molecular network, Indicates j Positive sample gradients, Indicates j negative sample gradients, Indicates j The sample fraction of positive samples, Indicates j The sample fraction of negative samples; Calculate the gradient of each layer of the classification subnetwork for the first loss function: Since a pair of positive samples and negative samples are input during training, it is necessary to calculate the gradient matrices of the positive samples and negative samples at the same time; According to the first loss function, the gradient matrix expression of the classification subnetwork is obtained, and according to the output layer gradient matrix of the classification subnetwork, the gradient matrices of each remaining layer are calculated in turn in the error backpropagation gradient descent method to obtain the gradient of each layer of the classification subnetwork for the first loss function; The expression of the gradient matrix of the classification subnetwork is as follows: ; in, Represents the classification subnetwork k The gradient matrix, , Indicates the second i The classification prediction probability of positive samples is Indicates the second i +1 classification prediction probability of negative samples, Indicates indivual, ; By calculating the gradient at the junction of the scoring subnetwork and the classification subnetwork, the gradient of the lower-level shared network is calculated, including the gradient passed to the input signal; using the required gradient, the coefficients and input signal of the deep learning network are updated, and the updated input signal vector is calculated. Normalize and generate an updated triplet vector.

[0055] In this embodiment, by using a classification loss function and a scoring loss function, and combining the error feedback gradient descent training method, the convergence speed of the deep learning network and the accuracy of the output data are improved, the application breadth of the present invention in artificial intelligence is broadened, and the computing cost is saved; by extracting nonlinear, high-level potential information between artificial intelligence entities and artificial intelligence relationships, a better artificial intelligence knowledge graph embedding effect is achieved.

[0056] A6. If the normalized updated input signal vector meets the preset conditions, the artificial intelligence knowledge graph embedding is completed; otherwise, return to step A3.

[0057] In this embodiment, according to the preset stopping condition: the classification accuracy is greater than the preset classification probability, and the total loss function Loss is less than a preset small positive number; if the stopping condition is met, it means that the artificial intelligence knowledge graph embedding is completed, otherwise, return to the step of selecting positive and negative samples according to the updated triple vector, and update the triple vector again.

[0058] In this embodiment, the present invention utilizes the nearest negative sample construction method to focus the learning of the deep learning network on the artificial intelligence triple vector, thereby accelerating the convergence speed of the deep learning network and improving the learning performance of the deep learning network; constructing a deep learning network, the present invention has the ability to extract potential information between artificial intelligence triples and reduces the impact of some samples on the output of the deep learning network; utilizing two different loss functions, classification and scoring, to accelerate the convergence speed of the deep learning network and increase the feature extraction capability of the deep learning network; and solving the problem of difficulty in extracting potential relationships between vectors and low training efficiency in existing artificial intelligence knowledge graph embedding technology.

Claims

1. A knowledge graph embedding method based on deep learning network training, characterized in that: The following steps are involved: S1. Represent each entity and relationship in the entity set and relationship set with a randomly generated vector that obeys a uniform distribution and has a preset dimension, normalize each vector, extract the corresponding relationship and entity vectors to form a triple vector and combine them into a training set; S2. Build a deep learning network and initialize the parameters of the deep learning network using a randomization method; S3. According to the training set, positive samples are obtained by random sampling, and negative samples are constructed according to the preset probability by using the nearest neighbor method and the common method; S4, inputting positive samples and negative samples into the deep learning network as input signals for training to obtain low-level feature signals, and inputting the low-level feature signals into the classification sub-network and the scoring sub-network to obtain the classification prediction probability and sample score respectively; S5. Update the parameters and input signals of the deep learning network according to the loss function, classification prediction probability and sample score by using the error backpropagation gradient descent training method, and normalize the updated input signals; S6. If the normalized updated input signal vector meets the preset conditions, the knowledge graph embedding is completed; otherwise, return to step S3.

2. The knowledge graph embedding method based on deep learning network training according to claim 1 is characterized in that: The S1 comprises the following steps: S101, according to the entity set and the relationship set, using a randomly generated vector that obeys uniform distribution and has a preset dimension to represent each entity and relationship in the entity set and the relationship set, and normalizing each vector to obtain an entity vector set and a relationship vector set; S102. According to the entity vector set and the relationship vector set, corresponding vectors in the triples are extracted to form triple vectors, and the triple vectors are combined into a training set.

3. The knowledge graph embedding method based on deep learning network training according to claim 1 is characterized in that: The S3 comprises the following steps: S301, according to the training set, obtain random triples by random sampling, and use the random triples as positive samples; S302, using a random number generator to generate random numbers uniformly distributed in a preset interval. If the random number is greater than a preset threshold, a nearest neighbor method is used to construct a negative sample, otherwise a common method is used to construct a negative sample.

4. The knowledge graph embedding method based on deep learning network training according to claim 2 is characterized in that: In S302, the negative sample is constructed using the nearest neighbor method, specifically as follows: According to the three components in the positive sample, a component is randomly selected as the replacement component, and according to the replacement component, multiple negative sample components are randomly selected using the entity set, and the sample distance between each negative sample and the corresponding positive sample component is calculated. The sample distances are sorted, and the negative sample component corresponding to the minimum sample distance is selected. The negative sample component replaces the replacement component in the positive sample to construct a negative sample.

5. The knowledge graph embedding method based on deep learning network training according to claim 1, characterized in that: The S4 comprises the following steps: S401, constructing a high-dimensional matrix using positive samples and corresponding negative samples, and inputting the high-dimensional matrix into a deep learning network as an input signal for training; S402, using a convolution layer and a pooling layer to process the input signal to obtain a low-level feature signal; S403, inputting the low-level feature signal into the classification sub-network and the scoring sub-network; S404, using the flattening layer in the classification subnetwork to flatten the low-level feature signal into a one-dimensional signal, after being connected through the fully connected layer, the classification prediction probability is calculated using a normalized exponential function; S405. Use the convolution module in the scoring subnetwork to extract high-level features of the low-level feature signal, use the maximum pooling layer to reduce the size of the high-level features, and calculate the sample score output by the scoring function.

6. The knowledge graph embedding method based on deep learning network training according to claim 1 is characterized in that: The S5 comprises the following steps: S501, calculating a first loss function according to the classification prediction probability and the true label, calculating a second loss function according to the sample score, and obtaining a total loss function by using the first loss function and the second loss function; S502, according to the second loss function, obtain the gradient matrix expression of the beater sub-network, and according to the output layer gradient matrix of the beater sub-network, use the error backpropagation gradient descent method to calculate the gradient of each layer except the output layer in turn, and calculate the gradient of each layer of the beater sub-network for the second loss function; S503, according to the first loss function, obtain the gradient matrix expression of the classification sub-network, and according to the output layer gradient matrix of the classification sub-network, use the error backpropagation gradient descent method to calculate the gradient of each layer except the output layer in turn, and calculate the gradient of each layer of the classification sub-network for the first loss function; S504, according to the gradient of the beating sub-network and the gradient of the classification sub-network, calculate the gradient of the junction of the beating sub-network and the classification sub-network, and calculate the gradient of the lower-layer network; S505. Use the gradient of each layer of the classification subnetwork, the gradient of each layer of the scoring subnetwork, the gradient at the junction of the scoring subnetwork and the classification subnetwork, and the gradient of the lower-layer network to update the coefficients and input signal of the deep learning network, and normalize the updated input signal vector.

7. The knowledge graph embedding method based on deep learning network training according to claim 6 is characterized in that: The expression of the total loss function is as follows: in, represents the total loss function, represents the first loss function, represents the second loss function, represents the regularization coefficient, Represents the model coefficient The second-order norm of represents the total number of positive samples, represents the true label, Indicates j The classification prediction probability, represents the judgment coefficient, Indicates that the current input sample is a positive sample, Indicates that the current input sample is a negative sample. Indicates j Sample scores.

8. The knowledge graph embedding method based on deep learning network training according to claim 7 is characterized in that: The expression of the gradient matrix of the molecular network is as follows: in, represents the gradient matrix of the molecular network, Indicates j Positive sample gradients, Indicates j negative sample gradients, Indicates j The sample fraction of positive samples, Indicates j The sample fraction of negative samples.

9. The knowledge graph embedding method based on deep learning network training according to claim 8, characterized in that: The expression of the gradient matrix of the classification subnetwork is as follows: in, Represents the classification subnetwork k The gradient matrix, , Indicates the second i The classification prediction probability of positive samples is Indicates the second i +1 classification prediction probability of negative samples, Indicates indivual, .

Images