A logic polymorphism-oriented principle graph representation learning method and system

Abstract

The application provides a logic polymorphism-oriented matter logic graph representation learning method and system, the matter logic graph representation learning method comprises the following steps: vectorizing nodes in a matter logic graph to obtain a preliminary vector representation; a composite event composed of events is constructed by using a tree structure; the logic relationship between events is considered, a periodic function is used as an event confidence mapping function, and the matter logic of the composite event is modeled; a negative sampling strategy is selected to perform negative sampling on the composite event; and marginloss is used as a loss function to train event embedding. By using the tree structure to model event combination and using the periodic function as the event score, the model can ensure the expression ability for complex events and ensure the embedding quality of each event, so as to be suitable for downstream tasks in different scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph, and particularly relates to a logic polymorphism-oriented fact graph representation learning method and system. BACKGROUND

[0002] The representation learning of a knowledge graph is a technical means for embedding a group of entities and the relationships between the entities into a low-dimensional dense vector space, so that a machine can use the same as a basis for rapid reasoning or prediction. The representation learning of a knowledge graph has been widely applied in downstream tasks such as reading comprehension, recommendation systems and QA.

[0003] However, due to the lack of event granularity representation in the traditional knowledge graph, it is difficult to predict and reason events, so the fact graph is used to model the fact taking event as the basic unit to better adapt to this scenario. Since there are a large number of logic between events, such as causal relationship, coexistence relationship and contradiction relationship, events are connected and combined through these logic. Under different prior knowledge, the state of the event will be transmitted through the logic combination chain, and the constructed fact graph will also have different states, which is called the polymorphism of the fact graph.

[0004] The polymorphic fact graph will produce a problem in the embedding process, that is, in the training model taking a non-convex function as a score function, the position of the event embedding in the vector space will change constantly under the constraint of multiple logic rules during the training process. Since there are many logical constraints between some events and other events, and there are many combination ways of these logical rules, this will cause the event embedding change often cannot take into account all constraints, and finally it is difficult to obtain good model training effect. SUMMARY

[0005] In order to overcome the above-mentioned deficiencies of the prior art, the present application provides a logic polymorphism-oriented fact graph representation learning method and system, which models the event combination by using a tree structure and uses a periodic function as the event score, which not only ensures the expression ability of the training model for complex events, but also ensures the embedding quality of each event, so as to be applicable to downstream tasks in different scenarios.

[0006] To solve the above problems, the first object of the present application is to provide a logic polymorphism-oriented fact graph representation learning method, which comprises:

[0007] S 100 : vectorizing the nodes in the fact graph to obtain a preliminary vector representation;

[0008] S 200: Construct a complex event composed of events in a tree structure;

[0009] S 300 : Considering the logical relationship between events, using a periodic function as an event confidence mapping function, modeling the logic of the complex event;

[0010] S 400 : Select a negative sampling strategy to negatively sample the complex event;

[0011] S 500 : Use margin loss as a loss function to train event embedding.

[0012] Optionally, in step S 100 , the vectorization of the nodes in the logic graph comprises:

[0013] S 110 : Obtain an embedding representation of a triple;

[0014] The embedding representation of the triple comprises: selecting a TransE model to perform representation learning on the logic graph to obtain a trained triple embedding model;

[0015] The triple embedding model comprises a head entity embedding, a relation embedding, and a tail entity embedding, and the distance formula of the triple embedding model is represented as:

[0016]

[0017] Wherein: (head, rel, tail) represents a triple, head represents an entity embedding, rel represents an entity embedding, and tail represents an entity embedding;

[0018] S 120 : Obtain an embedding of a basic event;

[0019] The embedding formula of the basic event is as follows: Δ=f(h,r,t)

[0020] Wherein: △ represents event embedding, h, r, t represent the embedding representation of the head entity, the relation, and the tail entity in the triple respectively, and f() represents a fusion operation.

[0021] Optionally, in step S 110 , the embedding method for obtaining the embedding representation of the triple comprises a method for fusing factual information and a method for fusing additional information.

[0022] Optionally, in step S 110 , the loss function of the embedding representation of the triple is:

[0023]

[0024] wherein: E represents a set of entities in a fact graph, R represents a set of relations, and (h', r, t') represents a triple after random negative sampling.

[0025] Optionally, in step S 200 , the embedding representation of the composite event is:

[0026] Δ1 = f(h1, r1, t1)

[0027] Δ2 = f(h2, r2, t2)

[0028] Δ 12 = g(Δ1, Δ2)

[0029] wherein: Δ1 and Δ2 represent embeddings of two events respectively, f() represents a fusion operation, Δ 12 represents a composite event composed of Δ1 and Δ2, and g() represents an average operation.

[0030] Optionally, in step S 400 , the negative sampling of the composite event according to the selected negative sampling strategy specifically includes:

[0031] S 410 : considering whether the event is a composite event during negative sampling;

[0032] S 420 : defining an event composed of a single triple as a first-order event, and so on, and representing as follows:

[0033] Δ1 = f(h1, r1, t1),

[0034] Δ 12 = g(Δ1, Δ2),

[0035] Δ 123 = g(g(Δ1, Δ2), Δ3),

[0036] Δ 1234 = g(g(Δ1, Δ2), g(Δ3, Δ4))

[0037] wherein: Δ1 is a first-order event, Δ 12 is a second-order event, Δ 123 and Δ 1234 are third-order events.

[0038] S 430 : dynamically sampling the event pair.

[0039] Optionally, in step S 500In the specific embodiment, the calculation expression of the loss function margin loss is as follows:

[0040]

[0041] Wherein, (Delta1, Delta2') represents the triplets after negative sampling, and gamma is a separation coefficient.

[0042] A second object of the present application is to provide a logic polymorphism-oriented fact graph representation learning system, comprising:

[0043] An event node vectorization module for vectorizing nodes in the fact graph;

[0044] A composite event representation module for representing composite events in a tree structure;

[0045] A fact logic modeler for analyzing the logical relationship between events and using a periodic function as an event pair confidence mapping function to fit the event pair to a suitable peak;

[0046] A negative sampling module for randomly replacing events in the event pair and online dynamically sampling the event pair;

[0047] An event embedding module for training event embedding.

[0048] A third object of the present application is to provide an electronic device comprising a memory and a processor, the memory being configured to store a program supporting the processor to execute the method according to any one of claims 1 to 7, and the processor being configured to execute the program stored in the memory.

[0049] A fourth object of the present application is to provide a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to implement the logic polymorphism-oriented fact graph representation learning method as described above.

[0050] Compared with the prior art, the present application has significant advantages and beneficial effects, which are embodied in the following aspects:

[0051] The present application uses a tree structure to model event combinations and uses a periodic function as a score function to set a suitable value for each event pair to ensure that under the pulling action of various logical rules, all event embeddings can also be placed in a suitable position in the vector space, laying a foundation for various downstream tasks. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 A flowchart of the logic polymorphism-oriented fact graph representation learning method in the embodiments of the present application;

[0053] Figure 2 The overall flowchart schematic diagram of the logic polymorphism-oriented case graph representation learning system in the embodiment of the present application is shown in the figure;

[0054] Figure 3 The representation diagram of the composite event tree structure in the embodiment of the present application is shown in the figure;

[0055] Figure 4 The sigmoid function curve in the embodiment of the present application is shown in the figure;

[0056] Figure 5 The sin2(x) function curve in the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0057] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0058] At present, the construction and representation learning technology of the case graph are relatively single in the modeling mode of events, such as event pair or event chain, which will lead to insufficient performance of the trained model on the scene or condition. In addition, for the whole case graph, different prior knowledge will lead to different states of the graph, and the existing scheme cannot represent such a case graph with polymorphism, and the main reasons are as follows:

[0059] Firstly, the traditional case graph does not consider the influence of the continuous propagation of prior knowledge through case logic on other events;

[0060] Secondly, the existing knowledge graph representation learning model generally adopts non-convex function optimization, and adopts sigmoid function as the cost function, which has obvious advantages, that is, the model can converge well, and there will be obvious separation between positive and negative examples.

[0061] However, in the case graph with polymorphism, complex events often contain multiple case logics, and involve many basic events, and these basic events will also participate in the composition of many complex events, if the non-convex function is used as the scoring function, the embedding of these events is difficult to meet all the case constraints.

[0062] To solve the above technical problems, please refer to Figure 1 In the embodiment of the present application, a logic polymorphism-oriented case graph representation learning method is provided, and the method comprises the following steps:

[0063] S 100 : vectorizing the nodes in the case graph to obtain a preliminary vector representation;

[0064] In a traditional knowledge graph, nodes represent concepts or entities, and the connections between nodes represent the relationships between concepts or entities. It is mainly applied to depict, store general or specific domain knowledge system, and used for knowledge query. The application of knowledge graph is relatively limited for expressing entities with time connection.

[0065] With the development of society, in the matter graph, nodes represent events, and the connections between nodes represent the relationships between events, such as causal relationship, connection relationship, etc. Such matter graph is mainly applied in finance and other industries for analyzing the causes of financial events or the results caused.

[0066] Specifically, in step S 100 , the vectorization of the nodes in the matter graph specifically includes:

[0067] S 110 : obtaining a triple embedding representation;

[0068] It needs to be further explained that an event is composed of triples or other events, that is, a basic event needs to be composed of triples, and the matter graph is a high-order supergraph based on the knowledge graph. Therefore, to obtain the embedding representation of the event, the embedding representation of the triple needs to be obtained first.

[0069] Specifically, in step S 110 , the embedding method for obtaining the embedding representation of the triple includes a method of fusing factual information and a method of fusing additional information.

[0070] Of course, there are many methods for embedding triples, and the embedding obtained in this step is not the final event representation, and additional information such as text information and associated paths does not need to be used to accurately describe it.

[0071] Specifically, in step S 110 , the obtaining of the embedding representation of the triple specifically includes:

[0072] selecting a TransE model to perform representation learning on the matter graph to obtain a trained triple embedding model;

[0073] The triple embedding model includes a head entity embedding, a relationship embedding, and a tail entity embedding, and the distance formula of the triple embedding model is represented as:

[0074]

[0075] Wherein: (head, rel, tail) represents a triple, head represents an entity embedding, rel represents an entity embedding, and tail represents an entity embedding.

[0076] Specifically, in step S110 In particular embodiments, the loss function of the embedding representation of the triple is:

[0077]

[0078] Wherein: E represents the entity set in the matter graph, R represents the relationship set, (h', r, t') represents the triple after random negative sampling.

[0079] S 120 : obtaining the basic event embedding;

[0080] The so-called basic event is an event composed of only triples. The expression formula of the basic event is as follows:

[0081] Δ=f(h,r,t)

[0082] Wherein: △ represents the event embedding, h, r, t represent the embedding representation of the head entity, the relationship and the tail entity in the triple respectively, and f() represents the fusion operation.

[0083] The fusion method can be to take the average of h, r and t, or to concatenate h, r and t, etc., but in this embodiment, concatenation is used as the fusion method, because this method can retain more features of the triple.

[0084] If the embeddings of the head entity, the tail entity and the relationship are all 50-dimensional, the dimension of the event representation △ after using the concatenation operation is 150-dimensional.

[0085] S 200 : constructing a composite event composed of events using a tree structure.

[0086] Unlike general matter graphs, in the embodiment of the application, events represented by single triples are not limited, and composite events composed of events are additionally considered, which is more in line with actual life and more in line with the characteristics of polymorphic matter graphs.

[0087] For example, in the online shopping scenario, the event pair that the user verifies the payment password and the balance in the user account is greater than the commodity amount can be described as a composite event, that is, the user satisfies the payment condition.

[0088] However, the user satisfying the payment condition does not mean that the commodity can be successfully purchased, and from the merchant side, it is also necessary to judge whether the commodity inventory is sufficient and whether the sale time has passed, and this event pair can be regarded as a composite event that the merchant side satisfies the sale condition. These two composite events can form a new event, that is, the user can successfully place an order to purchase the commodity. In this scenario, each event needs to be represented.

[0089] In order to adapt to this scenario, the embodiment of the present application adopts a tree structure to represent a complex event. That is, each composite event has a binary tree structure, and the leaf node of the tree is another event.

[0090] In step S 200 , from the embedding perspective, the embedding of the composite event is represented as the average of the embeddings of the two events, which is as follows:

[0091] Δ1 = f(h1, r1, t1)

[0092] Δ2 = f(h2, r2, t2)

[0093] Δ 12 = g(Δ1, Δ2)

[0094] where Δ1 and Δ2 represent the embeddings of the two events, respectively, f() represents a fusion operation, Δ 12 represents a composite event composed of Δ1 and Δ2, and g() represents an average operation.

[0095] Thus, the dimensions of all event embeddings are consistent, so that both basic events and composite events can be constrained to each other to describe more complex scenarios. In addition, the composite event is linked to the basic events that constitute it, and some characteristics of the basic events are retained.

[0096] S 300 : considering the logical relationship between events, using a periodic function as the confidence mapping function of the event pair, modeling the rational logic of the composite event;

[0097] The rational logic refers to the logical relationship between events, such as causality, contradiction, etc. Considering that some events themselves represent negation, and the polymorphism of the rational graph is propagated through the rational logic, the contradiction between events is mainly considered here.

[0098] In real life, events are mostly composite events, that is, they are composed of many basic events. This will cause the embedding of the basic event to change with the constraints of different composite events. After satisfying the constraints of the composite event Δ 12 , it may be difficult to satisfy the constraints of Δ 13 , and simultaneously satisfying many composite events may cause the embedding of the basic event to be constantly pulled in the vector space by the rational constraints.

[0099] Please refer to Figure 4 , when sigmoid is used as the score function, that is:

[0100] S 12 = sigmoid(g(Δ1, Δ2)

[0101]

[0102] where g() represents a dot product operation, S 12 represents the confidence that Δ1 and Δ2 have a contradiction relationship, and the higher the score is, the more likely it is that the two events have a contradiction relationship.

[0103] But sigmoid is a monotonic function, if the event embedding is affected by many event constraints and is pulled, it will be difficult to find the most suitable position, affecting the quality of representation learning.

[0104] Therefore, the embodiment of the present application proposes to use a periodic function as the event pair confidence mapping function.

[0105] Please refer to Figure 5 , taking the sine function as an example, that is:

[0106] S 12 =sin(g(Δ1,Δ2))

[0107] If the two events have a contradiction relationship, the score will tend to 1, which is reflected on the sine function curve as more tending to the wave crest, and vice versa.

[0108] At this time, in an ideal case, events with a contradiction relationship can be represented as:

[0109] 1-sin 2 (g(Δ1,Δ2))≤δ

[0110] δ is a number that tends to 0 infinitely. Using sin 2 as the score function can increase the range of event embedding adjustment in the same interval, and more wave crests can be fitted.

[0111] The above constraint can also be converted to:

[0112]

[0113] where k 12 is an integer, and each event pair will correspond to a different k, the purpose of which is to fit the event pair to the appropriate wave crest and expect it to satisfy as many constraints as possible.

[0114] S 400 : Selecting a negative sampling strategy to negatively sample the composite event.

[0115] Specifically, in step S 400 , the negative sampling of the composite event by the negative sampling strategy specifically includes:

[0116] S410 : Consider whether the event is a composite event when negative sampling;

[0117] Since the event pair can be composed of a basic event consisting of a single triple, it can also be composed of a composite event. Therefore, when negative sampling, whether the event is a composite event needs to be considered.

[0118] S 420 : Define the event composed of a single triple as a first-order event, and so on, and express as follows:

[0119] Δ1=f(h1,r1,t1),

[0120] Δ 12 =g(Δ1,Δ2),

[0121] Δ 123 =g(g(Δ1,Δ2),Δ3),

[0122] Δ 1234 =g(g(Δ1,Δ2),g(Δ3,Δ4))

[0123] Where: Δ1 is a first-order event, Δ 12 is a second-order event, Δ 123 and Δ 1234 are third-order events;

[0124] The composite event is one order higher than the highest order event in the basic event that constitutes it. When negative sampling, the randomly replaced event should be the same order as the event that has not been replaced before, to ensure that the event pair after negative sampling is opposite to the previous event pair, does not destroy the original data distribution, and improves the embedding effect.

[0125] S 430 : Online dynamic sampling of event pairs.

[0126] Considering that this model is based on event pairs as constraint objects, negative sampling will also randomly replace events in event pairs.

[0127] This method uses online event sampling, that is, in the training of each batch, event pairs are dynamically sampled. This way can get better embedding performance.

[0128] S 500 : Use margin loss as loss function to train event embedding.

[0129] After modeling and negative sampling of composite events, the next step is to train event embedding.

[0130] This method uses margin loss as the loss function,

[0131]

[0132] wherein: (Δ1, Δ'2) is the triplet after negative sampling, and γ is a separation coefficient.

[0133] Referring to Figure 2 as shown, Figure 2 The logic polymorphism-oriented event graph representation learning system provided by the embodiment of the application comprises:

[0134] An event node vectorization module is configured to vectorize nodes in the event graph.

[0135] A composite event representation module is configured to represent a composite event in a tree structure.

[0136] An event logic modeler is configured to analyze logical relationships between events and use a periodic function as an event pair confidence mapping function to fit the event pair to a suitable wave crest.

[0137] A negative sampling module is configured to randomly replace events in the event pair and perform online dynamic sampling on the event pair.

[0138] An event embedding module is configured to train event embedding.

[0139] The embodiment of the application further provides an electronic device, comprising a processor, a display, a memory, and computer program instructions stored in the memory and executable on the processor, and the processor is configured to implement the logic polymorphism-oriented event graph representation learning method described above.

[0140] The electronic device provided by the embodiment of the application can be used to execute the logic polymorphism-oriented event graph representation learning method provided by any one of the method embodiments, and the implementation principle and technical effects are similar, and thus will not be described herein.

[0141] The embodiment of the application further provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions, and when the computer execution instructions are executed on a computer, the computer is caused to execute the logic polymorphism-oriented event graph representation learning method described above.

[0142] Note that the computer-readable storage medium described above can be realized by any type of volatile or nonvolatile storage devices or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic storage, a flash memory, a magnetic disk, or an optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0143] Optionally, the readable storage medium is coupled to the processor so that the processor can read information from the readable storage medium and the readable storage medium can write information to the processor. Of course, the readable storage medium can also be a part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit. Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0144] Although the present disclosure has been disclosed as above, the scope of protection of the present disclosure is not limited to this. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications shall fall within the scope of the present disclosure.

Claims

1. A method for learning the representation of a logical polymorphism-oriented event graph, characterized in that, The aforementioned event graph representation learning method includes: S 100 Vectorize the nodes in the event graph to obtain a preliminary vector representation; S 200 A tree structure is used to construct composite events composed of events. Each composite event has a binary tree structure, and the leaf nodes of the tree are other events. An event pair consists of two basic events or composite events that have a logical relationship. For example, the event pair of a user verifying a payment password and the user's account balance being greater than the price of the goods is described as a composite event, meaning that the user meets the payment conditions. The embedded representation of the composite event is as follows: in and Let f represent the embedding of the two events respectively, and let f() represent the fusion operation. Indicates by and The composite event consists of g(), which represents the mean operation. S 300 Considering the logical relationships between events, a periodic function is used as the confidence mapping function for event pairs to model the logical reasoning of the composite events; S 400 : Select a negative sampling strategy to negative sample the composite event; The negative sampling strategy is as follows: the composite event is one order higher than the highest-order event among the basic events that constitute it. During negative sampling, the randomly replaced event must be at the same order as the previously unreplaced event to ensure that the event pair after negative sampling is opposed to the previous event pair. S 500 Use margin loss as the loss function to train event embeddings.

2. The event graph representation learning method oriented towards logical polymorphism according to claim 1, characterized in that, In step S 100 In this context, the vectorization of nodes in the event graph specifically includes: S 110 : Obtain the embedding representation of the triple; The specific steps of obtaining the triplet embedding representation include: selecting the TransE model to perform representation learning on the event graph to obtain the trained triplet embedding model; The triplet embedding model includes head entity embedding, relation embedding, and tail entity embedding. The distance formula for the triplet embedding model is expressed as: in: To represent a triple, Represents entity embedding, Represents entity embedding, Represents entity embedding; S 120 : Get the embedding of the basic event; The embedding formula for the basic event is as follows: Where: △ represents event embedding, h, r, and t represent the embedding representations of the head entity, relation, and tail entity in the triple, respectively, and f() represents the fusion operation.

3. The event graph representation learning method oriented towards logical polymorphism according to claim 2, characterized in that, In step S 110 In the above, the embedding method for obtaining the embedding representation of triples includes a method for fusing factual information and a method for fusing additional information.

4. The event graph representation learning method oriented towards logical polymorphism according to claim 2, characterized in that, In step S 110 In this context, the loss function for the embedding representation of the triples is: Where: E represents the set of entities in the logic graph, and R represents the set of relations. This represents a triple after random negative sampling.

5. The event graph representation learning method oriented towards logical polymorphism according to claim 1, characterized in that, In step S 400 In this context, the specific steps of selecting a negative sampling strategy to negatively sample the composite event include: S 410 When negative sampling, consider whether the event is a composite event; S 420 An event consisting of a single triple is defined as a first-order event, and so on, and is represented as follows: , , , in: It is a first-order event. It is a second-order event. and All are third-order events; S 430 : Dynamically sample event pairs.

6. The event graph representation learning method oriented towards logical polymorphism according to claim 1, characterized in that, In step S 500 In this context, the calculation expression for the loss function, margin loss, is as follows: in: This represents the triple after negative sampling. It is the separating coefficient.

7. A logic-based graph representation learning system oriented towards logical polymorphism, characterized in that, include: The event node vectorization module is used to vectorize nodes in the event graph; The composite event representation module is used to represent composite events using a tree structure. Each composite event has a binary tree structure, and the leaf nodes of the tree are other events. An event pair consists of two basic events or composite events that have a logical relationship. For example, the event pair of a user verifying a payment password and the user's account balance being greater than the price of the goods is described as a composite event, meaning that the user meets the payment conditions. The embedded representation of the composite event is as follows: in and Let f represent the embedding of the two events respectively, and let f() represent the fusion operation. Indicates by and The composite event consists of g(), which represents the mean operation. The event logic modeler is used to analyze the logical relationships between events and uses a periodic function as the confidence mapping function for event pairs to fit the event pairs to a suitable peak. The negative sampling module is used to randomly replace events in an event pair and perform online dynamic sampling of the event pair; The negative sampling strategy is as follows: the composite event is one order higher than the highest-order event among the basic events that constitute it. During negative sampling, the randomly replaced event must be at the same order as the previously unreplaced event to ensure that the event pair after negative sampling is opposed to the previous event pair. The event embedding module is used to train event embeddings.

8. An electronic device, characterized in that, The electronic device includes a memory and a processor; The memory is used to store a program that supports the processor in executing the learning method according to any one of claims 1 to 6, and the processor is configured to execute the program stored in the memory.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the logic-polymorphism-oriented event graph representation learning method as described in any one of claims 1 to 6.

Images