Knowledge graph link reasoning method

Abstract

The invention provides a knowledge graph link reasoning method, and belongs to the field of artificial intelligence and knowledge graphs. The method comprises the steps that a local structure is reserved through a social triple integrity hypergraph, global association is captured through a social relation semantic hypergraph, and complementary social network double hypergraph representation is constructed; based on composite feature fusion of heterogeneous interaction, homogeneous parallel interaction and overall interaction, and in combination with a hypergraph attention network, relationship-guided dynamic semantic propagation is realized; designing multi-channel comparative learning based on social data; the social triple score is optimized based on a marginal sorting loss function, and a knowledge graph link prediction task and a self-supervision comparison task are fused through a joint learning framework. The problems of insufficient local structure modeling, global semantic information splitting, weak representation generalization ability, high dependence on labeled data and the like existing in knowledge graph link prediction in an existing social network are solved, and link reasoning tasks in a dynamic, sparse and high-noise-disturbance social scene are difficult to effectively deal with.

Description

Technical Field

[0001] The present invention belongs to the fields of artificial intelligence and knowledge graphs, and particularly relates to a knowledge graph link reasoning method. Background Art

[0002] As a core infrastructure in the field of artificial intelligence, a knowledge graph efficiently expresses domain knowledge through a structured network of entities and relationships, and is widely applied in scenarios such as social networking, intelligent question answering, recommendation systems, semantic search, etc. One of its core tasks is link prediction, that is, inferring missing triples (head entity, relation, tail entity) to complete the integrity of the knowledge graph. Traditional methods mainly rely on graph neural networks (GNNs), which learn entity representations by aggregating neighbor information, but face significant challenges in practical applications.

[0003] Although GNN methods have made some progress in local structure modeling, their limitations are gradually emerging. First, the low connectivity of the knowledge graph makes it difficult to capture semantic associations between distant entities. For example, in the triple "writer - work - publisher", if there is no intermediate relationship connection between entities, the model cannot effectively infer high - order semantic associations, resulting in low prediction accuracy for long - tail entities. Second, the existing methods have insufficient expressive ability for local information. Most models use linear aggregation strategies to fuse neighbor features, making it difficult to distinguish the semantic differences of complex relationships (such as "father - son" and "teacher - student"). In addition, the semantic dynamic changes of entities under different relationships are not fully considered, further limiting the generalization ability of the model.

[0004] In recent years, hypergraph modeling techniques have been introduced to enhance the representation learning of knowledge graphs. By modeling triples and their associated entity sets as hyperedges, hypergraph methods can break through the binary connection limitation of traditional graph structures and capture complex interactions between multiple nodes. For example, some studies use hypergraph neural networks (HGNNs) to aggregate the features of nodes within hyperedges to improve the local information fusion effect. However, existing hypergraph methods still have deficiencies: First, they do not explicitly distinguish different feature types such as heterogeneous interactions and homogeneous interactions between entities and relationships within triples, resulting in insufficient modeling of interaction patterns; Second, they lack a dynamic attention mechanism for entity clustering under relationship constraints, and global semantic associations are not effectively utilized; Third, self - supervised strategies are not introduced during the training process, making it difficult to distinguish noise interference from real semantic associations, which limits the discriminability of the representation.

[0005] In a social platform, the behavioral interactions between users (such as following, commenting, liking, forwarding, etc.) constitute a dynamically evolving social graph. This graph can be regarded as a heterogeneous knowledge graph, where nodes represent users, content, or groups, and edges represent various types of behavioral relationships. However, actual social data has problems such as strong sparsity, large noise perturbations, and rapid dynamic changes. Traditional graph representation and recommendation algorithms are difficult to accurately capture multi - dimensional semantics, and there are the following technical problems: Insufficient dynamic interaction modeling: Existing models lack the ability to model the evolution of user interests over time and are difficult to effectively capture time-sensitive interaction behaviors.

[0006] Severe cold start problem for long-tail users: Newly registered or low-frequency active users lack sufficient behavior records, making it difficult for recommendation systems to accurately predict their interest preferences.

[0007] Noise interference affects the recommendation effect: There are a large number of non-genuine interactions (such as false clicks and abnormal operations) in social platforms, resulting in learning biases in the model.

[0008] Low discrimination degree of multi-dimensional relationships: There are multiple types of complex interactions between users and content, and traditional single-channel graph structures are difficult to express their semantic differences. Summary of the Invention

[0009] In view of the above deficiencies in the prior art, a knowledge graph link reasoning method provided by the present invention solves the problems of insufficient local feature modeling, global semantic fragmentation, and high data dependence in existing social network knowledge graph link reasoning methods.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is: A knowledge graph link reasoning method, including the following steps: S1. Retain local structures through a social triple integrity hypergraph and capture global semantic associations using a social relationship semantic hypergraph to construct a complementary dual hypergraph representation of the social network; S2. Based on the dual hypergraph representation of the social network, combine hypergraph attention networks to achieve social relationship-guided dynamic semantic propagation for each social node by fusing composite semantic representations of heterogeneous interactions, homogeneous co-mutual interactions, and overall interactions, where the social triple integrity hypergraph is used to capture local interaction features, including heterogeneous interactions, homogeneous co-mutual interactions, and overall interactions, and the social relationship semantic hypergraph is used to perceive global semantic differences; S3. Design multi-channel contrast learning based on structure-aware and context-dependent representations; S4. Use the designed multi-channel contrast learning to construct multi-channel auxiliary tasks, and optimize the ranking result of social triple scores based on a margin-based ranking loss function. Complete the reasoning of knowledge graph links through a joint learning framework that fuses the knowledge graph link prediction task and the self-supervised contrast task, where the self-supervised contrast task is the multi-channel auxiliary task.

[0011] The beneficial effects of the present invention are as follows: Through the collaborative optimization of social triple integrity modeling and global semantic association, the present invention solves the deficiencies of traditional link inference methods in local feature expression and global information fusion, and significantly improves the accuracy and generalization ability of knowledge graph link prediction for social networks. The modular design of the present invention supports flexible expansion, can adapt to the complex inference requirements of industrial-level knowledge graphs, and provides efficient technical support for scenarios such as intelligent question answering and recommendation systems.

[0012] Further, S1 includes the following steps: S101. Construct a triple hyperedge set based on the social triples of the knowledge graph, connect the social head entity, social relationship, and social tail entity to the same hyperedge together, and retain the integrity of the social triple, so as to retain local relationships through the social triple integrity hypergraph; S102. Construct a relationship hyperedge set based on the semantics of social relationships, cluster the social head entities and social tail entities with the same social relationship respectively to form global semantic associations, and construct a complementary dual hypergraph representation of the social network.

[0013] The beneficial effects of the above further solution are as follows: The dual hypergraph of the social network forms a positive interaction and complementarity - the social triple integrity hypergraph prevents the loss of information within the social triple, and the social semantic hypergraph breaks through the limitation of a single social triple. The cooperation of the two enables the model to simultaneously possess the capabilities of fine-grained structure preservation and global semantic perception.

[0014] Still further, S2 includes the following steps: S201. By explicitly modeling the multi-type interaction features among the social head entity, social relationship, and social tail entity, construct a composite semantic representation including heterogeneous interaction, homogeneous interaction, and overall interaction. Among them, the local interaction features include heterogeneous interaction, homogeneous co-interaction, and overall interaction, and the social triple integrity hypergraph is used to capture the local interaction features; S202. Fuse the composite semantic representation through a weight matrix to generate a hyperedge semantic vector. Among them, in the social node representation update stage, the gated aggregation mechanism is used to realize the directional propagation of hyperedge information to the target social node; S203. Use the social relationship semantic hypergraph to perceive the global semantic differences, and based on the directional propagation, introduce a dynamic attention mechanism, combine with the hypergraph attention network, and realize the relationship-guided dynamic semantic propagation to generate a structure-aware and context-dependent representation for each social node.

[0015] The beneficial effects of the above further solution are as follows: triple breakthroughs are achieved through multi-level interaction modeling, and the influence intensity between different elements is explicitly quantified through differentiable attention weights; the relationship guidance mechanism enables the same social entity to obtain different representations in different contexts; the adaptive aggregation characteristics of the graph attention network significantly enhance the modeling ability for long-tail relationships.

[0016] Furthermore, the expression of the heterogeneous interaction is as follows: ; where represents the heterogeneous semantic vector, represents the feature vector of the user node, represents the embedded representation of the interaction behavior between the user and the content, represents the feature vector of the content node, + represents the addition of feature vectors, represents the element-wise multiplication, represents the concatenation operation; The expression of the homogeneous interaction is as follows: ; where represents the feature similarity between homogeneous entities, represents the embedded representation of user A, represents the embedded representation of user B; The expression of the overall interaction is as follows: ; where represents the embedded representation that fuses the user, content, and timestamp, represents the embedded time information.

[0017] Furthermore, the S203 includes the following steps: Based on directional propagation, through the dynamic weight allocation mechanism, the contribution degrees of different social entities to the hyperedge semantic representation are distinguished. Among them, the expression of the attention coefficient is as follows: ; where represents the attention coefficient of the social node in the hyperedge , p represents the attention vector, represents the learnable attention projection vector, represents the activation function, represents the clustering center feature representation of the hyperedge , represents the feature representation of the social node, represents the hyperedge All social nodes in , represents a hyperedge and is the feature representation of a certain social node in Based on the differentiated contribution degree, in the social node representation update stage, a hierarchical attention mechanism is introduced to fuse multi-hyperedge information to generate a structure-aware and context-dependent representation for each social node. The updated representation is as follows: ; ; Among them, represents the updated representation of the social node , represents the contribution size of the hyperedge to the social node , represents the set of all hyperedges connected to the social node , represents the aggregation function from the hyperedge to the social node, represents the representation of the hyperedge , represents the final updated representation of the social node , represents the semantic representation vector of the hyperedge , represents the transpose of q , q represents a trainable attention vector.

[0018] The beneficial effect of the above further solution is that by introducing a dynamic weight allocation mechanism based on directed propagation, it effectively differentiates the contribution degrees of different target entities in the image to the semantics of visual relationships, thereby enhancing the discriminability and semantic expression ability of hyperedge representations. This mechanism can not only suppress the interference of redundant or irrelevant objects to semantic aggregation and improve the model's ability to model complex multi-object relationships, but also enhance the semantic consistency and context adaptability in the node representation update process. At the same time, this method does not rely on artificial prior structures, has good generality and robustness, and is applicable to object detection tasks in various image scenarios such as multi-scale and irregular structures.

[0019] Furthermore, the S3 includes the following steps: Use a multi-channel dynamic weight mechanism to organically fuse heterogeneous features to generate multi-channel features; Based on the multi-channel features, for each channel, construct positive and negative sample pairs using structure-aware and context-dependent representations and social network dual hypergraph representations; Based on the constructed positive and negative sample pairs, an improved loss function is designed to optimize the representation space, and the design of multi-channel contrast learning based on social data is completed.

[0020] The beneficial effects of the above further scheme are as follows: By constraining the semantic consistency between the hyperedge and the social node representation through the multi-channel contrast learning framework, maximizing the mutual information of positive samples and suppressing the similarity of negative samples, the adaptability of the model to noise interference and sparse data is significantly enhanced. The self-supervised signal provides supplementary training objectives for low-resource scenarios, alleviating the model degradation problem caused by insufficient labeled data. The channel dynamic weight mechanism automatically identifies the key feature channels in different semantic scenarios. For example, for high-frequency relationships, it strengthens the contribution of the global semantic channel, and for low-frequency relationships, it enhances the weight of the local structure channel, realizing the adaptive fusion of heterogeneous features.

[0021] Furthermore, the expression of the multi-channel feature is as follows: ; ; Among them, represents the multi-channel feature, represents the channel c 's contribution weight to the final representation of the social entity v , represents the social entity v 's representation in the channel c , t represents the social triple channel, r represents the social relationship channel, n represents the neighbor channel, represents the channels other than the current channel c , represents the learnable attention vector, represents the learnable weight matrix, represents the social entity's representation in the channel .

[0022] Furthermore, the expression of the improved loss function is as follows: ; Among them, represents the improved loss function, v represents the social entity, V represents the set of all social entities v , e represents the hyperedge, represents the set of hyperedges connected to the social entity v , represents the activation function, represents the consistency discriminator, Representation of the channel c Lower hyperedge e representation of Representation of social entity v in the channel c representation of Representation of negative sample hyperedge

[0023] Furthermore, the S4 includes the following steps: Construct a multi-channel auxiliary task using the designed multi-channel contrast learning; Embed the structure-aware and context-dependent representations as input, and calculate the social triple scores for ranking through a scoring function; Use the margin-based ranking loss function for optimization with the social triple score ranking result as input; Based on the optimization result, integrate the knowledge graph link prediction task and the self-supervised task through a joint learning framework to complete the inference of the knowledge graph link, where the link prediction task is the main task and the self-supervised contrast task is the multi-channel auxiliary task.

[0024] The beneficial effects of the above further solution are as follows: The margin-based ranking loss function directly optimizes the performance of social triple score ranking, enforces the semantic rationality constraint on the score difference between positive and negative samples, and significantly improves the ranking accuracy of the prediction results. The joint learning framework integrates the link prediction task and the self-supervised contrast task, balances the training intensity of the main and auxiliary tasks through hyperparameters, suppresses the overfitting risk while accelerating the model convergence. The lightweight scoring function supports efficient distributed computing, meets the real-time inference requirements of industrial-level knowledge graphs, and significantly improves the deployment efficiency and response speed of large-scale graph completion tasks.

[0025] Furthermore, the expression of the loss function of the joint learning framework is as follows: ; ; where, represents the loss function of the joint learning framework, represents the hyperparameter for regulating the influence of the multi-channel auxiliary task, represents the self-supervised auxiliary task loss, represents the margin-based ranking loss function, represents the user entity, represents the positive sample content, T represents the training social triple set, represents the margin hyperparameter, represents the social triple scoring function, represents the negative sample. Description of the drawings

[0026] Figure 1 This is the flowchart of the method of the present invention. Detailed implementation manners

[0027] The following describes the detailed implementation manners of the present invention to facilitate those skilled in the art of the present technology to understand the present invention. However, it should be clear that the present invention is not limited to the scope of the detailed implementation manners. For those of ordinary skill in the art of the present technology, as long as various changes are within the spirit and scope of the present invention defined and determined by the appended claims, these changes are obvious, and all inventions made using the concept of the present invention are within the scope of protection.

[0028] Embodiment 1 As Figure 1 shown, the present invention provides a knowledge graph link reasoning method, and its implementation method is as follows: S1. Construct a complementary dual hypergraph representation of the social network by retaining the local structure through the social triple integrity hypergraph and capturing the global semantic association using the social relationship semantic hypergraph. The implementation method is as follows: S101. Construct a triple hyperedge set based on the social triples of the knowledge graph , connect the social head entity, social relationship, and social tail entity to the same hyperedge together to retain the integrity of the social triple, so as to retain the local relationship through the social triple integrity hypergraph; S102. Construct a relationship hyperedge set based on the social relationship semantics , cluster the social head entities and social tail entities with the same social relationship respectively to form a global semantic association, and construct a complementary dual hypergraph representation of the social network.

[0029] In this embodiment, the dual hypergraph representation of the social network is used as the structural input of the social graph and is called in both the subsequent dynamic semantic propagation of S2 and the multi-channel contrast learning of S3. Among them: in S2, the triple integrity hypergraph is used to capture local interaction features (heterogeneous interaction, homogeneous and mutual, and overall interaction); the social relationship semantic hypergraph is used to perceive the global semantic difference; in S3, the representations generated by different hypergraphs are used as multi-channel inputs respectively to construct positive and negative samples for contrast learning, enhancing the discriminability of the node representations.

[0030] In this embodiment, the construction of the social triple integrity hypergraph: based on user behavior triples (such as <User A, like, Post B>, <User C, join, Group G>), construct a hyperedge structure. Each hyperedge connects the elements of a social triple of a behavioral action: social head entity (user), social relationship (behavior type), and social tail entity (content or group), so as to retain the integrity of the local interaction structure. Perform deduplication processing on the historical social behavior data to construct a sparse but effective hyperedge set.

[0031] Construction of Social Relationship Semantic Hypergraph: Based on the idea of semantic clustering, users or objects with the same social relationship are aggregated in terms of features. For example, for the "forwarding" behavior, users who frequently forward are aggregated to discover potential social circles; for the "joining group" behavior, users can be clustered according to the group theme to establish cross-community semantic associations and strengthen the expressive ability of the global structure.

[0032] In this embodiment, social triple hyperedges are constructed based on the knowledge graph. The social head entity, social relationship, and social tail entity are jointly connected to the same hyperedge to preserve the integrity of the social triple and ensure that local information is not lost. Specifically, it is as follows: First, traverse each social triple in the knowledge graph one by one ( h , r , t ), and use the social head entity h , social relationship r , and social tail entity t in the social triple as structural units to form a hyperedge. Then, all the hyperedges constructed from social triples are unified and summarized to form a triple hyperedge set . During the generation process, a duplicate detection mechanism needs to be introduced to filter out possible duplicate triples in the training set, so as to ensure the sparsity and structural effectiveness of the hyperedge set.

[0033] In this embodiment, relationship hyperedges are constructed based on social relationship semantics. The social head entities and social tail entities with the same social relationship are clustered respectively to form global semantic associations, enabling the model to capture long-distance semantic information and improve the reasoning ability. This process scans all triples in the knowledge graph that contain a specific social relationship r , and clustering sets are established for the corresponding social head entities and social tail entities respectively. When constructing the relationship hyperedge set , two subsets are formed for each type of social relationship: one is used to connect all social head entities centered on this social relationship, and the other connects social tail entities. This way can better reveal the potential semantic commonalities between social entities. At the same time, by integrating the social entity hyperedges of all social relationships, a social relationship semantic hypergraph is constructed, enabling the effective organization and modeling of semantic information at the global level.

[0034] S2. According to the dual hypergraph representation of the social network, based on the composite semantic representation fusion of heterogeneous interaction, homogeneous co-mutual, and overall interaction, combined with the hypergraph attention network, dynamic semantic propagation guided by social relationships is realized to generate a structure-aware and context-dependent representation for each social node. Among them, the social triple integrity hypergraph is used to capture local interaction features, and the local interaction features include heterogeneous interaction, homogeneous co-mutual, and overall interaction. The social relationship semantic hypergraph is used to perceive global semantic differences. The implementation method is as follows: S201. By explicitly modeling the multi-type interaction features among social head entities, social relationships, and social tail entities, construct a composite semantic representation that includes heterogeneous interactions, homogeneous interactions, and overall interactions. Among them, the local interaction features include heterogeneous interactions, homogeneous co-interactions, and overall interactions, and a social triple integrity hypergraph is used to capture the local interaction features; S202. Fuse the composite semantic representation through a weight matrix to generate a hyperedge semantic vector. Among them, in the social node representation update stage, the gated aggregation mechanism is used to achieve the directional propagation of hyperedge information to the target social node; S203. Utilize the social relationship semantic hypergraph to perceive the global semantic differences, and based on the directional propagation, introduce a dynamic attention mechanism, combined with the hypergraph attention network, to achieve relationship-guided dynamic semantic propagation, and generate a structure-aware and context-dependent representation for each social node (based on the directional propagation, introducing a dynamic attention mechanism, combined with the hypergraph attention network, its role is exactly to use the structure clustering in the social relationship semantic hypergraph to guide information propagation and complete global semantic perception). Specifically: Based on the directional propagation, through a dynamic weight allocation mechanism, distinguish the contribution degrees of different social entities to the hyperedge semantic representation; based on the distinguished contribution degrees, in the social node representation update stage, introduce a hierarchical attention mechanism to fuse multi-hyperedge information, and generate a structure-aware and context-dependent representation for each social node.

[0035] In this embodiment, the ultimate goal of the dynamic semantic propagation in S203 is to generate a structure-aware and context-dependent representation for each social node, and fusing multi-hyperedge information is the last stage to achieve this goal.

[0036] In this embodiment, the dynamic semantic propagation in S2 generates a structure-aware and context-dependent representation for each social entity, and this representation subsequently enters the multi-channel contrast learning in S3 as input features, which is used to construct positive and negative samples and optimize the representation space; when the S4 scoring function sorts the scores of social triples, this semantically enhanced representation is used as the embedding input.

[0037] In this embodiment, by explicitly modeling the multi-type interaction features among social head entities, social relationships, and social tail entities, construct a composite semantic representation that includes heterogeneous interactions, homogeneous interactions, and overall interactions. The heterogeneous interaction features are generated through the linear combination and non-linear mapping of entities and relationships.

[0038] Heterogeneous interaction modeling: Jointly model the user attribute features and behavior relationship features to generate heterogeneous semantic vectors: ; Among them, represents the heterogeneous semantic vector, represents the feature vector of the user node, An embedded representation indicating the interaction behavior between the user and the content Indicates the feature vector of the content node, + represents feature vector addition Indicates element-wise multiplication Indicates the concatenation operation

[0039] Homogeneous interaction modeling: Calculate the feature similarity between entities of the same type, such as potential friend identification ; Wherein Indicates the feature similarity between entities of the same type Indicates the embedded representation of user A Indicates the embedded representation of user B

[0040] Overall interaction modeling: Integrate user, content, and timestamp embeddings to achieve time-sensitive modeling ; Wherein Indicates the embedded representation integrating user, content, and timestamp Indicates the embedding of time information

[0041] The above features are fused by the weight matrix to form the hyperedge semantic vector ; Wherein Indicates the hyperedge e The embedded representation vector of Indicates the first heterogeneous interaction feature, modeling the element-wise interaction between "social head entity + social relationship" and "social tail entity - social relationship" Indicates the second heterogeneous interaction feature, modeling the element-wise joint feature cross of "social entity and social relationship" Indicates the homogeneous interaction feature, considering the similarity or correspondence between social head and tail target objects, and ignoring the relationship component Indicates the first overall interaction feature, directly summing up each part of the social triple to form an overall combined semantic vector Indicates the second overall interaction feature, retaining the complete information of each element in the social triple and jointly modeling W Indicates the trainable parameter matrix. In the social node representation update stage, the directional propagation of hyperedge information to the target node is achieved through the gated aggregation mechanism ; Wherein Indicates the social node v The final embedded representation of Indicates the activation function represents a learnable linear transformation matrix that converts the aggregated hyperedge information into a semantic representation in the social node space. represents a hyperedge aggregation function from hyperedges to social nodes represents the embedding representation of a hyperedge represents a hyperedge represents the set of all hyperedges connected to social nodes. This design significantly reduces the computational complexity while maintaining the strength of semantic association by restricting the interaction range to social nodes within shared hyperedges, and is particularly suitable for the distributed processing scenario of extremely large-scale social network knowledge graphs.

[0042] In this embodiment, a hypergraph attention network (HGAN) is used to implement semantic-aware feature propagation. This method differentiates the contribution degrees of different social entities to the hyperedge semantic representation through a dynamic weight allocation mechanism. First, the semantic clustering center feature is calculated based on the embedding vectors of social entities connected by hyperedges: ; where represents a hyperedge the clustering center feature representation of represents taking the average of multiple embedding vectors dimension by dimension to generate a central embedding represents the feature representation of a social node, representing the embedding of the current social node (such as a user, a post, etc.) represents a social node belongs to the hyperedge .

[0043] Since social entities near the clustering center often contain more representative semantic information, higher weights will be given to these entities when calculating the attention scores to ensure that core social entities receive greater attention during the feature aggregation process. Thus, the differences between different social entities in the hyperedge representation are effectively distinguished. The expression of the attention coefficient is as follows: ; where represents the attention coefficient of the social node in the hyperedge p represents the attention vector represents a learnable attention projection vector for learning the attention preference of nodes to the semantic clustering center represents an activation function used to enhance the non-linear expression ability before calculating the attention, and is often used to alleviate the problem of small gradient backpropagation represents the clustering center feature representation of the hyperedge represents the feature representation of a social node represents a hyperedge​​ All social nodes in, such as a certain user, , represents a hyperedge The feature representation of a certain social node in.

[0044] In this embodiment, the dynamic weight is used to explicitly model the contribution size of different social nodes to the hyperedge semantics when generating the hyperedge semantic representation, and as a conduction basis in the subsequent social node representation update stage, to transfer the hyperedge semantic information from the hyperedge to the social node, realizing semantic-driven embedding update.

[0045] In the social node representation update stage, a hierarchical attention mechanism is introduced to fuse multi-hyperedge information: ; ; Among them, represents the updated representation of the social node , represents the hyperedge 's contribution size to the social node , represents all hyperedge sets connected to the social node , represents the hyperedge 's aggregation function to the social node, represents the representation of the hyperedge , represents the social node 's final updated representation, represents the semantic representation vector of the hyperedge , represents q 's transpose, q represents a trainable attention vector.

[0046] In this embodiment, S2 realizes relationship-guided dynamic semantic propagation, including two sub-processes: First, inside each hyperedge, semantic weighting between social nodes and clustering centers is completed through the hypergraph attention mechanism (HGAN); then, at the social node level, semantic information from different hyperedges is fused by introducing a hierarchical attention mechanism, and finally, the social node representation with globally enhanced semantics is obtained.

[0047] S3. Based on the representation of structure awareness and context dependence, design multi-channel contrast learning based on social data, and its implementation method is as follows: Use the multi-channel dynamic weight mechanism to organically fuse heterogeneous features to generate multi-channel features; Based on multi-channel features, for each channel, positive and negative sample pairs are constructed by using structure-aware and context-dependent representations as well as social network dual hypergraph representations. Based on the constructed positive and negative sample pairs, an improved loss function is designed to optimize the representation space, and the design of multi-channel contrast learning based on social data is completed.

[0048] In this embodiment, the multi-channel contrast learning designed in S3 is used as an auxiliary task and forms a joint optimization objective with the main task (social triple score ranking) in S4. The InfoNCE loss function is used to strengthen the semantic consistency of social triples, thereby promoting the scoring function to learn more discriminative embeddings and suppressing the interference of noise samples on the scoring results.

[0049] In this embodiment, to enhance the generalization ability and robustness of the model, a multi-channel contrast learning mechanism is introduced.

[0050] In this embodiment, the organic integration of heterogeneous features is achieved through a multi-channel dynamic weight allocation mechanism. For social entities v , a set of channel weight coefficients is constructed, and its calculation process is defined as: ; The final comprehensive representation of social entities is generated by weighted fusion of multi-channel features: ; Among them, represents multi-channel features, represents channel c 's contribution weight to the final representation of social entity v , represents social entity v 's representation in channel c , t represents the social triple channel, r represents the social relationship channel, n represents the neighbor channel, represents the channels other than the current channel c and is used as other channel identifiers for normalization in the denominator, also taken from the set { t , r , n}, represents the learnable attention vector, represents the learnable weight matrix used to project the entity representations of different channels into a unified attention space, represents the representation of social entity in channel . The selective aggregation of social entity representations from different channels is achieved through the attention mechanism.

[0051] In this embodiment, to enhance the semantic consistency between hyperedges and social node representations, a multi-channel contrast learning framework is proposed. For each channel , positive and negative sample pairs are constructed. The positive sample pair consists of a social node v and its connected hyperedge . The positive sample is the user behavior interaction that actually occurs; the negative sample is randomly sampled from the interactions that never occur.

[0052] Optimize the representation space based on the improved InfoNCE loss function: ; Among them, represents the improved loss function, v represents a social entity, V represents the set of all social entities v , e represents a hyperedge, represents the set of hyperedges connected to the social entity v , represents an activation function, represents a consistency discriminator, represents a channel c the representation of the hyperedge e under, represents the representation of the social entity v in the channel c , represents the negative sample hyperedge representation, randomly sampled from the set of hyperedges irrelevant to the social entity v , representing an incorrect or irrelevant semantic combination, such as a behavior circle that has no direct relationship with the user / target.

[0053] In this embodiment, a discriminator D is designed to measure the consistency score between the hyperedge and the social node representation. The specific implementation is the dot product operation of two vectors. This loss function forces the hyperedge representation to accurately capture the common semantics of its connected nodes and reject the interference features of irrelevant nodes by maximizing the mutual information of positive sample pairs and suppressing the similarity of negative sample pairs.

[0054] S4. Use the designed multi-channel contrast learning to construct a multi-channel auxiliary task, and optimize the sorting result of the social triple score based on the margin-based ranking loss function. Complete the inference of the knowledge graph link through the joint learning framework that fuses the knowledge graph link prediction task and the self-supervised contrast task. Among them, the self-supervised contrast task is a multi-channel auxiliary task, and its implementation method is as follows: Use the designed multi-channel contrast learning to construct a multi-channel auxiliary task; Take the structure-aware and context-dependent representations as embedded inputs, and calculate the social triple score ranking through a scoring function; Taking the sorted result of social triple scores as input, it is optimized using a margin-based ranking loss function; Based on the optimization result, through a joint learning framework, the knowledge graph link prediction task and the self-supervised task are integrated to complete the reasoning of the knowledge graph links. Among them, the link prediction task is the main task, and the self-supervised contrast task is the multi-channel auxiliary task.

[0055] In this embodiment, the scoring function in S4 is responsible for calculating the matching degree of each social triple, and the ranking loss function takes the social triple score result as input for optimization, thereby reversely adjusting the learning direction of the embedding representation.

[0056] In this embodiment, the social network double hypergraph modeling in S1 provides the hypergraph structure and connection mode for S2 and S3; the feature modeling and dynamic semantic propagation in S2 generate structure-aware and context-dependent representations for S3 and S4, S3 constructs a multi-channel auxiliary supervision path for S4; S4 link prediction optimization integrates the main and auxiliary tasks and outputs the final reasoning result.

[0057] In this embodiment, given the embeddings of social entities and social relationships, the social triple scores are calculated through a scoring function to calculate the loss. The margin-based ranking loss function: ; Among them, the "+" sign at the end of the margin-based ranking loss function formula is the "positive value truncation" operator in the hinge loss function. If the prediction is correct (i.e., the positive sample score is much higher than the negative sample), then this term is 0 and no loss is generated; if the prediction is incorrect or the confidence is insufficient (i.e., the difference between the positive and negative sample scores is less than λ), then a positive loss is generated, where λ represents a margin threshold in the ranking loss lambda.

[0058] In this embodiment, the joint training framework: integrates the knowledge graph link prediction task and the self-supervised task, where the link prediction task is the main task and the self-supervised task is the auxiliary task. The final joint learning objective is defined as follows: ; Among them, represents the loss function of the joint learning framework, represents the hyperparameter used to regulate the influence of the multi-channel auxiliary task, represents the self-supervised auxiliary task loss, represents the margin-based ranking loss function, represents the user entity, represents the positive sample content, T represents the training social triple set, represents the margin hyperparameter, represents the social triple scoring function, Indicates negative samples, incorrect relationships, or incorrect entities.

[0059] In summary, the present invention is applied to social graph link prediction, and the effects include: significantly improved accuracy of user relationship prediction; significantly increased click-through rate of cold-start user recommendations; maintaining good robustness under noise interference; and showing excellent practical application effects in advertising recommendation and friend discovery tasks. By constructing a multi-perspective hypergraph structure, designing a feature enhancement network, and introducing a mutual information optimization strategy, the present invention significantly improves the prediction accuracy of missing entities and relationships in the knowledge graph, and can also be widely applied to fields such as intelligent search, semantic reasoning, and big data analysis.

[0060] Embodiment 2 To further verify the applicability of the present invention in cross-domain applications, the following provides a specific application scenario for the field of image processing, especially for object detection tasks, to illustrate the effectiveness and actual technical effects of the present invention in structural modeling and semantic reasoning.

[0061] In image object detection tasks, there are often semantic relationships between multiple target entities in an image (such as "person - carrying - backpack", "car - parked at - roadside"). However, existing detection methods mainly focus on bounding box prediction and target classification, lacking in-depth modeling of the semantic structure between targets, and there are the following technical problems: Weak multi-object relationship modeling ability: Traditional detection models ignore the context interaction between targets and only make independent judgments based on single targets, making it difficult to characterize semantic cooperation and behavioral dependencies; Fuzzy expression of similar targets: There is a lack of structured representation among multiple pedestrians in an image, and it is impossible to model group behavior patterns or location cooperation relationships; Lack of structure-aware training mechanism: Existing object detection loss functions mainly focus on classification and localization accuracy, lacking auxiliary supervision to guide the model to perceive the relationship structure between targets; Serious sparse annotation problem of semantic relationships in images: The interaction relationships between targets often lack clear annotations, and the self-supervised signals are missing, affecting the semantic understanding ability.

[0062] Therefore, this embodiment proposes an image object detection auxiliary reasoning method combining multi-channel hypergraph modeling and self-supervised optimization, aiming to introduce a structured modeling mechanism to enhance the semantic expression ability between targets and improve the understanding depth and robustness of the detection model. The method includes the following steps: S1: Construct a dual hypergraph structure for object detection, and its implementation method is as follows: S11: Triplet integrity hypergraph construction: Construct triplet hyperedges with visual triplets (e.g., (<pedestrian, holding, backpack>), (<vehicle, parked at, roadside>)) as units. Each hyperedge connects a subject object, a semantic relation, and an object object, which is used to model the local structure and retain the integrity of object interactions in the image.

[0063] S12: Relational semantic hypergraph construction: Based on semantic relation categories (such as "carrying", "riding", "placing", etc.), cluster target entities with the same relation to construct relational hyperedges, realizing global semantic perception and semantic consistency modeling.

[0064] S2: Dynamic interaction feature fusion and semantic propagation, and its implementation method is as follows: S21: Heterogeneous interaction modeling: Fuse the semantic information of the subject object and the semantic relation, and the object object and the semantic relation to generate a heterogeneous interaction representation : ; Among them, represents element-wise multiplication, represents the concatenation operation, represents the feature vector of the subject object, represents the feature vector of the semantic relation, represents the feature vector of the object object.

[0065] S22: Homogeneous interaction modeling: Calculate the semantic interaction features between similar objects to enhance the consistency of visual similar semantics : ; Among them, and respectively represent the embedding representations of object A and object B.

[0066] S23: Overall interaction modeling: Fuse the target category features, spatial position encoding, and semantic attributes to form an overall representation : ; Among them, represents the embedding representation of the target category, represents the embedding representation of the semantic attribute, represents the embedding representation of the spatial position encoding.

[0067] S24: Hypergraph attention network (HGAN) propagation: To highlight the contribution of key objects in the image to relation modeling, introduce semantic clustering centers , and calculate the attention weights: ; Among them, represents a hyperedge the attention coefficient of the target node in represents a learnable attention projection vector for learning the attention preference of nodes to the semantic clustering center, represents the index variable of the target node, indicating a certain target node inside the hyperedge represents a hyperedge in the channel c the semantic clustering center representation under, such as the shared semantic vector corresponding to the relationship classes "holding" or "placing", represents the target node the embedding feature vector of represents the activation function.

[0068] Furthermore, a hierarchical attention mechanism is introduced to update the target node representation: ; Among them, represents the updated target node i represents a vector, which is the final visual representation of the target node i (such as a certain detection box) after fusing the semantics of multiple relationship hyperedges, represents the target node i the set of all hyperedges connected by represents the target node i the fusion weight of the hyperedge to control the influence degree of different semantic channels on the update of this target representation, represents a learnable linear transformation matrix, represents a hyperedge the semantic representation vector of

[0069] S3: Multi-channel contrast learning based on object detection data, and its implementation method is as follows: S31: Construction of positive and negative samples: The positive sample pair is composed of the object with a real interaction relationship in the image and its corresponding hyperedge; the negative sample pair is randomly sampled from the object area where no semantic interaction occurs.

[0070] S32: Improved InfoNCE loss function: Adopt the node-hyperedge contrast learning method to maximize the mutual information of positive samples and suppress the interference of negative samples: ; Among them, represents the loss function, c represents the channel type, represents the target node index, referring to a certain detection target in the image, e represents a hyperedge, Denote the target node All the hyperedges involved, indicating the position of the target in all relational structures Denote the activation function Denote the consistency discriminator Denote the channel c Lower hyperedge e representation Denote the detected entity in the channel c representation Denote the negative sample hyperedge representation, indicating the semantic structure that is structurally similar but not participated by the target (used for interference).

[0071] S4: Jointly optimize the object detection task and the structural relationship reasoning task, and the implementation method is as follows: S41: Margin ranking loss optimizes the performance of structural relationship modeling: Use margin ranking loss to improve the judgment accuracy of visual semantic triples: ; Among them, Denote the semantic matching scoring function between the target and its context Denote the negative sample Denote the ranking loss, guiding the model to identify whether there is a real semantic relationship between targets Denote the head entity or subject entity, the detected target in the image, such as "person" Denote the tail entity (positive sample), the object target that has a real semantic relationship with the main target Denote the training sample set, that is, the set of all entity combinations that form "positive sample relation pairs" Denote the margin hyperparameter, ensuring that the model can more confidently judge that the positive sample is greater than the negative sample

[0072] S42: Joint training framework design: Jointly optimize the main object detection tasks (such as classification and localization), structural reasoning, and self-supervised tasks to form a unified loss function : ; Among them, and Both denote the weight regulation factors Denote the classification loss Denote the bounding box regression loss

[0073] The method of this embodiment is applied to the image object detection task and has the following technical effects: Significantly enhance the semantic structure modeling ability between targets, improve the detection context consistency and structural integrity; introduce a structure-aware auxiliary supervision signal to improve the detection accuracy of small targets and occluded targets; use a self-supervised mechanism to enhance the model expression ability in unlabeled scenarios and improve the generalization effect; have good cross-image semantic transfer ability and be applicable to high-order vision tasks such as behavior recognition, scene understanding, and autonomous driving in complex scenarios.

Claims

1. A knowledge graph link reasoning method, characterized in that It includes the following steps: S1. Construct a complementary dual hypergraph representation of the social network by retaining the local structure through the social triple integrity hypergraph and capturing the global semantic associations using the social relationship semantic hypergraph. S2. Based on the dual hypergraph representation of the social network, achieve social relationship-guided dynamic semantic propagation by combining the hypergraph attention network through the fusion of composite semantic representations of heterogeneous interaction, homogeneous co-mutual, and overall interaction, and generate a structure-aware and context-dependent representation for each social node. Among them, the social triple integrity hypergraph is used to capture local interaction features, and the local interaction features include heterogeneous interaction, homogeneous co-mutual, and overall interaction. The social relationship semantic hypergraph is used to perceive global semantic differences. S3. Design multi-channel contrast learning based on social data based on the structure-aware and context-dependent representation. S4. Construct multi-channel auxiliary tasks using the designed multi-channel contrast learning, optimize the ranking result of social triple scores based on the margin ranking loss function, and complete the reasoning of knowledge graph links through a joint learning framework that fuses the knowledge graph link prediction task and the self-supervised contrast task. Among them, the self-supervised contrast task is the multi-channel auxiliary task.

2. The knowledge graph link reasoning method according to claim 1, characterized in that The S1 includes the following steps: S101. Construct a hyperedge set of triples based on the knowledge graph of social triples , connect the social head entity, social relationship, and social tail entity to the same hyperedge together, and retain the integrity of the social triples, so as to retain local relationships through the integrity hypergraph of social triples; S102. Construct a set of relational hyperedges based on social relationship semantics , cluster the social head entities and social tail entities with the same social relationship respectively to form a global semantic association, and construct a complementary dual hypergraph representation of the social network.

3. The knowledge graph link reasoning method according to claim 1, characterized in that The S2 includes the following steps: S201. Construct a composite semantic representation containing heterogeneous interaction, homogeneous interaction, and overall interaction by explicitly modeling the multi-type interaction features among social head entities, social relationships, and social tail entities. Among them, the local interaction features include heterogeneous interaction, homogeneous co-mutual, and overall interaction. The social triple integrity hypergraph is used to capture local interaction features. S202. Fuse the composite semantic representation through a weight matrix to generate hyperedge semantic vectors. Among them, in the social node representation update stage, the gated aggregation mechanism is used to achieve the directional propagation of hyperedge information to the target social node. S203. Use the social relationship semantic hypergraph to perceive global semantic differences, and based on the directional propagation, introduce a dynamic attention mechanism, combine the hypergraph attention network, and achieve relationship-guided dynamic semantic propagation to generate a structure-aware and context-dependent representation for each social node.

4. The knowledge graph link reasoning method according to claim 3, wherein The expression of the heterogeneous interaction is as follows: ； Among them, represents the heterogeneous semantic vector, represents the feature vector of the user node, represents the embedded representation of the interaction behavior between the user and the content, represents the feature vector of the content node, and + represents the feature vector addition, represents the element-wise multiplication, represents the concatenation operation; The expression of the homogeneous interaction is as follows: ； Among them, represents the feature similarity between entities of the same type, represents the embedded representation of user A, represents the embedded representation of user B; The expression of the overall interaction is as follows: ； Among them, represents the embedded representation that fuses the user, content, and timestamp, represents the embedding of time information.

5. The knowledge graph link reasoning method according to claim 4, characterized in that The S203 includes the following steps: Based on the directional propagation, through a dynamic weight allocation mechanism, distinguish the contribution degrees of different social entities to the hyperedge semantic representation. Among them, the expression of the attention coefficient is as follows: ； Among them, represents the hyperedge the attention coefficient of the social nodes in it, p represents the attention vector, represents the learnable attention projection vector, represents the activation function, represents the hyperedge the clustering center feature representation of represents the feature representation of the social nodes, represents the hyperedge all the social nodes in , represents the hyperedge the feature representation of a certain social node in Based on the distinguished contribution degrees, in the social node representation update stage, introduce a hierarchical attention mechanism to fuse multi-hyperedge information and generate a structure-aware and context-dependent representation for each social node. The updated representation is as follows: ； ； Among them, represents a social node The updated representation, represents a hyperedge The contribution size to the social node is represents all hyperedges connected to the social node set, represents a hyperedge The aggregation function to the social node, represents a hyperedge The representation of represents a social node The final updated representation of represents a hyperedge The semantic representation vector of represents q The transpose of q represents a trainable attention vector.

6. The knowledge graph link reasoning method according to claim 1, characterized in that The S3 includes the following steps: Organically fuse heterogeneous features using a multi-channel dynamic weight mechanism to generate multi-channel features. Based on the multi-channel features, for each channel, construct positive and negative sample pairs using the structure-aware and context-dependent representation and the dual hypergraph representation of the social network. Based on the constructed positive and negative sample pairs, design an improved loss function to optimize the representation space and complete the design of multi-channel contrast learning based on social data.

7. The knowledge graph link reasoning method according to claim 6, wherein The expression of the multi-channel features is as follows: ； ； Among them, represents multi-channel features, represents a channel c the contribution weight finally represented for the social entity v represents the social entity v in the channel c representation t represents the social triple channel, r represents the social relationship channel, n represents the neighbor channel, represents the channels except the current channel c represents a learnable attention vector, represents a learnable weight matrix, represents the representation of the social entity in the channel representation​​ 8. The knowledge graph link reasoning method according to claim 7, wherein The expression of the improved loss function is as follows: ； Among them, represents the improved loss function, v represents the social entity, V represents all social entities v of the set, e represents the hyperedge, represents the hyperedge set connected to the social entity v connected, represents the activation function, represents the consistency discriminator, represents the channel c under the hyperedge e representation, represents the social entity v in the channel c representation, represents the negative sample hyperedge representation.

9. The knowledge graph link reasoning method according to claim 1, wherein The S4 includes the following steps: Construct a multi-channel auxiliary task using the designed multi-channel contrast learning; Take the structure-aware and context-dependent representation as the embedding input, and calculate the social triple score ranking through a scoring function; Take the social triple score ranking result as the input, and optimize it using a margin-based ranking loss function; Based on the optimization result, integrate the knowledge graph link prediction task and the self-supervised task through a joint learning framework to complete the inference of the knowledge graph link, where the link prediction task is the main task and the self-supervised contrast task is the multi-channel auxiliary task.

10. The knowledge graph link reasoning method according to claim 9, characterized in that The expression of the loss function of the joint learning framework is as follows: ； ； Among them, represents the loss function of the federated learning framework, represents the hyperparameter used to regulate the influence of multi-channel auxiliary tasks, represents the self-supervised auxiliary task loss, represents the margin-based ranking loss function, represents the user entity, represents the positive sample content, T represents the set of training social triples, represents the margin hyperparameter, represents the social triple scoring function, represents the negative sample.

Images