A parameter-embedding collaborative enhanced knowledge graph aware recommendation method

Abstract

This invention discloses a parameter-embedding collaborative enhancement method, system, device, and medium for knowledge graph-aware recommendation, aiming to solve the embedding saturation problem in knowledge graph recommendation. The method constructs a collaborative knowledge graph integrating user-item interaction data and the knowledge graph, and performs linear feature propagation by simplifying GNN. During the training phase, a robust embedding integration module and a graph reactivation module are enabled to smooth the embedding evolution trajectory and reactivation degradation representation, respectively, and combined with efficient contrastive learning to enhance representation discriminativeness. A joint loss function is used to achieve collaborative optimization of parameters and embeddings, and the training-specific module is removed during the inference phase, resulting in zero additional overhead. This invention effectively maintains the high-rank structure of the embedding space, improves generalization ability in sparse scenarios, and outperforms existing state-of-the-art methods in terms of Recall and MRR on multiple public datasets, making it suitable for personalized recommendation scenarios such as e-commerce and content platforms.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of artificial intelligence, knowledge graphs, and personalized recommendation systems, and particularly to the application of Graph Neural Networks (GNNs) in knowledge graph-aware recommendation. Specifically, this invention relates to an intelligent recommendation method that integrates a lightweight parameterization mechanism and embedded activation modules, as well as a recommendation system, electronic device, and computer-readable storage medium built based on this method. This invention is particularly suitable for solving key technical problems in knowledge graph-aware recommendation scenarios, such as decreased model discriminative power due to embedding saturation, stagnant recommendation performance, and insufficient generalization ability in sparse scenarios. Background Technology

[0002] With the widespread adoption of mobile internet and the explosive growth of digital content, recommender systems have become a core bridge connecting users with massive amounts of information, and are widely used in e-commerce, social media, content platforms, and other scenarios. Traditional recommendation methods rely on user-item interaction data to model preferences, but face two core challenges: data sparsity and cold start.

[0003] To alleviate the aforementioned problems, Knowledge Graph-aware Recommendation (KGR) has emerged. This type of method significantly improves recommendation performance in sparse scenarios by introducing an external knowledge graph (KG) and fusing semantic information of items with user interaction signals. With the rise of graph neural networks, models such as KGAT and LightKG further enhance the representation capabilities of users and items by aggregating neighborhood features through high-order information propagation.

[0004] Despite the progress made in existing knowledge graph-aware recommendation models, a core flaw that is often overlooked during training is embedding saturation, which manifests in the following four main problems:

[0005] 1. Representation collapse problem: As the number of training rounds increases, the embedding vectors of users and items gradually converge to low-rank subspaces, the covariance spectrum collapses rapidly, representation diversity is lost, and the model's discriminative power drops sharply.

[0006] 2. Inefficient mitigation methods: Existing methods often delay saturation by "expanding" the embedding dimension and deepening the network structure, which leads to a linear increase in inference latency, contradicting the industry's demand for lightweight deployment; while post-processing regularization methods such as dropout and gradient pruning cannot effectively restore representation diversity after embedding collapse.

[0007] 3. Lack of dynamic intervention mechanism: Existing technologies ignore the dynamic evolution process of embedded saturation, and have not established an effective monitoring and real-time intervention mechanism. They can only passively wait for the recommended indicators to stagnate, and cannot make targeted adjustments when saturation first appears.

[0008] 4. Sparse scenarios highlight contradictions: In sparse data scenarios, the scarcity of supervision signals amplifies the embedding saturation problem, severely limiting the model's generalization ability. Such scenarios are precisely the core application scenarios of knowledge graph recommendation.

[0009] In summary, how to construct a knowledge graph-aware recommendation framework that can proactively perceive and alleviate embedding saturation during the training phase without increasing inference overhead is a technical challenge that urgently needs to be addressed in this field. Summary of the Invention

[0010] The purpose of this invention is to provide a knowledge graph perception recommendation method, system, device and medium with parameter-embedding collaborative enhancement. By designing a lightweight intervention module dedicated to the training phase, the embedding saturation problem can be dynamically alleviated without increasing inference overhead, thereby improving the discriminative power and generalization ability of the recommendation system.

[0011] Technical solution:

[0012] In a first aspect, the present invention provides a knowledge graph-aware recommendation method with parameter-embedding collaborative enhancement, comprising the following steps:

[0013] S1. Data Acquisition and Graph Structure Initialization Steps: Acquire raw data for the recommendation scenario, including user-item interaction records and knowledge graph data (including entities, relations, and triples); Based on the raw data, construct a Collaborative Knowledge Graph (CKG), which integrates the user-item interaction graph and the knowledge graph; Simultaneously, initialize low-dimensional embedding vectors for all users, items, and knowledge graph entities.

[0014] S2. Simplified GNN Layer Feature Propagation Steps: On the collaborative knowledge graph, a simplified graph neural network is used to perform feature propagation and aggregation operations. Scalar encoding relationships are adopted, and neighborhood node features are aggregated through a linear aggregation framework to generate node embeddings for each layer. The embeddings from each layer are weighted and averaged to obtain preliminary user and item representations. Mathematically, this is expressed as:

[0015] S3. Robust Embedding Integration Step (REIM): The Robust Embedding Integration module is enabled during the training phase, dynamically integrating the embedding evolution trajectory through a multi-scale attention mechanism. Embedding channels are grouped, spatial attention weights are extracted through parallel sub-networks, and cross-spatial information aggregation is combined to smooth and optimize the path, suppressing embedding collapse trends. This module is only effective during the training phase and is removed during inference.

[0016] S4. Graph Reactivation Step (GRAM): The graph reactivation module is enabled after each graph propagation to perform channel-spatial recalibration on the embeddings. Channel weights are generated using global statistics, spatial weights are generated by fusing the maximum and mean responses, and degenerate embedding representations are reactivated by combining channel dropout and Gaussian noise injection. This module is also only active during the training phase and is removed during the inference phase.

[0017] S5. Efficient Contrast Learning Steps: Construct an efficient contrastive layer that directly minimizes the similarity between different node embeddings, enhancing representation discriminativity. Introduce node similarity and neighborhood quantity weights to optimize the contrastive loss, alleviate the oversmoothing problem, eliminate the need to generate subgraphs, and reduce computational overhead. Mathematically, this is expressed as:

[0018] S6. Joint Loss Optimization Steps: Construct a joint loss function that includes the recommendation main task loss, contrastive loss, and regularization term. Update all model parameters through gradient backpropagation to achieve collaborative optimization of parameters and embeddings. Mathematically, this is expressed as:

[0019] S7. Recommendation Result Generation Steps: During the inference phase, the REIM and GRAM modules are removed. The trained model parameters are used to calculate predicted ratings for users and candidate items. A recommendation list is generated based on these ratings. The predicted ratings are calculated as follows:

[0020] Secondly, the present invention provides a knowledge graph perception recommendation system with parameter-embedding collaborative enhancement, comprising:

[0021] The data initialization module is used to build a collaborative knowledge graph and initialize the embedding vectors of users, items, and entities. A simplified GNN module is used to perform feature propagation through a linear aggregation framework to generate preliminary node representations. A robust embedding integration module is used to smooth the embedding evolution trajectory during the training phase and suppress collapse. The graph reactivation module is used to recalibrate embedded features and reactivate degenerate representations during the training phase. A high-efficiency comparison module is used to enhance embedding discriminativeness and alleviate over-smoothing. The joint optimization module is used to update model parameters based on the joint loss function. The recommendation prediction module is used to generate user-item prediction scores during the inference phase and output a recommendation list.

[0022] Technical effects:

[0023] The beneficial effects of this invention include:

[0024] 1. Proactively alleviate embedding saturation: Through the synergistic effect of REIM and GRAM modules, intervene in the embedding evolution process in real time, maintain the high-rank structure and semantic diversity of the embedding space, and fundamentally solve the representation collapse problem.

[0025] 2. Zero inference overhead: The REIM and GRAM modules are only enabled during the training phase and completely removed during the inference phase, without adding extra parameters or computational burden, meeting the lightweight deployment needs of the industry.

[0026] 3. Strong robustness in sparse scenarios: In sparse data scenarios, the parameter-embedding collaborative enhancement mechanism effectively utilizes the semantic information of the knowledge graph, significantly improving the model's generalization ability, with the MRR@10 index improving by up to 21.94%.

[0027] 4. Balancing performance and efficiency: The simplified GNN layer and efficient comparison layer reduce the basic complexity of the model. The collaborative enhancement mechanism improves recommendation accuracy without sacrificing efficiency. The Recall, MRR, NDCG and other metrics on multiple public datasets are better than the existing state-of-the-art methods. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall framework of the parameter-embedding collaborative enhancement recommendation method provided in the embodiments of the present invention.

[0029] Figure 2 This is a detailed flowchart illustrating the workflow of the Robust Embedded Integration Module (REIM) in an embodiment of the present invention.

[0030] Figure 3 This is a detailed flowchart illustrating the workflow of the Graphic Reactivation Module (GRAM) in an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0032] Example 1: Detailed explanation of the recommended method and process.

[0033] This embodiment details a parameter-embedded joint saturation-resolution network (PESatNet) for knowledge graph-aware recommendation. The core logic of this method is as follows: Figure 1 As shown, the embedding saturation problem is alleviated through the synergistic effect of parameter optimization and embedding activation during the training phase. The overall process is as follows:

[0034] Step S1: Data acquisition and graph structure initialization.

[0035] In the backend of a real recommendation system, the raw data is first processed and a basic graph structure is built.

[0036] Data definition:

[0037] User set ,in This represents the total number of users.

[0038] Item Collection ,in This represents the total number of items.

[0039] Interactive data , indicating user With items Interactions have occurred (such as clicks, purchases, ratings).

[0040] Knowledge graph data ,in For a collection of entities, For a set of relations, Representing entities and Existence Relationship .

[0041] Graph construction:

[0042] Collaborative Knowledge Graph (CKG): Integrates user-item interaction graphs and knowledge graphs, with node sets as follows: edge set is Construct the adjacency matrix and perform normalization (such as Laplace normalization).

[0043] Embedded initialization:

[0044] For each user ,thing and entity c initialization 3D embedding vector and The system uses a Xavier distribution for random initialization and is continuously updated during training.

[0045] Step S2: Simplify feature propagation in GNN layers.

[0046] A lightweight linear aggregation framework is adopted to avoid embedding saturation acceleration caused by complex nonlinear transformations.

[0047] Relational scalar encoding: encoding each relation Encode as a pair of learnable scalars and , respectively representing entities arrive , arrive This significantly reduces computational complexity by highlighting the importance of relationships.

[0048] Hierarchical feature fusion: For the first Layer propagation, nodes Aggregate its domain nodes The features are calculated as shown in formula S2 of the invention. Wherein, the neighborhood set... It includes all nodes connected by interaction relationships and knowledge graph relationships.

[0049] Multi-layer embedded fusion: The embedded vectors obtained from layer propagation are weighted and averaged to generate preliminary user and item representations. and Preserve semantic information at each layer.

[0050] Step S3: Robust Embedded Integration (REIM Module).

[0051] like Figure 2 As shown, the REIM module integrates embedded evolutionary trajectories through a multi-scale attention mechanism and is only enabled during the training phase.

[0052] Feature grouping: Dividing the embedded channels into Sub-characteristics Each group of dimensions is This ensures that spatial semantic features are evenly distributed.

[0053] Parallel subnetworks:

[0054] Perform global average pooling along the horizontal and vertical directions to obtain and It captures long-distance dependencies and retains location information.

[0055] After concatenation, intermediate features are generated through 1×1 convolution, segmented, reshaped, and then gated with a Sigmoid gate to obtain spatial weights. and Adjust the input features.

[0056] Cross-spatial learning:

[0057] The outputs of the 1×1 branch and the 3×3 branch are subjected to global average pooling and Softmax normalization respectively to obtain the channel descriptors.

[0058] Global spatial attention representations at different scales are generated through matrix multiplication, aggregated, and then gated with a Sigmoid gate to obtain the final weights and calibrate the input features.

[0059] Output reshaping: Reshape the output of all groups to the original embedding dimensions to complete the integration process.

[0060] Step S4: Graph reactivation (GRAM module).

[0061] like Figure 3 As shown, the GRAM module performs embedding recalibration after each graph propagation, which is enabled only during the training phase.

[0062] Input camouflage and shape restoration: Dimensional entity embedding rearrangement The pseudo-feature map is used to facilitate 2D convolution operations; after processing, an inverse transformation is performed to restore the original shape.

[0063] Channel recalibration: Calculate the global average pooling result of the pseudo-feature map, and generate channel weights through two levels of 1×1 convolution. The pseudo-feature map is multiplied element-wise to enhance the effective channel features.

[0064] Spatial recalibration: Calculate the maximum and mean responses along the channel dimension, concatenate them, and then generate spatial weights through convolution. The original pseudo-feature map is multiplied element-wise and added back to itself to form the input-level residual.

[0065] Channel dropout and noise injection: Perform 1-D channel dropout (dropout probability 0.2) on the recalibrated features and add Gaussian noise (σ=0.01) to improve generalization ability.

[0066] Step S5: Efficient comparative learning.

[0067] Construct a contrast layer that does not require the generation of subgraphs to enhance embedding discriminativeness.

[0068] Node similarity calculation: Measure nodes and The degree of overlap in the neighborhood.

[0069] Neighborhood quantity weight: This reflects the degree to which nodes are affected by excessive smoothing.

[0070] Contrast loss optimization: Minimize the weighted similarity of different user (or item) embeddings, as shown in formula S5 of the invention, to enhance the uniqueness of the representation.

[0071] Step S6: Joint loss optimization.

[0072] By collaboratively optimizing model parameters through multi-task loss functions, the parameters and embeddings are enhanced in a synergistic manner.

[0073] Recommendation main task loss: The BPR loss is used to maximize the difference between the predicted ratings of users for positive sample items and the ratings for negative sample items. The formula is:

[0074] Step S7: Generate recommendation results.

[0075] The inference stage retains only the simplified GNN layer and the recommendation prediction module, ensuring zero additional overhead.

[0076] Feature fusion: Utilizing trained embedding vectors and model parameters to generate the final user representation. and the final representation of the item .

[0077] Rating prediction: Calculating user ratings and items Predicted score .

[0078] Recommendation list generation: Sort candidate items in descending order of predicted scores, and select the Top-N items to recommend to the user.

[0079] Example 2: Recommendation System Architecture.

[0080] This embodiment provides a recommendation system architecture based on the above method, which adopts a microservice architecture design and includes the following core modules:

[0081] Data initialization module: Connects to the data warehouse, reads user-item interaction logs and knowledge graph data, constructs a sparse adjacency matrix (CSR format) of the collaborative knowledge graph, and performs ID mapping and embedding initialization.

[0082] Simplify the GNN module: Deploy a linear aggregation framework, periodically update the embedding vectors of users, items, and entities, and store them in the feature storage system.

[0083] Training Enhancement Module: Contains REIM and GRAM submodules, which are activated only during model training to perform embedding integration and reactivation operations.

[0084] Contrastive learning module: Deploys an efficient contrastive layer independently, calculates the contrastive loss in real time, and participates in gradient backpropagation.

[0085] Model optimization module: It adopts the Adam optimizer and synchronously updates the parameters of all modules based on the joint loss function, supporting distributed training (Parameter Server architecture).

[0086] Online inference module: Remove the training enhancement module on the server side, and use the trained parameters to respond to user requests in real time and generate a recommendation list.

[0087] Example 3: Electronic device and storage medium.

[0088] The present invention also provides an electronic device, comprising:

[0089] Processor: can be a central processing unit (CPU), graphics processing unit (GPU), tensor processing unit (TPU), or field-programmable gate array (FPGA).

[0090] Memory: Used to store computer programs and massive amounts of graph data, including high-speed random access memory (DRAM) and non-volatile memory (such as NVMe SSD).

[0091] Communication interface: Used for data transmission with clients or other servers.

[0092] Bus: Connects the above components.

[0093] When the processor executes the computer program stored in the memory, it implements each step of the parameter-embedding collaborative enhancement knowledge graph perception and recommendation method described in Embodiment 1.

[0094] In addition, the present invention also provides a computer-readable storage medium (such as a hard disk, optical disk, USB flash drive, cloud storage space) storing computer instructions that, when executed by a processor, implement the above-described method.

[0095] Example 4: Model Complexity and Performance Analysis.

[0096] Time complexity: The overall model complexity is on par with mainstream GNN recommendation models (such as LightKG). The REIM and GRAM modules only introduce... The computational cost of the constant term is less than 1%; the efficient contrast layer avoids subgraph generation, with a complexity of O(n log n). In large-scale map scenarios, this can be ignored.

[0097] Experimental performance: Experiments on four public datasets, Amazon-Book, MovieLens-1M, Book-Crossing, and Last.FM, show that the model of this invention (PESatNet) outperforms the state-of-the-art baseline model (SOTA).

[0098] Recommendation accuracy: Recall@10 improved by an average of 4.64%-13.02%, and MRR@10 improved by an average of 1.37%-21.94%, with significant advantages, especially in sparse scenarios.

[0099] Anti-saturation capability: The embedding covariance rank remains stable during training, and the maximum eigenvalue decreases by less than 5%, which is far better than the baseline model (decreases by >30%).

[0100] Inference efficiency: Comparable to LightKG inference speed, with no additional latency.

[0101] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A knowledge graph perception and recommendation method with parameter-embedding collaborative enhancement, characterized in that, Includes the following steps: Acquire user-item interaction data and knowledge graph data, construct a collaborative knowledge graph that integrates the interaction data and knowledge graph data, and initialize embedding vectors for user nodes, item nodes, and entity nodes in the collaborative knowledge graph; based on the collaborative knowledge graph, perform linear feature propagation and aggregation through a simplified graph neural network to generate preliminary user representations and item representations; during the training phase, enable a robust embedding integration module to perform multi-scale attention integration on the embedding evolution trajectory of the preliminary representations to smooth and optimize the path; After each graph propagation, the graph reactivation module is enabled to perform channel-space recalibration and noise injection on the embedding vectors, reactivating the degenerate representation. The weighted similarity of different node embeddings is minimized through an efficient contrastive layer to enhance the discriminative power of the representation. A joint loss function is constructed, which includes the recommendation task loss, contrastive loss, and regularization term, and the model parameters are updated through gradient backpropagation. During the inference phase, the robust embedding integration module and the graph reactivation module are removed, and the recommendation score is calculated based on the optimized user representation and item representation to generate a recommendation list.

2. The method according to claim 1, characterized in that, The simplified graph neural network performs linear feature propagation and aggregation steps including: encoding each relation in the knowledge graph into a pair of learnable scalars, representing the importance of the bidirectional relation respectively; for each node, aggregating the embedding vectors of its neighboring nodes, and calculating the current layer embedding by combining the relation scalar and the neighborhood size normalization factor; and weighting the embedding vectors of each layer to obtain the preliminary user representation and item representation.

3. The method according to claim 1, characterized in that, The robust embedding integration module includes the following steps: grouping the embedding vectors by channel dimension to obtain multiple sets of sub-features; extracting spatial attention weights in the horizontal and vertical directions for each set of sub-features through parallel sub-networks; generating a multi-scale global attention representation by cross-spatial information aggregation; calibrating the sub-features using the attention representation, and reshaping them to obtain the integrated embedding vector.

4. The method according to claim 1, characterized in that, The operation steps of the graph reactivation module include: rearranging the embedded vectors into pseudo-feature maps to facilitate channel-space recalibration; generating channel weights through global statistics to enhance effective channel features; fusing the maximum and mean responses of the embedded vectors to generate spatial weights and form input-level residual connections; and performing channel dropout and Gaussian noise injection on the recalibrated embedded vectors to improve generalization ability.

5. The method according to claim 1, characterized in that, The calculation steps of the efficient comparison layer include: calculating node similarity based on the degree of overlap of node neighborhoods; designing a weighting factor according to the size of the node neighborhood to reflect the degree of influence of excessive smoothing on the node; and minimizing the similarity of different user or item embeddings by using node similarity and weighting factor as constraints.

6. The method according to claim 1, characterized in that, The joint loss function is a weighted sum of the Bayesian personalized ranking loss, user comparison loss, item comparison loss, and L2 regularization term for the recommendation main task, and the parameters and embedding are optimized collaboratively through gradient backpropagation.

7. The method according to claim 1, characterized in that, The method for calculating recommendation scores based on optimized user and item representations is vector inner product, and the top-N items are selected by sorting the scores in descending order to generate a recommendation list.

8. A knowledge graph perception and recommendation system with parameter-embedding collaborative enhancement, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

Images