GNN Negative Sampling Using Adaptive Self-Adversarial Feedback

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional graph neural network (GNN) training methods face challenges in balancing the exploitation of hard negative samples while avoiding false negatives, particularly in heterogeneous graphs, leading to issues like vanishing gradients and over-training on false negatives.

Innovation Solution

Adaptive Self-Adversarial (ASA) negative sampling technique that uses a trained model from a prior iteration to evaluate the hardness of negative samples and selects those causing high gradients for training, reducing false negatives by leveraging positive relationships.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If hard negative sampling is used to identify challenging relationships for optimization, then model training effectiveness is improved, but false negative rate increases which corrupts the learning process

Engineering Contradiction:

Improvetraining effectivenessVSAvoidfalse negative rate

Core Design Contradiction:

ReliabilityVSObject-affected harmful factors

Solution Approach 1:

The patent implements a feedback mechanism where the trained model from the current iteration evaluates negative samples in the next iteration. The model scores negative samples based on their predicted probability, and this feedback loop allows the system to dynamically adjust which negative samples are selected, balancing hardness with reliability to avoid false negatives while maintaining training effectiveness.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The patent changes the parameter used to select negative samples from static random selection to dynamic model-based scoring. By using the trained model's predicted probability as the selection criterion and adjusting the temperature parameter, the system adapts the difficulty of negative samples throughout training, resolving the contradiction between hardness and false negative rate.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If random negative sampling is used for simplicity and efficiency, then computational cost is reduced, but vanishing gradient problem occurs as most negative samples can be easily classified

Engineering Contradiction:

Improvetraining efficiencyVSAvoidgradient quality

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent applies self-service by having the trained model itself evaluate and select negative samples without requiring external complex systems like GANs. The model uses its own learned representations to identify challenging negative samples, achieving both efficiency and gradient quality by leveraging its own training progress.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent transforms the negative sampling process by changing from uniform random sampling to sampling based on model-predicted probabilities. This parameter change ensures that negative samples are selected according to their difficulty level, preventing the vanishing gradient problem while maintaining computational efficiency.

Inventive Principle:

Principle #35Parameter changes

3Reliability

If GAN-based negative sampling is used to identify challenging relationships, then gradient quality is improved, but model complexity and training cost increase significantly

Engineering Contradiction:

Improvegradient qualityVSAvoidmodel complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent merges the negative sampling function with the main GNN training process. Instead of using a separate GAN model to generate negative samples, the system combines the scoring function into the existing GNN architecture, eliminating the need for additional generator-discriminator components while maintaining gradient quality.

Inventive Principle:

Principle #5Merging (Combining)

Solution Approach 2:

The patent eliminates the need for external complex systems by having the trained model itself serve as the negative sampler. The model uses its own learned parameters to evaluate and select negative samples, replacing the GAN-based approach with a self-sufficient mechanism that reduces overall system complexity.

Inventive Principle:

Principle #25Self-service

4Ease of operation

If uniform thresholding is applied to identify false negatives, then decision boundary is simplified, but performance degrades in heterogeneous graphs where decision boundaries vary

Engineering Contradiction:

Improvedecision boundary simplicityVSAvoidfalse negative detection accuracy

Core Design Contradiction:

Ease of operationVSReliability

Solution Approach 1:

The patent applies local quality by using model-specific probability scores instead of a global uniform threshold. Each negative sample is evaluated based on its local characteristics as predicted by the trained model, allowing the system to adapt to heterogeneous graph structures where different regions have different decision boundaries, thereby improving false negative detection accuracy.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS12505350B2Adaptive self-adversarial negative sampling for graph neural network training

Publication Date: 2025.12.23 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

A graph neural network (GNN) training method, system, and computer program product in a graph, include generating, by the computing device, one or more one or more hypothetical edges between two or more nodes of a plurality of nodes of a graph neural network, testing, by the computing device, to determine whether the one or more generated hypothetical edges should be connected by using negative sampling, and permanently connecting, by the computing device, the one or more tested hypothetical edges if the negative sampling indicates the connectivity.