Spherization Layer for Angle-Based Representation Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing representation learning methods in artificial neural networks face performance degradation due to the loss of information when using angle-based similarity measures, as they disperse learning information across Euclidean norms and angles, leading to inaccurate representation learning.
Innovation Solution
A spherization layer is introduced, comprising an angularization unit that converts hidden vectors into angle vectors within a specific range, a conversion unit that maps these vectors to a hyperspherical plane, and a learning unit that learns representations using only angles, ensuring all information is captured without loss by positioning vectors on a hyperspherical surface with hyperplanes fixed to the origin.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If inner product is used for learning representations, then learning information is dispersed across Euclidean norm and angle, but information loss occurs when using only angle-based similarity measures
Solution Approach 1:
The patent extracts only the angular component from the inner product by using a spherization layer that projects vectors onto a hyperspherical surface. This extraction eliminates the Euclidean norm component while preserving the angle information, thereby preventing information loss when using angle-based similarity measures without requiring complex modifications to the entire learning system.
Solution Approach 2:
The patent applies spheroidality by introducing a spherization layer that transforms the representation space into a hyperspherical manifold. This curvature-based transformation ensures that all vectors lie on a hyperspherical surface, making angle-based similarity measures effective while maintaining all learning information in the angular relationships between vectors.
2Measurement precision
If angle-based similarity measures are used, then information loss occurs, but computational simplicity is maintained
Solution Approach 1:
The patent applies preliminary action by pre-processing the vector representations through a spherization layer before similarity measurement. This preliminary transformation ensures that vectors are positioned on a hyperspherical surface with consistent norms, so that subsequent angle-based similarity measurements accurately reflect the learned representations without information loss.
3Loss of information
If all learning information is applied to angles, then information loss is prevented, but the learning mechanism becomes more complex
Solution Approach 1:
The patent introduces a spherization layer as an intermediary component between the standard neural network layers and the output layer. This intermediary transforms vectors into a hyperspherical representation, enabling angle-based similarity measures to capture all learning information without requiring fundamental changes to the learning mechanism itself.
Data Source
AI summary
The present invention relates to representation learning in an artificial neural network, and more specifically, to a device and method for learning representations using a spherization layer, which places all hidden vectors on a hyperspherical surface, and learns representations using only angles on the basis of hyperplanes fixed to the origin. According to an embodiment of the present invention, as all hidden vectors are represented on a hypersphere in a space of one dimension higher, and representation learning is performed thereon using only angles through the hyperplanes fixed to the origin, the problem of performance degradation of artificial neural networks can be solved by ensuring that all information learned by the artificial neural network from input data is contained in the angle without loss.