Non-linear Lattice Layer for Partial Monotonicity in Neural Networks
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks struggle to effectively model non-linear relationships between input features and output in partially monotonic regression tasks, leading to inefficiencies and potential unfairness in scoring systems.
Innovation Solution
A non-linear lattice layer is introduced in the Deep Lattice Network (DLN) by subdividing the unit hypercube into more granular sub-hypercubes, allowing for more detailed parameterization and relative positioning of vertices to ensure partial monotonicity, which enhances the network's ability to handle non-linear relationships and maintain monotonicity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a standard neural network is used for regression, then the model can learn complex non-linear relationships, but it cannot guarantee monotonicity constraints on output
Solution Approach 1:
The input space is segmented into a lattice structure with vertices representing discrete input combinations. Each vertex is assigned a parameter that respects monotonicity constraints, allowing the model to capture non-linear relationships while guaranteeing monotonic output behavior through the structured segmentation of the input domain.
2Measurement precision
If the unit hypercube is subdivided by a large number k to increase vertex granularity, then the lattice layer can better model non-linear relationships, but the number of parameters increases significantly to k^S
Solution Approach 1:
The patent applies local quality by allowing different levels of subdivision in different dimensions based on the importance and variability of each input feature. This selective refinement enables high measurement precision in critical regions while keeping the overall parameter count manageable through non-uniform grid spacing and adaptive subdivision strategies.
3Ease of manufacture
If the lattice layer uses a fixed grid structure, then the implementation is simple, but it cannot adapt to different data distributions and complexities
Solution Approach 1:
The lattice structure transitions from a static fixed grid to a dynamic adaptive structure where the grid spacing and vertex positions are adjusted based on the learned data distribution. This allows the lattice layer to maintain implementation simplicity through its structured nature while gaining adaptability to different data complexities through learned parameter modifications.
Data Source
AI summary
A computer-implemented method for training a lattice layer in a Deep Lattice Network includes preparing parameters of vertices, each of the parameters corresponding to each vertex of a subdivided unit hypercube defined by subdividing an S-dimensional unit hypercube by a s predetermined number k with k vertices and defining each parameter by identifying one vertex in a specific order, identifying a first set of vertices that appear before the identified vertex in the specific order, identifying a second set of vertices that appear before the identified vertex in the specific order, defining a lower bound as a maximum value among values of vertices in the first set of vertices, defining an upper bound as a minimum value among values of vertices in the second set of vertices, and defining the parameter of the identified vertex based on the lower bound, the upper bound, and a parameter corresponding to the identified vertex.