Neural Network Quantile Normalization for Domain-Shifted Measurement Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural networks used for processing measurement data in automated driving systems face challenges in maintaining robustness against domain shifts, where the distribution of input data changes from that used during training.
Innovation Solution
The method involves normalizing the work results of selected neurons in the neural network using a target distribution and an inverse cumulative density function, which maps the work results to unit values in the interval [0,1], preserving their ranking while compressing or stretching numerical distances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Stability of the object's composition
If traditional normalization techniques are used to stabilize training, then gradient stability is improved, but robustness against domain shifts deteriorates
Solution Approach 1:
The patent changes the normalization parameter from statistical moments (mean and variance) to quantiles derived from a target distribution. This parameter transformation allows the normalization to be more robust to domain shifts while maintaining training stability, as quantiles are less sensitive to distributional changes than mean and variance.
Solution Approach 2:
The patent introduces a target distribution as an intermediary between the input data and the neural network processing. This target distribution serves as a reference framework that guides the normalization process, enabling the system to adapt to different input distributions while maintaining consistent processing behavior.
2Loss of time
If work results are normalized using statistical moments, then training convergence is improved, but invariance to domain shifts deteriorates
Solution Approach 1:
The patent transforms the normalization parameters from statistical moments (mean, variance) to quantile-based parameters derived from a target distribution. This parameter change maintains training convergence while providing invariance to domain shifts, as quantiles naturally adapt to different distributions without requiring retraining.
3Productivity
If conventional normalization is applied to all neurons, then computational efficiency is improved, but processing robustness deteriorates
Solution Approach 1:
The patent applies different normalization strategies to different groups of neurons based on their functional importance. Critical neurons use quantile-based normalization for robustness, while less critical neurons use standard normalization for efficiency. This local differentiation maintains overall robustness while preserving computational efficiency.
Data Source
AI summary
A method for processing measurement data in a neural network including a plurality of layers of neurons. In the method: inputs supplied to each neuron are processed according to parameters associated with the neuron to produce a work result; a group of neurons is selected; a target distribution for the work results of the neurons of the group is defined; an inverse cumulative density function of the target distribution is provided; the work results are mapped with a predetermined unit function onto unit values in an interval, so that the unit value associated with each work result occupies the same rank on the list of all unit values as the corresponding work result occupies on the list of all work results; normalized work results are calculated from the unit values using the inverse cumulative density function; the normalized work results are further processed to produce outputs of the neural network.