Invertible Factorization for Classifier Invariance Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Classifiers, especially deep neural networks, require substantial labeled training data and existing data augmentation methods only allow for limited invariances to be learned, leading to suboptimal classification accuracy.
Innovation Solution
A computer-implemented method using an invertible factorization model to train classifiers by disentangling latent representations, allowing for augmentation based on both low-level and high-level factors, and employing neural architecture search to optimize factor disentanglement and classification accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing data augmentation methods are used to train classifiers, then the classifier can learn some invariances, but the amount of invariances that can be learned is limited, leading to suboptimal classification accuracy
Solution Approach 1:
The invention segments the data augmentation process by separating low-level factors (brightness, noise, contrast) from high-level factors (rotation, translation, scaling) and processing them through distinct computational pathways in the neural network architecture, allowing each type of transformation to be learned independently and combined effectively
Solution Approach 2:
The invention adds a new dimension to data augmentation by introducing explicit factor representations that capture both low-level and high-level transformations simultaneously, enabling the classifier to learn invariances across multiple dimensions of variation rather than being constrained to traditional single-type augmentations
2Measurement precision
If substantial amounts of labeled training data are collected to improve classification accuracy, then the classifier performance improves, but the cost and time required for data collection and labeling increases
Solution Approach 1:
The invention creates synthetic copies of training data by applying learned low-level and high-level factor transformations to existing labeled examples, generating additional training instances without requiring new data collection or labeling efforts
Solution Approach 2:
The invention performs preliminary learning of transformation factors during the training process, establishing a foundation that enables efficient data augmentation and improves classification accuracy without requiring extensive additional labeled data to be collected beforehand
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Computer-implemented method for training a classifier (60), wherein the classifier (60) is configured to determine an output signal (y) characterizing a classification of an input signal (x), wherein training the classifier (60) comprises the steps of: • Determining a first training input signal (xi); • Determining a first latent representation (z) comprising a plurality of factors (z1, z2, z3, z4) based on the first training input signal (xi) by means of an invertible factorization model (61), wherein the invertible factorization model (61) is characterized by: - A plurality of functions (F1, F2, F3, F4), wherein any one function (F1, F2, F3, F4) from the plurality of functions is continuous and almost everywhere continuously differentiable, wherein the function (F1, F2, F3, F4) is further configured to accept either the training input signal (xi) or at least one factor (z1, z2, z3, z4) provided by another function (F1, F2, F3, F4) of the plurality of functions (F1, F2, F3, F4) as an input and wherein the function (F1, F2, F3, F4) is further configured to provide at least one factor (z1, z2, z3, z4), wherein the at least one factor (z1, z2, z3, z4) is either provided as at least part of the latent representation (z) or is provided as at least part of an input of another function (F1, F2, F3, F4) of the plurality of functions, wherein there exists an inverse function that corresponds with the function (F1, F2, F3, F4), is continuous, is almost everywhere continuously differentiable and is configured to determine the input of the layer based on the at least one factor (z1, z2, z3, z4) provided from the function (F1, F2, F3, F4); • Determining a second latent representation by adapting at least one factor (z1, z2, z3, z4) of the first latent representation (z); • Determining a second training input signal based on the second latent representation by means of the invertible factorization model; • Training the classifier based on the second training input signal.