Neural Embedding Inversion via Mixed-Integer Optimization

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing artificial intelligence systems face challenges in understanding the meaning, features, and characteristics of data samples when these samples do not explicitly indicate them, leading to a lack of comprehensive understanding in datasets used for training AI systems.

Innovation Solution

The method involves generating forward neural embeddings for data samples, determining inverse neural embeddings, and generating inverse feature sets by minimizing a target function under constraints that each element of the feature set has non-negative integer values, using mixed-integer quadratic programming to solve for the inverse feature sets.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If forward neural embeddings are generated for data samples, then the representation of data in embedding space is improved, but the ability to recover meaningful features from these embeddings deteriorates

Engineering Contradiction:

Improveembedding representation accuracyVSAvoidfeature recoverability

Core Design Contradiction:

Measurement precisionVSLoss of information

Solution Approach 1:

The patent applies inversion by formulating and solving an optimization problem that recovers the feature set from the neural embedding. Instead of only transforming features to embeddings (forward direction), the system performs the reverse operation by minimizing a target function that measures the difference between the generated embedding and the target embedding, thereby recovering the original or approximate feature set.

Inventive Principle:

Principle #13The other way round (Inversion)

2Measurement precision

If inverse neural embeddings are determined through optimization, then feature set recovery is improved, but the computational complexity increases

Engineering Contradiction:

Improvefeature recovery accuracyVSAvoidcomputational complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The patent extracts the essential components needed for inversion by formulating a targeted optimization problem. Rather than performing complex full-scale neural network inversion, the system extracts and optimizes only the necessary feature set variables that map to the given neural embedding, reducing the overall computational burden while maintaining recovery accuracy.

Inventive Principle:

Principle #2Taking out (Extraction)

3Manufacturing precision

If mixed-integer quadratic programming is used to solve for inverse feature sets, then the precision of inverse feature sets is improved, but the processing time increases

Engineering Contradiction:

Improveinverse feature set precisionVSAvoidprocessing time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent employs parameter changes by adjusting the target function and constraints of the mixed-integer quadratic programming problem. By modifying the objective function to measure embedding difference and setting appropriate constraints, the system achieves precise feature set recovery while the solver efficiently handles the optimization to balance precision and processing time.

Inventive Principle:

Principle #35Parameter changes

Data Source

PatentUS12001790B2Methods for exact and approximate inverses of neural embedding models

Publication Date: 2024.06.04 CAPITAL ONE SERVICES LLC

AI summary

Systems, apparatuses, and methods are described for inverting neural embeddings. One or more forward neural embeddings associated with meanings, features, and/or characteristics of data samples may be generated for one or more data samples. One or more inverse neural embeddings associated with the one or more forward neural embeddings may be determined. One or more inverse feature sets for the one or more inverse neural embeddings may be generated.