Neural Data Storage Compression Using Low-Dimensional Parameter Vectors
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage methods for high-dimensional data require significant storage space, even when using auto encoders, as they still need to store compressed low-dimensional data and network model parameters, leading to inefficiencies in storage capacity.
Innovation Solution
A method involving affine transformation to generate multiple low-dimensional parameter vectors from a high-dimensional parameter vector, combined with a trained neural network model, allows for storing these vectors to represent multiple high-dimensional data points, thereby reducing storage needs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Volume of stationary object
If auto encoders are used to compress high-dimensional data to low-dimensional data, then storage space is reduced to a certain extent, but the storage space still requires significant capacity to store both compressed low-dimensional data and network model parameters
Solution Approach 1:
The patent extracts only the essential low-dimensional parameter vectors from the high-dimensional data, separating the core information from the redundant data. By storing only these extracted parameter vectors rather than the complete compressed data plus model parameters, the storage requirement is significantly reduced while maintaining the ability to reconstruct the original high-dimensional data when needed.
Solution Approach 2:
The patent performs preliminary transformation of high-dimensional data into low-dimensional parameter vectors using affine transformation before storage. This preliminary action converts the data into a compact form that captures the essential information, enabling efficient storage and subsequent reconstruction without needing to retain the full original data or complete model parameters.
2Volume of stationary object
If multiple high-dimensional data points are represented by a single low-dimensional parameter vector, then storage efficiency is improved, but the complexity of the transformation and reconstruction process increases
Solution Approach 1:
The patent changes the parameter representation from high-dimensional data points to low-dimensional parameter vectors through affine transformation. By transforming the data into a different parameter space with fewer dimensions, the storage efficiency is improved while the transformation process remains mathematically systematic and manageable through established linear algebra operations.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
Embodiments of the present application provide a data storage method, data acquisition method and device thereof. The method includes allocating an N-dimensional first parameter vector for N pieces of to-be-stored data; performing N-dimensional permutation on the first parameter vector, to obtain N second parameter vectors each having N dimensions; constructing a neural network model that maps the current second parameter vectors to expected data samples of the N pieces of to-be-stored data; adjusting model parameters of the neural network model and/or the first parameter vector until expected data samples of the N pieces of to-be-stored data regress to the N pieces of to-be-stored data, the expected data samples being obtained from the current second parameter vectors based on the trained neural network model; storing the current first parameter vector. The embodiments of the present application make the storage of the first parameter vector equivalent to storing N pieces of to-be-stored data, which reduces high-dimensional data to low-dimensional data for storage, thus greatly reducing the storage space.