Tensor Neural Networks for Accurate Target Condition Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Classical neural networks face memory and processing limitations due to large weight matrices, leading to inadequate storage and prolonged training times, which hinder their ability to accurately determine target conditions or characteristics.
Innovation Solution
The use of tensor networks, specifically matrix product operators (MPOs) with tensors and activation functions, replaces traditional weight matrices, enabling more efficient storage, training, and processing, allowing for more complex neural network models and accurate determination of target conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If the number of hidden layers and neurons in classical neural networks is increased to handle complex data sets and achieve higher accuracy, then the neural network's capability to determine target characteristics improves, but the weight matrices become excessively large, causing memory storage limitations and prolonged training times
Solution Approach 1:
The patent segments the weight matrices into multiple tensors organized in tensor networks (such as matrix product operators). Instead of storing a single large weight matrix, the system divides it into smaller tensor components that can be stored and processed separately, reducing memory requirements while maintaining the network's representational capacity.
Solution Approach 2:
The patent transitions from classical two-dimensional weight matrices to multi-dimensional tensor structures. By adding dimensional complexity to the weight representation, the system can encode the same information more efficiently, reducing the total number of parameters needed while maintaining or improving model accuracy.
2Measurement precision
If the size of weight matrices is increased to model complex relationships in data, then the neural network's accuracy improves, but the training time becomes excessively long and computational resources are overwhelmed
Solution Approach 1:
By segmenting the training process into smaller tensor operations, the system can process updates more efficiently. Each tensor can be trained independently or in smaller batches, reducing the computational burden per iteration and allowing for faster convergence without sacrificing accuracy.
Solution Approach 2:
The patent changes the parameter representation from dense weight matrices to sparse or structured tensors. This parameter transformation allows for more efficient computation during training, reducing the number of operations required while maintaining the network's ability to learn complex patterns.
3Adaptability or versatility
If classical neural networks use large weight matrices to process data with many features, then the network can handle complex problems, but the computing devices cannot store or process the weight matrices efficiently
Solution Approach 1:
The patent segments the weight matrix into multiple smaller tensors that can be stored in available memory. This segmentation allows complex models to run on devices with limited memory resources, as the tensor components can be loaded and processed in manageable chunks rather than requiring the entire weight matrix to fit in memory simultaneously.
Solution Approach 2:
By transforming the weight structure from 2D matrices to multi-dimensional tensors, the patent creates a more compact representation that reduces the memory footprint. This dimensional transformation allows the same computational power to be achieved with fewer total parameters, making the system adaptable to devices with constrained resources.
Data Source
AI summary
A device or system configured to: receive a set of data associated with a monitored target, the set of data having N features, where N is a natural number greater than one; input the N features of the received set of data into a trained neural network for determining a condition or characteristic of the target with a plurality of sets of historical data associated with the target, each set of the plurality of sets of historical data having N features, the neural network at least having N inputs and one or more outputs, the neural network having one or more hidden layers, each hidden layer being a tensor network in the form of a matrix product operator, MPO, with a respective plurality of tensors and having a respective predetermined activation function per hidden layer or per tensor in the MPO; and input the N features into the neural network, determining a condition or characteristic of the target by processing the one or more outputs. Also, a device or system configured to train such neural network.