Neural Network Parameter Genome Connectome Construction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep neural networks lack the complexity and functionality of biological neural networks, particularly in processing spatial and temporal data, storing information, and utilizing feedback loops, limiting their problem-solving capabilities compared to biological brains.
Innovation Solution
A method for forming neural networks using a parameter genome to characterize connections between neurons, enabling the construction of spiking neural networks with feedback architecture, allowing for efficient genetic algorithms to evolve networks for specific tasks by generating probability maps and expanding them into large connectomes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional deep neural networks are used with static connection weights and backpropagation, then the network can be trained on large datasets, but the network lacks the complexity and functionality of biological neural networks, particularly in processing spatial and temporal data and utilizing feedback loops
Solution Approach 1:
The patent implements feedback loops in the neural network architecture, allowing outputs to be fed back into the network for iterative processing. This enables the network to process temporal data and maintain state information, mimicking biological neural network behavior while enhancing adaptability for sequential and spatial-temporal tasks
Solution Approach 2:
The patent introduces dynamic connection weights that can change over time and adapt during processing, rather than static weights. This allows the network to adjust its connectivity patterns based on input data characteristics, improving its ability to handle varying spatial and temporal patterns in the data
2Adaptability or versatility
If recurrent neural networks are used to work with time-series data, then the network can process sequential information, but the network still just fits data to functions and remains limited in depth and breadth of problems it can solve
Solution Approach 1:
The patent employs a hierarchical network structure where multiple layers of processing units are nested within each other, with each layer handling different levels of abstraction. This nested architecture enables the network to solve complex problems by breaking them down into smaller sub-problems at different hierarchical levels, increasing both depth and breadth of problem-solving capability
Solution Approach 2:
The patent divides the neural network into distinct functional modules and processing stages, each specialized for specific tasks. This segmentation allows the network to handle diverse problem types through dedicated modules while maintaining overall system coherence, thereby expanding the breadth of solvable problems without proportionally increasing overall complexity
3Productivity
If spiking neural networks are used to achieve more brain-like operation with spatial-temporal signals, then the network can do more advanced computing, but the network is challenging to work with and train to do specific tasks
Solution Approach 1:
The patent introduces an intermediary training layer or conversion mechanism that translates between traditional backpropagation-friendly representations and spiking neural network operations. This intermediary allows the network to be trained using well-established methods while still achieving brain-like spiking behavior during actual computation, thereby maintaining ease of training while gaining advanced computing capabilities
Data Source
AI summary
A method of forming a neural network includes specifying layers of neural network neurons. A parameter genome is defined with numerical parameters characterizing connections between neural network neurons in the layers of neural network neurons, where the connections are defined from a neuron in a current layer to neurons in a set of adjacent layers, and where the parameter genome has a unique representation characterized by kilobytes of numerical parameters. Parameter genomes are combined into a connectome characterizing all connections between all neural network neurons in the connectome, where the connectome has in excess of millions of neural network neurons and billions of connections between the neural network neurons.