Memristor Neural Network Drop-Connect Control Unit
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Neural network devices face complexity issues due to high algorithm complexity, leading to over-fitting and increased operation time, which existing normalization and regularization methods fail to adequately address.
Innovation Solution
The implementation of a neural network device using drop-connect and dropout functions, specifically designed for memristor-based systems, where switches control the connection and dropout of signals across memory elements to normalize and regularize calculations, thereby reducing complexity and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If normalization and regularization are applied to reduce algorithm complexity, then over-fitting is reduced and operation time decreases, but device complexity increases due to additional control units and switches
Solution Approach 1:
The patent introduces a normalization control unit as an intermediary component that manages the complexity of normalization operations. This control unit coordinates multiple switches and memory elements to implement drop-connect and dropout functions, thereby reducing over-fitting while keeping the complexity management centralized and systematic rather than scattered throughout the neural network device.
2Manufacturing precision
If drop-connect and dropout functions are implemented using switches and memory elements, then calculation normalization is achieved, but manufacturing complexity increases
Solution Approach 1:
The patent segments the normalization function into discrete controllable units - individual switches connected to specific memory elements can be independently controlled to implement drop-connect. This segmentation allows precise control over which connections are dropped, achieving calculation normalization while enabling modular manufacturing where each switch-memory element pair can be fabricated and tested independently before assembly.
3Productivity
If multiple switches and control units are added to implement dropout and drop-connect, then operation time is reduced through better normalization, but the number of device components increases
Solution Approach 1:
The patent merges the dropout and drop-connect control functions into a single normalization control unit that manages both types of operations. By combining these functions under one control unit rather than having separate dedicated units for each function, the patent reduces the overall number of control components while still achieving the performance benefits of both dropout and drop-connect for reducing operation time and improving productivity.
Data Source
AI summary
A neural network device may include an input unit suitable for applying input signals to corresponding first lines, a calculating unit including memory elements cross-connected between the first lines and second lines, wherein the memory elements have respective weight values and generate product signals of input signals of corresponding first lines from among the plurality of first lines and weights to output the product signals to corresponding second lines from among the second lines, a drop-connect control unit including switches connected between the plurality of first lines and the plurality of memory elements, and suitable for randomly dropping a connection of an input signal applied to a corresponding memory element from among the plurality of memory elements, and an output unit connected to the plurality of second lines, and suitable for selectively activating signals of the plurality of second lines to apply the activated signals to the input unit and performing an output for the activated signals when the calculating unit performs generating of the product signals a set number of times.