Neural Network Execution Unit With Rmask Vector Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Implementing neural network techniques like Drop out and Drop connect efficiently using known computer technology is non-trivial and can impact performance.
Innovation Solution
An execution unit for a processor that implements Drop out or Drop connect using a single instruction, specifically the rmask instruction, which randomly masks values in a vector to efficiently apply these techniques.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If Drop out or Drop connect techniques are implemented using known computer technology (CPU/GPU), then neural network performance can be improved by reducing overfitting, but the implementation becomes non-trivial and impacts processing performance
Solution Approach 1:
The patent extracts the Drop out and Drop connect functionality from complex CPU/GPU implementations and implements it directly in hardware within the neural network processing unit. The masking operation is separated as a distinct hardware component that can be applied independently to activation values or weights, simplifying the overall implementation while maintaining the performance benefits of reducing overfitting.
Solution Approach 2:
The patent introduces a masking operation as an intermediary step between the forward propagation of activation values and the computation of weighted sums. This masking layer acts as a mediator that randomly zeroes out selected values based on dropout rates, enabling Drop out and Drop connect techniques without requiring complex software implementations on CPUs or GPUs.
2Reliability
If Drop out or Drop connect are implemented using standard processing units, then overfitting can be reduced, but processing speed and efficiency deteriorate
Solution Approach 1:
The patent replaces the mechanical/software-based Drop out implementation on CPUs or GPUs with a dedicated hardware masking operation within the neural network processing unit. This hardware substitution performs the random zeroing of activation values or weights in parallel with the main computation pipeline, maintaining processing speed while achieving overfitting reduction through efficient bitwise operations rather than sequential software processing.
3Reliability
If complex implementations of Drop out or Drop connect are used, then neural network performance can be improved, but the device complexity and instruction overhead increase
Solution Approach 1:
The patent merges the Drop out and Drop connect functionality into the core neural network processing unit, combining the masking operation with the existing computation pipeline for processing activation values and weights. This integration eliminates the need for separate instruction sequences to handle dropout, reducing instruction overhead while maintaining the performance improvements from reduced overfitting.
Solution Approach 2:
The patent creates a universal masking operation that can function in multiple modes: masking activation values for Drop out, masking weights for Drop connect, or neither for standard processing. This multi-functional masking unit is controlled by configuration parameters that specify the dropout rate and target (activation or weight), eliminating the need for separate hardware or instruction paths for different dropout variants.
Data Source
Figure 1
Figure 1A
Figure 2~3
AI summary
The present invention relates to an execution unit for executing a computer program comprising a sequence of instructions, which include a masking instruction. The execution unit is configured to execute the masking instruction which when executed by the execution unit masks randomly selected values from a source operand of n values and retains other original values from the source operand to generate a result which includes original values from the source operand and the masked values in their respective original locations.