Event Neural Networks With Delta Updates for Efficient Video Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing computer vision systems prioritize accuracy over efficiency, leading to high computational costs and power consumption, especially in real-time applications with significant camera motion, as they fail to leverage temporal persistence in video data.
Innovation Solution
Implementing event neural networks (EvNets) that utilize event neurons with state variables for long-term memory, applying transmission policies to reduce computations by transmitting only when significant changes occur, and using delta-based updates to minimize redundant processing across multiple layers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional neural networks are used to prioritize accuracy in computer vision tasks, then measurement precision is improved, but use of energy and computational cost increase significantly
Solution Approach 1:
The patent segments the neural network processing into discrete event-based computations, where only changed pixel values trigger neuron activations. This segmentation allows the system to process video data incrementally rather than computing full frames, significantly reducing energy consumption while maintaining accuracy for detecting changes.
Solution Approach 2:
The system implements periodic transmission of neuron outputs based on change thresholds rather than continuous transmission. Neurons only transmit their delta values when significant changes occur, creating a periodic action pattern that reduces computational overhead and power usage compared to continuous processing of all video frames.
2Measurement precision
If conventional neural networks process every video frame completely, then measurement precision is improved, but productivity decreases due to redundant computations
Solution Approach 1:
The patent extracts and processes only the delta values (changes) from video frames rather than processing entire frames. By taking out only the changed portions and transmitting them through the neural network, the system maintains accuracy for change detection while dramatically improving processing speed by eliminating redundant computations of unchanged regions.
Solution Approach 2:
The system performs partial action by processing only the necessary portions of video data that contain changes. Instead of fully processing every frame, the neural network receives and processes only the delta inputs that exceed certain thresholds, achieving sufficient accuracy for the task while improving productivity through selective processing.
3Use of energy by moving object
If delta-based updates are used to reduce redundant processing, then use of energy decreases, but device complexity increases due to state variable management
Solution Approach 1:
The neural network neurons perform self-service by automatically maintaining their own state variables (a, b, d) and computing their own delta outputs. Each neuron independently tracks its activation history and determines when transmission is necessary, reducing the need for external control mechanisms and managing complexity through decentralized self-management.
Solution Approach 2:
The system manages complexity by changing the parameter representation from full frame pixel values to delta values. By transforming the data parameter from absolute intensities to relative changes, and by using state variables to track neuron activations, the system reduces energy consumption while the parameter transformation itself manages the complexity through mathematical relationships.
4Productivity
If transmission policies are applied to reduce computations, then productivity is improved, but measurement precision may deteriorate due to information loss
Solution Approach 1:
The transmission policy applies local quality by using different transmission thresholds for different neurons and different regions of the video data. Instead of a uniform threshold, the system allows neurons processing different types of changes to have different transmission criteria, maintaining precision for important changes while improving productivity by filtering less significant variations.
Solution Approach 2:
The system uses feedback mechanisms where neurons monitor their own state variables and the transmitted delta values to determine when transmission is necessary. This feedback loop ensures that transmission policies maintain accuracy by transmitting information when changes exceed thresholds while reducing unnecessary transmissions, balancing productivity and precision through adaptive decision-making.
Data Source
AI summary
In accordance with some embodiments, systems, methods, and media for generating and using neural networks having improved efficiency for analyzing video are provided. In some embodiments, the method comprises: providing image data to a trained neural network; receiving, at a neuron, a delta-based input Δin from a previous layer; generating an output g(Δin) of a linear transform g; generating an updated state variable a based on g(Δin) and a current a; generating an output ƒ(a) of an activation function ƒ based on updated a; generating an updated state variable d based on a current d, a state variable b, and ƒ(a); generating an updated b based on output ƒ(a); transmitting d to a next layer based on a transmission policy and subtracting the value from d; and receiving an output from the trained neural network that represents a prediction based on the image data.