Gated Linear Networks for Online Learning

The patent segments the learning process by introducing side gates that independently control weight selection for different input contexts. Each side gate operates autonomously to modulate connections between neurons based on side information, enabling localized learning decisions without requiring full-network backpropagation. This segmentation allows online learning to proceed with reduced computational overhead by dividing the global weight update problem into independent local decisions.

The patent implements dynamic weight selection through side gates that adaptively modulate connection strengths based on incoming side information. Rather than using fixed weights or requiring extensive retraining, the side gates dynamically adjust which weights are active for each prediction, enabling the network to adapt to new data streams in real-time with minimal processing power. This dynamic mechanism allows the network to learn online by selectively activating appropriate weight sets based on contextual side information.

The patent introduces side gates as intermediary components between the input layer and hidden layers, and between hidden layers and the output layer. These side gates act as mediators that filter and modulate information flow based on side information, enabling the network to disambiguate sub-optimal or correlated examples. By inserting this intermediary control mechanism, the network can achieve more reliable convergence because the side gates prevent erroneous weight updates that would otherwise occur when processing ambiguous or correlated training data.

The patent changes the parameter selection mechanism by using side gates to dynamically select which weight parameters are active for each prediction based on side information. Rather than relying on fixed parameter initialization or extensive training to converge from sub-optimal data, the side gates enable the network to switch between different parameter sets (weights) depending on the side information context. This parameter switching capability allows the network to achieve accurate predictions and reliable convergence even when trained on sub-optimal or correlated data by selecting appropriate parameters for each input context.

The patent segments the learning process into independent local decisions at each neuron, controlled by side gates. Instead of requiring global backpropagation that processes the entire network sequentially, each neuron's side gate independently determines weight selection based on local side information and local prediction errors. This segmentation of the learning decision into independent neuronal units enables parallel computation across the network, as each neuron can update its weights independently without waiting for gradients to propagate through the entire network.

The patent enables each neuron to perform self-service learning through its side gate, which autonomously selects weights based on local side information and local prediction performance. Rather than relying on centralized backpropagation to coordinate learning across the network, each neuron independently adjusts its own weights through its side gate mechanism. This self-service capability allows neurons to learn in parallel and distributed fashion, as each unit autonomously improves its own predictions without requiring synchronized global updates.

The patent extracts the complex weight update computation from conventional backpropagation and replaces it with simpler side gate mechanisms that select pre-defined weight sets based on side information. By taking out the intensive gradient computation and weight optimization steps, the network retains adaptive learning capability while reducing computational requirements to levels suitable for resource-constrained devices. The side gates use simple conditional logic rather than intensive mathematical optimization, enabling deployment on mobile and embedded systems.

The patent employs computationally inexpensive side gate operations that can be executed rapidly and discarded after each prediction, replacing expensive iterative optimization routines. Each side gate performs a single lightweight operation to select appropriate weights based on side information, rather than executing costly gradient descent iterations. This use of cheap, single-step computational objects enables the network to achieve adaptive learning on resource-constrained devices where computational capacity is limited and energy is precious.