Expanded Convolutional Training Layers for Low-Cost Inference

The patent applies dynamics by making the network architecture adaptive between training and inference phases. During training, the network dynamically expands convolutional layers into expanded training layers with K×K and 1×1 filters to improve learning capacity and accuracy. During inference, it dynamically collapses these expanded layers back into standard convolutional layers, maintaining the original computational cost. This dynamic transformation allows the system to have different effective architectures for different operational phases, resolving the contradiction between training accuracy and inference efficiency.

The patent utilizes parameter changes by modifying the network structure parameters during training (expanding layers with additional filters and layers) and then transforming these parameters back to the original configuration for inference. The expanded training layers learn optimal parameters that are then projected back onto the standard convolutional layer parameters, enabling the model to achieve higher accuracy while maintaining the same inference computational footprint through parameter transformation rather than structural permanence.

The approach enables dynamic architectural transformation where the computational load is temporarily increased during training through expanded layers, then reduced back to original levels during inference. This dynamic adjustment allows the system to benefit from complex architecture capabilities during learning while maintaining energy efficiency during deployment, effectively decoupling training complexity requirements from inference energy consumption.

The patent applies preliminary action by performing the computationally intensive expanded layer transformations during the training phase, where high computational resources are available. The learned parameters from these expanded structures are then compressed and stored for efficient inference. This preliminary processing allows the system to extract maximum learning benefit from complex architectures upfront, then use lightweight structures for energy-constrained inference operations.

The system dynamically adjusts its architectural complexity based on operational phase. During training, it temporarily adopts expanded layers with higher device requirements to achieve better performance. During inference, it collapses back to the original architecture that can run on lower-performance processors. This dynamic adaptation resolves the contradiction by making device requirements temporary and phase-dependent rather than permanent.

The expanded training layers act as temporary, disposable computational structures used only during the training phase. These complex layers are created, utilized for learning, and then discarded in favor of the simpler original architecture for inference. This approach allows the system to temporarily access high-performance computational capabilities when needed for training, then revert to affordable, low-performance processor compatibility for deployment.