Convolutional Neural Network Normalization via Filter Matrix Integration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for operating artificial neural networks, particularly convolutional neural networks, require computationally intensive normalization steps during training and inference, which can be time-consuming and demand powerful hardware, limiting efficiency and scalability.
Innovation Solution
The method integrates normalization operations into the convolution and shift operations of the neural network's filter matrices, allowing for combined normalization and feature transformation in a single step, eliminating the need for separate normalization processes and reducing computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If separate normalization operations are performed before convolution operations in neural networks, then data standardization and feature highlighting are improved, but computational complexity and processing time increase significantly
Solution Approach 1:
The patent combines separate normalization operations (standardization, normalization) with convolution operations into a single integrated operation. The normalization parameters (mean, standard deviation) are incorporated directly into the convolution filter, allowing both normalization and feature extraction to occur simultaneously in one computational step, thereby reducing overall computational complexity while maintaining data standardization benefits
Solution Approach 2:
The convolution operation is enhanced to perform multiple functions simultaneously: it executes feature extraction through traditional convolution while also performing normalization operations using integrated normalization parameters. This multi-functional approach eliminates the need for separate normalization steps, reducing computational overhead without sacrificing the precision benefits of data standardization
2Measurement precision
If separate normalization operations are performed before convolution operations in neural networks, then feature highlighting is improved, but processing time increases significantly
Solution Approach 1:
The patent merges normalization operations with convolution operations into a single integrated computational step. By incorporating normalization parameters (mean, standard deviation) directly into the convolution filter, the system performs both normalization and feature extraction simultaneously, eliminating the sequential execution of separate operations and thereby reducing total processing time while maintaining feature highlighting capabilities
3Measurement precision
If powerful hardware is used to perform normalization operations, then normalization accuracy is improved, but hardware requirements and system cost increase
Solution Approach 1:
The patent integrates normalization operations into the convolution layer, allowing normalization to be performed using the same computational resources already allocated for convolution. This eliminates the need for dedicated high-power hardware specifically for normalization, as the normalization computation is absorbed into the existing convolution operation's hardware requirements, thereby maintaining normalization accuracy without increasing hardware specifications
Solution Approach 2:
The convolution operation is designed to perform both traditional feature extraction and normalization functions simultaneously. This multi-functionality allows the existing convolution hardware to handle normalization tasks without requiring additional or more powerful dedicated normalization hardware, thus maintaining normalization accuracy while avoiding increased hardware requirements and system cost
Data Source
Figure 1~2
Figure 3~4
AI summary
The invention relates to a method for operating an artificial neural network (10) which comprises at least one convolution layer (12a-c), which is configured to transform an input matrix (I) of the convolution layer (12a-c) into an output matrix (A) on the basis of a convolution operation and a shift operation. The method comprises determining at least one first and one second normalization quantity on the basis of entries in the input matrix (I) and/or on the basis of a training data set, determining a modified filter matrix (/) on the basis of an original filter matrix (f) and on the basis of at least one of the first normalization quantity and the second normalization quantity, and determining a modified shift matrix (5) on the basis of an original shift matrix (b) and on the basis of at least one of the first normalization quantity and the second normalization quantity. The method further comprises transforming the input matrix (I) into the output matrix (A) by applying the modified filter matrix (/) and the modified shift matrix (5). In this manner, it is possible advantageously to combine a normalization operation with the convolution operation and the shift operation.