Integrated CReLU and Convolution Operation for CNN Efficiency
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional CNNs with CReLU operations incur high computational loads due to separate execution of CReLU and convolution operations, leading to increased detection complexity without a corresponding increase in computational efficiency.
Innovation Solution
The method integrates CReLU and convolution operations by selecting appropriate parameters based on input value ranges, using a comparator, selector, and multiplier to apply piecewise linear continuous functions, reducing unnecessary operations and enhancing detection accuracy without increasing computational load.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If CReLU and convolution operations are performed separately in conventional CNNs, then detection accuracy can be improved through complex operations, but computational load increases significantly
Solution Approach 1:
The patent merges the CReLU activation operation and convolution operation into a single integrated operation. By combining these two separate operations that were traditionally executed sequentially into one unified computational step, the patent reduces the total number of operations required while maintaining the functional benefits of both CReLU's detection accuracy improvement and convolution's feature extraction capability.
2Measurement precision
If multiple separate layers (bias layers, scale layer, activation layers, concatenation layer) are used to achieve complex detection, then detection accuracy improves, but device complexity increases
Solution Approach 1:
The patent integrates multiple separate functional layers (bias layers, scale layer, activation layers, and concatenation layer) into a single unified operation block. This merging reduces the number of discrete layers and components in the network architecture while preserving the cumulative effect of all these operations, thereby simplifying the device structure without sacrificing detection accuracy.
Solution Approach 2:
The integrated operation block performs multiple functions simultaneously: it applies bias parameters, scales values, applies activation functions, and performs concatenation all within a single computational unit. This multi-functionality allows the system to achieve the same complex detection capabilities as multiple separate layers would provide, but with a more compact and less complex architectural design.
Data Source
AI summary
A method of learning parameters of a CNN is provided. The method includes steps of: (a) allowing an input value to be delivered to individual multiple element bias layers; (b) allowing the scale layer connected to a specific element bias layer to multiply a predetermined scale value by an output value of the specific element bias layer; (c) (i) allowing a specific element activation layer connected to the scale layer to apply activation function, and (ii) allowing the other individual element activation layers to apply activation functions to output values of the individual element bias layers; (d) allowing a concatenation layer to concatenate an output value of the specific element activation layer and output values of the other element activation layers; (e) allowing the convolutional layer to apply the convolution operation to the concatenated output; and (f) allowing a loss layer to acquire a loss during a backpropagation process.