Neural Network Receptive Field Non-Linear Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
State-of-the-art artificial neural networks (ANNs) with linear receptive fields struggle to accurately predict image appearance and are prone to adversarial attacks, failing to emulate human perception abilities, especially under noise or texture changes, which limits their effectiveness in computer vision and other structured data processing tasks.
Innovation Solution
A computer-implemented method using a neural network with a novel receptive field formulation that combines input values non-linearly, defined by the expression INRF(x)=∑yi∈N mi(u(yi)) - λ∑yi∈N ωiσ(u(yi)) - ∑yj∈Nk(x)g(yj-x)u(yj), where mi, ωi, and g are kernel weights, and σ is a non-linear function, allowing for adaptive processing of structured data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If linear receptive fields are used in ANNs, then the model structure is simple and computationally efficient, but the predictive power and robustness against noise and adversarial attacks deteriorate
Solution Approach 1:
The patent transforms the receptive field from a linear combination to a non-linear combination by introducing a non-linear function σ that operates on the weighted sum of inputs. This parameter change in the combination operation enables the model to capture complex patterns while maintaining computational efficiency, resolving the contradiction between structural simplicity and predictive robustness.
Solution Approach 2:
The receptive field combines multiple kernel functions (m, ω, g) with different characteristics to form a composite non-linear operation. This composite structure integrates the strengths of different kernel types while mitigating their individual limitations, achieving both computational efficiency and enhanced robustness against noise and adversarial attacks.
2Productivity
If linear receptive fields are used, then computation is efficient, but performance under noise and texture changes deteriorates
Solution Approach 1:
By changing the combination operation from linear to non-linear through the application of function σ, the model achieves better performance under noise and texture changes while preserving computational efficiency. The non-linearity allows the model to adapt to variations in input data without requiring excessive computational resources.
3Ease of manufacture
If linear receptive fields are used, then the model is easier to implement, but vulnerability to adversarial attacks increases
Solution Approach 1:
The introduction of non-linearity through function σ in the receptive field operation increases resistance to adversarial attacks while maintaining ease of implementation. The non-linear transformation makes it more difficult for adversarial examples to manipulate the model's predictions, yet the overall model structure remains straightforward to implement using standard computational operations.
Data Source
AI summary
The present invention is related to a computer implemented method for processing structured data, wherein the method is based on an artificial neural network at least comprising a neural unit with a receptive field that combines the input values in a non-linear manner.The method is a specific machine learning method wherein the structured data may be for instance sound streams or images. The method, according to specific embodiments may be applied to multi-channel structured data.