Fourier Feature Crossing for Semantic Context in Deep Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current deep learning methods for processing information, such as texts and images, face high computational complexity and inefficiency in determining contextual semantic information, leading to low accuracy in semantic information processing.
Innovation Solution
The implementation of a fast Fourier transform-based feature crossing process for target vectors in an input sequence, followed by a feature perception process using an attention module to obtain a target sequence representing semantic information correlated to other objects, reduces computational complexity and improves processing efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If global information perception is performed to determine contextual semantic information, then the accuracy of semantic information is improved, but the computational complexity and processing time increase significantly
Solution Approach 1:
The patent segments the global information perception task into local feature extraction and feature crossing operations. Instead of processing all global information at once, the method divides the sequence into segments, extracts features locally, and then performs feature crossing between segments. This segmentation reduces the computational complexity from O(N²) to O(N) while maintaining the ability to capture contextual dependencies.
Solution Approach 2:
The patent introduces feature crossing as an intermediary mechanism between local feature extraction and global context understanding. The feature crossing operation acts as a mediator that combines features from different positions in the sequence, enabling global information perception through localized operations. This intermediary approach allows the model to capture long-range dependencies without the quadratic complexity of self-attention mechanisms.
2Measurement precision
If global information perception is performed to determine contextual semantic information, then the accuracy of semantic information is improved, but the processing efficiency decreases
Solution Approach 1:
The patent segments the global information perception task into local feature extraction and feature crossing operations. Instead of processing all global information at once, the method divides the sequence into segments, extracts features locally, and then performs feature crossing between segments. This segmentation reduces the computational complexity from O(N²) to O(N) while maintaining the ability to capture contextual dependencies.
Solution Approach 2:
The patent replaces the mechanical self-attention mechanism with a feature crossing operation based on fast Fourier transform. The traditional self-attention mechanism requires quadratic computational operations, while the feature crossing approach uses frequency domain transformations to achieve linear complexity. This substitution maintains the functional capability of capturing global dependencies while dramatically improving processing efficiency.
3Loss of information
If feature crossing is performed using traditional fully connected layers, then the relationship between target vectors is determined, but the computational complexity increases at square level
Solution Approach 1:
The patent replaces the mechanical fully connected layer approach with a feature crossing operation based on fast Fourier transform. The traditional fully connected approach requires quadratic computational operations to determine relationships between all pairs of vectors. The feature crossing method uses frequency domain transformations to achieve the same relationship determination with linear complexity, substituting the mechanical O(N²) system with an efficient O(N) system.
Solution Approach 2:
The patent changes the parameter space by transforming the feature crossing operation from the time domain to the frequency domain using fast Fourier transform. This parameter transformation allows the model to capture relationships between vectors more efficiently. By operating in the frequency domain, the model can determine vector relationships with linear computational complexity instead of quadratic complexity, while preserving the informational content.
Data Source
AI summary
An information processing method, an apparatus, an electronic device, a computer readable storage medium and a computer program product are provided. The method includes performing a fast Fourier transform-based feature crossing process on at least two target vectors in an input sequence of target information to obtain an output sequence of target information, and performing a feature perception process on the output sequence of the target information to obtain a target sequence of the target information, wherein the target sequence represents semantic information of each target object in the target information correlated to other target objects in the target information.