Neural Network Forward Computation Using GPU Texture Storage
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing forward computation of neural networks in GPUs is inefficient due to the linear storage of buffers in internal memory, leading to prolonged data reading and storage times.
Innovation Solution
Implementing a method that uses texture storage structures, such as RGBA-channel 3D or 2D texture storage, to store input and weight data separately, allowing the GPU to access and process data more efficiently by using texture coordinates as indexes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If buffer structure is used for data storage in GPU, then data can be stored in internal memory, but the linear storage structure causes prolonged data reading and storage time
Solution Approach 1:
The patent transforms the linear storage structure into a two-dimensional texture storage structure by introducing texture coordinates (u, v) as indexing mechanisms. Instead of accessing data sequentially through linear buffers, the GPU can access data elements through 2D texture coordinates, enabling parallel access patterns and significantly reducing data retrieval time during forward propagation computations.
2Ease of manufacture
If linear buffer storage is used, then implementation is simple, but GPU data access efficiency is low
Solution Approach 1:
The patent changes the storage parameters from linear buffer indexing to 2D texture coordinate indexing. By mapping data elements to texture coordinates and utilizing the texture storage structure's inherent 2D organization, the system achieves faster GPU data access while maintaining implementation feasibility through standard OpenGL texture operations.
3Speed
If texture storage structure is used, then data access speed increases, but storage structure complexity increases
Solution Approach 1:
The patent leverages the universal texture storage structure that serves multiple functions: it provides fast 2D data access for forward propagation, supports various data layouts through texture coordinate mapping, and can be implemented using standard OpenGL APIs. This multi-functionality reduces the need for specialized storage structures while achieving high data access speeds.
Data Source
AI summary
A method and an apparatus for forward computation of a neural network are provided. The method includes: obtaining input data and weight data of at least one data processing layer; storing the input data by using a first texture storage structure to obtain first texture data; storing the weight data by using a second texture storage structure to obtain second texture data; and performing data processing of the at least one data processing layer based on the first texture data and the second texture data, to obtain output data of the at least one data processing layer. For the at least one data processing layer in the neural network, the input data and the weight data are respectively stored in corresponding texture storage structures.