Adapter Card CNN Processing With Overlapped Memory and Convolution
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep learning architectures, such as VGG16 neural networks, face limitations in achieving high throughput in terms of frames per second due to bottlenecks in processing operations, particularly in convolution, pooling, and classification, resulting in low frame rates of about 500 to 1000 fps.
Innovation Solution
Implementing image processing through an artificial neural network on an adapter card within a host-computing system, utilizing multi-core processing and partitioned memory fragments to overlap data transfer and convolution operations, enabling simultaneous processing of image data, filter-weights, and instructions, and generating image feature maps for efficient image classification.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional deep learning architectures (VGG16) are used for image processing, then image classification functionality is achieved, but throughput is limited to 500-1000 fps due to bottlenecks in convolution, pooling, and classification operations
Solution Approach 1:
The memory system is divided into multiple partitions (first partition for image data, second partition for filter weights, third partition for instructions, fourth partition for output data), with each partition further segmented into fragments. This segmentation enables parallel access and processing, allowing multiple cores to simultaneously process different fragments of image data, filter weights, and instructions, thereby increasing throughput to approximately 2000 fps while managing complexity through structured organization.
Solution Approach 2:
The patent introduces a new dimension of parallelism by utilizing multi-core processing architecture where multiple processor cores operate simultaneously on different memory fragments. This dimensional expansion from single-core sequential processing to multi-core parallel processing enables the system to achieve doubled throughput (2000 fps) by distributing convolution, pooling, and classification operations across multiple cores.
2Productivity
If data transfer and computation are performed sequentially, then system simplicity is maintained, but processing throughput is limited due to idle time during data transfer
Solution Approach 1:
The system performs preliminary actions by pre-loading filter weights into the second partition of memory and organizing image data into the first partition before processing begins. This preliminary organization and pre-positioning of data and weights enables immediate parallel processing without sequential waiting, allowing computation to overlap with data transfer operations and thereby increasing throughput while maintaining operational simplicity through structured memory layout.
Solution Approach 2:
The patent implements continuity of useful action by enabling overlapping of data transfer and computation operations through multi-core processing. While one core transfers data between memory partitions, other cores simultaneously perform convolution and pooling operations on available data. This continuous utilization of processing resources eliminates idle time and maintains high throughput of approximately 2000 fps without complicating system operation.
3Productivity
If multi-core processing with partitioned memory is implemented, then throughput increases to 2000 fps, but memory management and data transfer complexity increases
Solution Approach 1:
The memory is segmented into four distinct partitions, each dedicated to specific data types: first partition for input image data, second partition for filter weights, third partition for processing instructions, and fourth partition for output feature maps and classified data. Each partition is further divided into fragments that can be independently accessed by different processor cores. This segmentation reduces memory management complexity by providing clear separation of concerns and enabling parallel access patterns.
Solution Approach 2:
The patent introduces intermediary buffer regions within each memory partition that facilitate efficient data transfer between producer and consumer cores. These intermediary buffers act as mediators, allowing one core to produce data while another core consumes it without direct synchronization overhead. This intermediary mechanism simplifies multi-core coordination while maintaining high throughput of approximately 2000 fps.
Data Source
AI summary
The present subject matter refers a method of image processing through an artificial neural network implemented in an adapter card in a host-computing system. The method comprises receiving image data from a host processor into a first fragment of first partition of a memory and simultaneously subjecting the captured features of the image data to a convolution procedure within a second fragment of the first partition through multi-core processing; receiving filter-weights from an on-board random access memory (RAM) into a first fragment of second partition of the memory via a storage controller and simultaneously subjecting the received filter weights to said convolution procure through a second fragment of the second partition through multi-core processing; receiving processing instruction from host into a first fragment of a third partition of the memory and simultaneously subjecting the received instructions to assist said convolution procure through a second fragment of the third partition through multi-core processing; generating an image feature map based on said convolution procedure to cause labelling of the received image data; and storing the labelled images into a first fragment of a fourth partition of the memory and simultaneously transferring the labelled images through a second fragment of the fourth partition back to the host.