Collective Logic Device for High Bandwidth Floating Point Addition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In parallel computing systems, floating point number additions across multiple computing nodes require significant software overhead and fail to fully utilize high network bandwidth due to the need for processor involvement in reduction operations.
Innovation Solution
Implementing a hardware logic device within the parallel computing system that converts floating point numbers to integers, performs summation, and converts back to floating point numbers in a single pass, utilizing a collective logic device with a front-end for conversion, an ALU for addition, and a back-end for normalization, thereby reducing overhead and maximizing network bandwidth.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If software is used to perform floating point number additions across multiple computing nodes, then the addition operation can be completed, but significant software overhead is incurred and network bandwidth is not fully utilized
Solution Approach 1:
The patent replaces software-based floating point addition with a hardware logic device that performs the same function. The hardware device includes floating point to integer converters, an integer adder tree, and normalization logic, substituting the software processing mechanism with a dedicated hardware circuit that operates in parallel and does not consume processor cycles or software overhead.
Solution Approach 2:
The patent introduces an intermediary hardware logic device that sits between the network and the computing nodes. This device receives floating point numbers from the network, converts them to integers, performs addition, and converts the result back to floating point format. This intermediary handles the computation without involving the computing node processors, thus eliminating software overhead while maximizing network bandwidth utilization.
2Productivity
If processors are involved in reduction operations for floating point additions, then accurate results can be obtained, but the high network bandwidth cannot be fully utilized due to processor bottlenecks
Solution Approach 1:
The patent extracts the floating point addition functionality from the processor and places it in a separate hardware logic device. By taking out this specific function from the general-purpose processor, the system can utilize the full network bandwidth without the processor becoming a bottleneck. The hardware device operates independently and in parallel with the network data flow.
Solution Approach 2:
The patent segments the floating point addition operation into distinct hardware components: floating point to integer converters, an integer adder tree, and normalization logic. This segmentation allows each component to operate independently and in parallel, eliminating the sequential processing bottleneck that would occur if a single processor handled the entire operation.
3Productivity
If floating point numbers are processed in multiple passes through the collective logic device, then conversion and addition can be performed, but the number of network hops increases and bandwidth efficiency decreases
Solution Approach 1:
The patent merges the floating point to integer conversion, integer addition, and floating point normalization operations into a single pass through the hardware logic device. Instead of requiring separate network hops for each operation, all transformations and computations are performed in one continuous data flow through the hardware circuit, maximizing bandwidth efficiency.
Solution Approach 2:
The patent ensures continuous useful action by processing floating point numbers through the entire conversion-addition-normalization pipeline in a single uninterrupted pass. The hardware logic device maintains continuous data flow without requiring the data to be sent back to the network or computing nodes between operations, thus eliminating idle time and maximizing bandwidth utilization.
Data Source
AI summary
To add floating point numbers in a parallel computing system, a collective logic device receives the floating point numbers from computing nodes. The collective logic devices converts the floating point numbers to integer numbers. The collective logic device adds the integer numbers and generating a summation of the integer numbers. The collective logic device converts the summation to a floating point number. The collective logic device performs the receiving, the converting the floating point numbers, the adding, the generating and the converting the summation in one pass. One pass indicates that the computing nodes send inputs only once to the collective logic device and receive outputs only once from the collective logic device.