Logical Data Shuffling via Circuit Switching

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

In parallel computing systems, data shuffling among processing nodes is inefficient due to the need for physical movement of nodes, which presents electrical and mechanical challenges and creates inefficiencies, especially in communication-limited environments like database operations and map-reduce processes.

Innovation Solution

A method for logical data shuffling that involves arranging processing nodes in a coordinated communication topology, allowing for data redistribution without physical movement, by grouping nodes into pairs for bi-directional data exchange, and using circuit switching for logical node movement, enabling data to be re-distributed within the topology.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If physical movement of processing nodes is used for data shuffling, then data can be redistributed among nodes, but electrical and mechanical challenges arise and efficiency decreases

Engineering Contradiction:

Improvedata shuffling efficiencyVSAvoidelectrical and mechanical complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent replaces the mechanical system of physically moving processing nodes with an electrical/electronic system that logically rearranges node connections through circuit switching. The nodes remain physically stationary while their logical positions and data routing are dynamically changed through electrical control signals, eliminating mechanical complexity while maintaining data redistribution capability.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The patent creates logical copies of node positions and connection topologies through software-controlled circuit switching configurations. Instead of moving physical nodes, the system creates virtual representations of node arrangements and data routing paths, allowing multiple logical configurations to be implemented using the same physical hardware infrastructure.

Inventive Principle:

Principle #26Copying

2Productivity

If processing nodes are physically moved for data exchange, then data shuffling can be performed, but communication efficiency is reduced

Engineering Contradiction:

Improvedata exchange efficiencyVSAvoidtime for physical movement

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The patent pre-arranges circuit switching configurations and logical node pairings before data exchange operations begin. By preparing the logical topology and routing paths in advance through software control, the system eliminates the time required for physical node movement and reconfiguration, allowing data exchange to proceed immediately along pre-established logical paths.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements dynamic logical reconfiguration of node connections through circuit switching that can be changed rapidly in response to data exchange requirements. The logical topology becomes a dynamic parameter that can be adjusted electronically without physical movement, allowing the system to adapt connection patterns instantly to optimize data shuffling efficiency.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS10152337B2Logical data shuffling

Publication Date: 2018.12.11 INTERNATIONAL BUSINESS MACHINE CORPORATION

AI summary

Embodiments relate to data shuffling by logically rotating processing nodes. The nodes are logically arranged in a two or three dimensional matrix. Every time two of the nodes in adjacent rows of the matrix are aligned, adjacent nodes exchange data. The positional alignment is a logical alignment of the nodes. The nodes are logically arranged and rotated, and data is exchanged in response to the logical rotation.