AI Chip Extended Address Processing to Reduce Cross-Node Delay
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing collective communication methods in multi-chip cluster parallel computing involve inefficient data transmission and processing, leading to increased delay and memory overheads due to multiple data copies and synchronization steps.
Innovation Solution
The artificial intelligence chip employs an extended address space and processor to dynamically determine operations based on address information, combining data reception and processing into a single step, thereby reducing the number of required steps and optimizing performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If data is transmitted through system memory and NIC in multiple steps, then data can be shared between nodes, but transmission delay increases and communication efficiency deteriorates
Solution Approach 1:
The patent segments the address space into multiple regions, where each region can be independently mapped to different physical memory locations or remote nodes. This allows the system to directly access specific data segments without going through the entire system memory path, reducing transmission delay while maintaining data sharing capability.
Solution Approach 2:
The patent introduces an address translation mechanism as an intermediary that maps virtual addresses to physical addresses or remote node addresses. This intermediary enables direct memory access between nodes by translating addresses on-the-fly, eliminating the need for multiple copy steps through system memory and NIC.
2Ease of operation
If data is copied multiple times between chip memory and system memory, then data can be accessed by different components, but memory overhead and energy consumption increase
Solution Approach 1:
The patent uses address translation to create virtual copies of memory addresses rather than physical data copies. The address translation mechanism allows multiple components to access the same physical data through different virtual addresses, eliminating the need for actual data copying while maintaining accessibility.
Solution Approach 2:
The address translation mechanism serves multiple functions simultaneously: it enables data accessibility for different components, reduces memory overhead by avoiding duplicate storage, and lowers energy consumption by eliminating repeated data transfer operations.
3Reliability
If four-step communication process is used for data transmission, then data can be transferred between nodes, but communication complexity and synchronization overhead increase
Solution Approach 1:
The patent merges the address translation and data transmission operations into a single unified process. By integrating the address translation mechanism with the memory access path, the system can perform address translation and data transfer simultaneously, reducing the four-step process to fewer operations while maintaining reliable data transmission.
Solution Approach 2:
The patent performs address translation in advance or concurrently with data access requests, rather than as separate sequential steps. This preliminary action prepares the address mapping before data transmission begins, eliminating the need for multiple sequential steps and reducing communication complexity.
4Adaptability or versatility
If extended address space is implemented to support more operations, then operational versatility improves, but address space management complexity increases
Solution Approach 1:
The patent segments the extended address space into multiple regions, each with specific functions or mappings. This segmentation allows the system to support diverse operations by directing different address regions to appropriate targets, achieving versatility while managing complexity through organized regional allocation.
Solution Approach 2:
The patent implements dynamic address translation that can adaptively map address regions based on operational requirements. This dynamic approach allows the system to support multiple operations with a single flexible address space management mechanism, achieving versatility without proportional increases in complexity.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A data operation method and an artificial intelligence chip (100_1, 200) including a chip memory (140, 240), a computing processor (130, 230), a base address register (110, 210), and an extended address processor (120, 220) are provided. The base address register access an extended address space in the chip memory. The extended address processor receives a command carrying first data and address information. The extended address processor determines an operation mode of the first data according to the address information. When the address information points to a first section of the extended address space, the extended address processor performs a first operation on the first data. When the address information points to a section other than the first section of the extended address space, the extended address processor notifies the computing processor of the operation mode and the computing processor performs a second operation on the first data.