Descriptor Synchronization for Neural Network Data Processing

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

As AI algorithms become more complex, the increasing amount of data and data dimensions processed by processors lead to reduced efficiency due to the need for processors to determine data addresses based on specified parameters in data-read instructions, making data access inefficient.

Innovation Solution

A data synchronization method using descriptor synchronization instructions that indicate the shape of tensor data, allowing processors to efficiently access and synchronize tensor data by transmitting only the necessary information, reducing unnecessary data transmission and processing complexity.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Ease of operation

If processors determine data address based on parameters specified in data-read instructions, then data access can be controlled, but processing efficiency decreases due to increased complexity

Engineering Contradiction:
Improvedata access controlVSAvoidprocessing efficiency
Core Design Contradiction:
Ease of operationVSProductivity

Solution Approach 1:

The patent applies preliminary action by pre-defining data access patterns and address calculation rules before actual data processing. The system pre-configures descriptor information that contains data shape, stride, and offset parameters, so that during runtime, processors can directly use these pre-calculated values without performing complex address determination operations, thus improving processing efficiency while maintaining data access control

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The patent introduces an intermediary mechanism in the form of a descriptor structure that mediates between the processor and the actual data. This descriptor contains pre-computed address information and data metadata, allowing the processor to access data through simplified instructions while the intermediary handles the complex address calculation and data layout management, thereby resolving the contradiction between controlled access and processing efficiency

Inventive Principle:
Principle #24Intermediary (Mediator)

2Adaptability or versatility

If the amount of data and data dimensions increase to handle complex AI algorithms, then algorithm capability improves, but synchronization overhead increases

Engineering Contradiction:
Improvealgorithm capabilityVSAvoidsynchronization overhead
Core Design Contradiction:
Adaptability or versatilityVSLoss of time

Solution Approach 1:

The patent applies segmentation by dividing the data synchronization process into independent descriptor-level operations. Instead of synchronizing entire large datasets, the system segments data into tensor elements with associated descriptors that can be synchronized independently. This allows processors to synchronize only the necessary descriptor information rather than all data, significantly reducing synchronization overhead while maintaining the ability to handle complex multi-dimensional AI algorithms

Inventive Principle:
Principle #1Segmentation

Solution Approach 2:

The patent uses copying by creating and transmitting lightweight descriptor copies between processors rather than copying actual data. The descriptor contains metadata about the data (shape, stride, offset) that can be replicated without the burden of the actual data payload. This copying approach allows complex AI algorithms to be supported across multiple processors while minimizing the time loss associated with data synchronization

Inventive Principle:
Principle #26Copying

Data Source

PatentUS11687339B2Data processing method and apparatus, and related product
Publication Date: 2023.06.27 CAMBRICON TECH CO LTD
  • US11687339B2 patent drawing
  • US11687339B2 patent drawing
  • US11687339B2 patent drawing

AI summary

The present disclosure provides a data processing method and an apparatus and a related product. The products include a control module including an instruction caching unit, an instruction processing unit, and a storage queue unit. The instruction caching unit is configured to store computation instructions associated with an artificial neural network operation; the instruction processing unit is configured to parse the computation instructions to obtain a plurality of operation instructions; and the storage queue unit is configured to store an instruction queue, where the instruction queue includes a plurality of operation instructions or computation instructions to be executed in the sequence of the queue. By utilizing the above-mentioned method, the present disclosure can improve the operation efficiency of related products when performing operations of a neural network model.