Dynamic Engine Pipeline Depth for Latency-Sensitive Work Queues

Solution Overview

Problem

Existing methods for improving pipeline depth in computer processors are inefficient, leading to inefficiencies in execution latency.

Innovation Solution

Implement a method and apparatus for dynamically adjusting pipeline depth to optimize performance, utilizing a pipeline configuration to improve execution latency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If deep pipelining is used for small descriptors, then IOPS is improved, but execution latency increases for large descriptors

Engineering Contradiction:

ImproveIOPSVSAvoidexecution latency

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The pipeline depth is made dynamically configurable rather than fixed, allowing the system to adjust the number of pipeline stages based on workload characteristics. The engine configuration circuitry receives latency and throughput requirements as inputs and dynamically configures the pipeline depth accordingly, enabling the system to optimize for either IOPS or latency depending on the specific workload demands.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the structural parameter of pipeline depth based on workload requirements. By configuring different numbers of pipeline stages (e.g., fewer stages for latency-sensitive workloads, more stages for throughput-oriented workloads), the system adapts its processing characteristics to match the specific performance needs of different descriptor sizes and workload types.

Inventive Principle:

Principle #35Parameter changes

2Productivity

If deep pipelining is used to handle multiple descriptors, then throughput is improved, but waiting time increases for individual descriptors

Engineering Contradiction:

ImprovethroughputVSAvoidwaiting time

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The pipeline depth is made dynamically configurable rather than fixed, allowing the system to adjust the number of pipeline stages based on workload characteristics. The engine configuration circuitry receives latency and throughput requirements as inputs and dynamically configures the pipeline depth accordingly, enabling the system to optimize for either IOPS or latency depending on the specific workload demands.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the structural parameter of pipeline depth based on workload requirements. By configuring different numbers of pipeline stages (e.g., fewer stages for latency-sensitive workloads, more stages for throughput-oriented workloads), the system adapts its processing characteristics to match the specific performance needs of different descriptor sizes and workload types.

Inventive Principle:

Principle #35Parameter changes

3Device complexity

If fixed pipeline depth is used, then device complexity is reduced, but adaptability to different workload requirements deteriorates

Engineering Contradiction:

Improvepipeline configurationVSAvoidworkload adaptability

Core Design Contradiction:

Device complexityVSAdaptability or versatility

Solution Approach 1:

The pipeline depth is made dynamically configurable rather than fixed, allowing the system to adjust the number of pipeline stages based on workload characteristics. The engine configuration circuitry receives latency and throughput requirements as inputs and dynamically configures the pipeline depth accordingly, enabling the system to optimize for either IOPS or latency depending on the specific workload demands.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The engine pipeline is designed to serve multiple workload types and performance requirements through a single configurable structure. By making the pipeline depth adjustable, the same hardware infrastructure can be optimized for different scenarios including latency-sensitive workloads, throughput-oriented workloads, and various descriptor sizes, eliminating the need for multiple fixed pipelines.

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12632290B2Method and apparatus for dynamically adjusting pipeline depth to improve execution latency

Publication Date: 2026.05.19 INTEL CORP

AI summary

Apparatus and method for managing pipeline depth of a data processing device. For example, one embodiment of an accelerator comprises: an interface to receive a plurality of work requests from a plurality of clients; and a plurality of engines to perform the plurality of work requests; wherein the work requests are to be dispatched to the plurality of engines from work queues, the work queues to store a work descriptor per work request, each work descriptor to include information needed to perform a corresponding work request, wherein the work queues include a first and a second work queue to store work descriptors associated with first latency characteristics and second latency characteristics, respectively; engine configuration circuitry to configure a first engine to have a first pipeline depth based on the first latency characteristics and to configure a second engine to have a second pipeline depth based on the second latency characteristics.