Multicore LLM Resource Allocation by Processing Phase
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing processing infrastructures for Large Language Models (LLMs) face inefficiencies in resource allocation and utilization, leading to performance bottlenecks, increased costs, and resource waste due to misallocation and underutilization.
Innovation Solution
A technique for improved resource adaptation in LLM processing platforms, utilizing LLM signatures to optimize power and resource distribution across various resources based on learning from processing phases, and implementing dynamic resource allocation to mitigate bottlenecks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If resources are allocated uniformly across all processing phases of LLM, then resource distribution is simple and stable, but resource utilization efficiency deteriorates due to misallocation and underutilization
Solution Approach 1:
The patent implements dynamic resource allocation that adapts to different processing phases of LLM execution. The system monitors processing phase transitions and dynamically adjusts resource distribution based on phase-specific requirements, transforming the static uniform allocation into a dynamic adaptive mechanism that resolves the contradiction between efficiency and complexity.
Solution Approach 2:
The system changes resource allocation parameters based on processing phase identification. By detecting phase transitions in LLM execution and adjusting resource distribution parameters accordingly, the system optimizes resource utilization for each phase while maintaining manageable complexity through structured parameter management.
2Speed
If more resources are allocated to processor cores, then LLM processing speed improves, but power consumption increases
Solution Approach 1:
The patent implements periodic resource allocation that matches the periodic nature of LLM processing phases. Resources are allocated intensively during computationally demanding phases and reduced during lighter phases, creating a periodic allocation pattern that maintains processing speed when needed while reducing overall power consumption.
Solution Approach 2:
The system applies partial resource allocation by providing full resources only to processor cores during specific processing phases that require intensive computation, while using reduced resources during other phases. This partial action approach optimizes the speed-power tradeoff by avoiding excessive resource allocation throughout the entire execution cycle.
3Loss of energy
If resources are reallocated dynamically based on processing phase, then resource utilization improves, but system complexity increases
Solution Approach 1:
The patent implements a feedback mechanism that monitors processing phase transitions and uses this information to guide resource allocation decisions. The system continuously observes execution progress, identifies phase boundaries, and feeds this information back to the resource manager, enabling intelligent resource distribution that reduces waste while maintaining acceptable system complexity through structured feedback loops.
Solution Approach 2:
The resource allocation system serves itself by automatically detecting processing phases and making allocation decisions without external intervention. The LLM execution environment provides self-service resource management by monitoring its own state and adjusting resource distribution autonomously, reducing the need for complex external control mechanisms while minimizing resource waste.
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
It is provided an apparatus comprising interface circuitry, machine-readable instructions, and processing circuitry to execute the machine-readable instructions. The machine-readable instructions comprise instructions to identify a processing flow pattern of a large language model, LLM, wherein the LLM is executed on a processor circuitry comprising a plurality of processor cores and wherein the processing flow pattern comprising a plurality of processing phases. The machine-readable instructions further comprise instructions to identify a processing phase of the LLM from the processing flow pattern. The machine-readable instructions further comprise instructions to allocate processing resources to the processor circuitry based on the identified processing phase of the LLM.