On-Device Generative AI Task Routing Under Resource Constraints

The patent segments AI workload into two parts: lightweight tasks executed locally on client devices and complex tasks executed on server devices. This segmentation allows the system to leverage client device resources for simple operations while reserving server resources for computationally intensive tasks, thereby reducing overall infrastructure energy consumption while maintaining necessary computational power.

The patent introduces an intermediary mechanism that determines which tasks should be executed locally versus on the server. This intermediary decision layer optimizes resource allocation by analyzing task characteristics and routing them appropriately, reducing unnecessary server infrastructure energy consumption while ensuring complex tasks still receive adequate computational power.

The patent segments task execution based on data sensitivity and computational requirements. Sensitive tasks that require data privacy are executed locally on client devices using lightweight models, while less sensitive but computationally intensive tasks are sent to servers. This segmentation maintains data privacy for sensitive operations while overcoming local device computational limitations for complex tasks.

The patent applies local quality by executing only the necessary computational operations locally on client devices rather than requiring full-power models everywhere. Lightweight models are deployed on client devices specifically for tasks requiring data privacy, while full-power models remain on servers for tasks where data privacy is less critical, optimizing the balance between privacy and computational power.

The patent changes the parameter of model size and complexity based on device capabilities. Instead of deploying full-power models on all devices regardless of specifications, the system adapts model parameters (size, complexity, architecture) to match the computational resources available on each device, improving both accessibility and efficiency simultaneously.

The patent segments the AI model execution strategy based on device type and task requirements. Different model variants are deployed on different device tiers, with lightweight models on consumer devices and full-power models on powerful devices or servers. This segmentation eliminates resource conflicts by matching appropriate models to appropriate platforms while maintaining broad accessibility.