Computational Flow Graph Scheduling via Integer Linear Programming

The patent creates multiple copies of the computational flow graph (first copy, second copy, third copy) to handle different data batches in parallel. This copying approach allows the system to process multiple batches simultaneously without requiring expert manual fusion for each case, making the solution universally applicable to various model structures while maintaining high data transfer efficiency through automated scheduling.

The patent replaces manual expert experience-based vertex fusion with an automated scheduling system that uses integer linear programming to generate optimal execution schedules. This substitution of mechanical manual processes with automated computational methods enables the system to adapt to any model structure automatically, improving both productivity and versatility.

The patent segments the computational flow graph into multiple independent copies that can be processed in parallel across different devices. Each copy represents a separate data batch that can be executed simultaneously on different computing units, thereby improving computational parallelism while maintaining efficient resource utilization through the segmented structure.

The patent introduces a temporal dimension by creating multiple copies of the flow graph for different time steps (first copy for first batch, second copy for second batch, etc.). This allows computations to be performed in parallel across both spatial devices and temporal batches, significantly improving computational parallelism while managing resource consumption through coordinated scheduling.

The patent creates multiple copies of the computational flow graph to process different data batches in parallel, rather than copying vertices within a single graph. This approach reduces data transfer time by enabling simultaneous processing of multiple batches while avoiding the increased computational consumption that would result from duplicating vertices and their computations within a single model execution.

The patent segments the model execution into multiple independent batches, each represented by a separate flow graph copy. This segmentation allows the system to process batches in parallel without duplicating computational operations within each batch, thereby reducing data transfer time while maintaining efficient computational consumption through shared model parameters across batches.

The patent performs preliminary actions by pre-loading model parameters and frequently accessed data into high-speed cache memory before the actual inference computation begins. The scheduling system anticipates data access patterns and pre-fetches necessary data, thereby reducing the impact of limited bandwidth storage media during the critical computation phase and maintaining high data transmission effective speed.

The patent introduces high-speed cache memory as an intermediary between the limited bandwidth storage media and the computing units. This intermediary layer stores frequently accessed data and model parameters, allowing the computing units to access data at high speed without being constrained by the slow storage media bandwidth, thereby resolving the bottleneck while maintaining cost-effective hardware architecture.