The invention provides a multi-platform
large model dynamic fragmentation calculation method and
system based on computing power self-sensing, and relates to the technical field of computing power
resource scheduling. According to the calculation method, due to bandwidth normalization, temperature factors and a complete computing power self-sensing mechanism, stable execution performance can be kept, reasoning
delay jitter can be reduced, the computing power prediction capacity of mu ST can actively adjust fragments before the computing power is reduced, passive redeployment is avoided, and re-fragmentation and migration overhead can be reduced; the multi-
granularity fragmentation
system supports multiple structures, and the deployable range of the model can be effectively improved; a longer reasoning sequence and a higher number of concurrent users can be supported through state-level fragmentation, and the
throughput of the long sequence is improved; according to the composite search method combining differentiatable Bandit and GFlowNet, a global sub-optimal even optimal fragmentation strategy can be quickly found in a complex search space, the exploration time can be shortened, and the common
local optimum problem of a
heuristic method is avoided; the
system can be expanded to more than ten cloud-side-end devices for cooperation, and the expansion capability is remarkably improved.