The invention discloses a financial scene-oriented model reasoning method and device, equipment and a storage medium, and relates to the field of model reasoning. According to the extracted
semantic information, the model structure and the
system bandwidth resources, determining a
delay estimated value for reasoning the current task; performing
score evaluation on the current task based on the task priority information and the
delay estimated value, and updating the task
queue and the KV cache sharing degree of the
queue in combination with the SLO information; based on the KV cache sharing degree and the
queue task number, the task queues form task batches, and the task batches are sent into a GPU
video memory for model calculation; and when the GPU computing resources are occupied by the low-priority tasks in the task batch, acquiring SLO information of not executing the high-priority tasks, and calculating the
preemption state of executing the high-priority tasks through a profit-cost quantification mechanism. According to the method and the device, the task
delay, the priority and the KV cache sharing degree are dynamically evaluated, and the benefit-cost quantitative
preemption mechanism is combined, so that the method and the device have the advantages of improving the high-priority task response speed and the
overall efficiency of the
system.