The invention provides a
large model reasoning task
processing system and method and a medium, an event encapsulation module encapsulates reasoning task parameters into structured events through a preset event encapsulation rule and stores the structured events into a distributed
message queue, asynchronous decoupling of task requests and model instance
processing is achieved, and the efficiency of reasoning task
processing is improved. And the task concurrent response efficiency is improved. Secondly, the event processing module dynamically allocates matched model instances according to reasoning events in the
message queue, resources are called in real
time based on event loads, the allocation mode of whole card monopolization or fixed quota of a static
resource pool is changed, and the problems of resource fragmentation and scrambling are avoided. And finally, the resource prediction module analyzes historical
resource consumption data, event resource demand characteristics and
resource consumption data of the model instance, pre-judges the resource demand of the next stage in advance and formulates a
resource allocation strategy, so that the computing resources can be elastically expanded and contracted according to the actual demand, and the
utilization rate of the computing resources is effectively improved.