1results about How to "Implement multiplexing" patented technology

This application relates to a multi-DNN inference scheduling method and system for in-memory computing-accelerated edge servers. The method involves receiving inference requests; reading the model parameters of a target DNN model from a cached DNN model file based on the inference request; and generating scheduling parameters based on these parameters. The scheduling parameters include the number of copies of the target DNN model, the batch size for processing the target DNN model per run, and a model packaging strategy for splitting the target DNN model. Based on the scheduling parameters, the inference request is constructed into a task. A PIM accelerator is invoked to process the task according to a runtime function pre-compiled for the target DNN model, generating the corresponding inference result. This application overcomes the limitation of current PIM accelerators that only optimize single DNN models, enabling concurrent scheduling of multiple types of DNN inference tasks to meet the diverse needs of edge IoT devices.