Runtime sharing of multiple neural network models in a computing system

By identifying and managing shared memory regions and private regions in the computing system, the problem of redundant storage of neural network models is solved, enabling efficient sharing of multiple models and optimized utilization of GPU resources, thereby improving the efficiency of the computing system.

CN122847696APending Publication Date: 2026-09-29INTERNATIONAL BUSINESS MACHINE CORPORATION
0 Cites 0 Cited by

Patent Information

Application Number
CN202580017476.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2024-02-27
Filing Date
2025-01-08
Publication Date
2026-09-29

AI Technical Summary

Technical Problem

In existing technologies, neural network models deployed on computing systems are each deployed in separate containers, resulting in redundant storage and waste of GPU resources, and a lack of effective resource sharing and isolation mechanisms.

Method used

By identifying physical memory devices in shared areas of the computing system, storing the common parts of the base model, and storing the difference files in private areas, runtime sharing of multiple neural network models is achieved, and logical memory pointers are used to manage the common and unique parts of the model.

Benefits of technology

It improves the efficiency of the computing system by reducing the storage copies of duplicate parameters, optimizing the utilization of GPU resources, and enabling efficient sharing and isolation of multiple neural network models.

✦ Generated by Eureka AI based on patent content.
Patent Text Reader

Abstract

Computer-implemented methods for runtime sharing of multiple neural network models in a computing system are provided. Aspects include receiving a request to store a first neural network model (NNM) in a computing system, obtaining parameters of the first NNM, and identifying a base model from a model database that corresponds to the first NNM. Aspects also include identifying duplicate parameters of the first NNM and the base model, and generating a difference file corresponding to non-duplicate parameters of the first NNM and the base model. Aspects further include identifying a physical memory device in the computing system that includes a shared region, the shared region including at least a portion of the base model, and loading parameters from the difference file in a private region of the physical memory device.
Need to check novelty before this filing date? Find Prior Art