A deep network model storage system

Abstract

This invention provides a deep neural network model storage system, relating to the field of deep neural networks. It consists of four modules: a user module, a model file analysis module, a model storage update module, and a fine-grained model storage module. The user module receives system submissions of deep neural network models from users, determines the layer structure information and newly added layers of the submitted model through the model analysis module and the model storage module, and stores the model file in the fine-grained model storage module, thus realizing the storage and retrieval of the neural network model. The beneficial effects of this invention are: it combines the advantages of cloud model file repositories and distributed file storage, improves the storage resource utilization efficiency of the storage system through storage scheduling and fine-grained neural network model file storage methods, and enhances the user's service experience.

Description

Technical Field

[0001] This invention relates to the field of deep neural networks, and more particularly to a deep neural network model storage system. Background Technology

[0002] With the advent of the digital and intelligent era, the development and breakthroughs in artificial intelligence technology have provided new methodologies for meeting the demands of intelligent applications. Among these, deep learning technology, especially deep neural network technology, has become a key driving force. Deep learning is an important branch of machine learning. By constructing deep neural network models, it can automatically extract features and patterns from data, achieving efficient data processing and decision-making. Deep neural network technology has already achieved remarkable success in many fields, such as image recognition, natural language processing, and intelligent recommendation systems. Thanks to its powerful data processing and analysis capabilities, deep learning methods have significantly improved data utilization efficiency while reducing the difficulty of analyzing massive amounts of data. Therefore, deep learning technology not only possesses broad application potential and exploration space but also greatly accelerates the progress of the information digitization industry.

[0003] However, to provide deep learning inference tasks, it is first necessary to pull the required deep learning model files from a remote model repository via a remote object storage system (such as Amazon S3 object storage service). Current model file retrieval solutions primarily use the complete model required for different tasks as the unit of transmission. For example, well-known open deep learning model repositories like Hugging Face and Model Zoo only provide the parameter files of the complete model through their model file retrieval interfaces. Research shows that the parameter file size of mainstream deep learning neural network models, such as MobileNet, InceptionResNet, and SE-ResNet, ranges from 15MB to 500MB. Faced with the current scenario of large-scale inference task requests, the amount of data stored in model files increases dramatically with the increase in the types of inference services, significantly increasing the storage cost of model files in a cloud computing environment. Simultaneously, when facing massive user requests, data transmission typically consumes a large amount of bandwidth, placing a huge load on the core network and causing high latency, thus reducing the user experience.

[0004] To alleviate the storage and transmission pressure on models, the industry commonly employs model compression to reduce model size and improve transmission efficiency. While such solutions improve the performance of deep learning inference tasks to some extent, they do not completely solve the difficulties inherent in deep learning inference. First, in the current mobile digital era, most deep learning applications need to be performed at the edge or on mobile devices, while existing optimization solutions are typically designed for data center environments. Unlike data center environments, network transmission capabilities are limited in edge computing environments comprised of edge devices and mobile devices, and centralized cloud storage service solutions cannot effectively address the high latency caused by model file transmission. Second, research on deep neural network models used in various open-source projects reveals a significant amount of redundant structures in the deep learning models used for various types of inference tasks. Therefore, for different businesses, the backbone network models used contain a large number of reusable redundant weight tensors. Summary of the Invention

[0005] To address the problem of low storage resource utilization efficiency in storage systems, this invention provides a deep neural network model storage system, which consists of four modules: a user module, a model file analysis module, a model storage update module, and a fine-grained model storage module.

[0006] The user module receives the user's neural network model pull request and the system submission of the user's neural network model, including the model file, model name and model version;

[0007] The model file analysis module is used to analyze the specific layer structure information of the model file and extract the tensor feature information of each neural network layer.

[0008] The model storage update module is used to determine the new layers in the model file;

[0009] The fine-grained model storage module is used for the main storage and retrieval functions of the model storage system;

[0010] The user module receives system submissions of deep neural network models from users. Through the model analysis module and model storage module, it determines the layer structure information and newly added layers of the submitted model, and stores the model file in the fine-grained model storage module, thus realizing the storage of neural network models and making them available for retrieval.

[0011] Furthermore, the user module implements system submission of user models through the API.

[0012] Furthermore, the neural network model layer structure includes layer types, model topology, and model layer tensor parameters.

[0013] Furthermore, the model file analysis module obtains the specific layer structure information within the model file by printing the model structure information, and uses the MD5 information feature extraction algorithm to obtain the tensor feature information of each neural network layer.

[0014] Furthermore, the specific process by which the model storage update module confirms the additional storage needs of model files is as follows: the neural network layer clusters within the model files already stored in the system are perceived through the global model storage index and the global layer file storage index. The tensor features of each layer of the user-submitted model are compared with the layer file indexes in the storage system using the global model hash index and the global fine-grained layer tensor index. Neural network layers that do not exist in the current storage system are added as new layers, and additional storage is added to them. Otherwise, the information of the new layers is marked as empty. The global model storage index and the global layer file storage index use the global model hash index and the global fine-grained layer tensor index, respectively.

[0015] Furthermore, the fine-grained model storage module is divided into four sub-modules: model request analysis sub-module, storage decision sub-module, model storage sub-module, and service proxy sub-module.

[0016] The request analysis submodule is used to record and analyze model file retrieval requests from the model storage system and predict the retrieval request rate for each model in the next time period.

[0017] The storage decision submodule collects the current resource status and model file storage information of each service node in the storage system, and combines the prediction results of the request analysis submodule to adjust the real-time storage location of model files on the distributed storage system, thereby achieving optimal file location deployment within a specific time period according to real-time service requirements.

[0018] The model storage submodule is responsible for fine-grained storage of each model file in units of deep neural network model layer tensors, and for updating and maintaining the model file index and neural network layer structure index in the distributed storage system to ensure that the storage location of the model is traceable.

[0019] The service proxy submodule is responsible for reconstructing the required model files in the distributed cluster at the layer level when a model retrieval request is received, and for selecting the model file transfer proxy node based on the location of the received model retrieval request.

[0020] Furthermore, the steps for storing the deep neural network model in the system include:

[0021] S11. Users submit deep neural network models to the storage system through the user module, and cache them in the storage service node closest to the user.

[0022] S12. Call the model analysis module to process the submitted model and obtain the specific layer structure information of the submitted model file and the tensor feature information of each neural network layer.

[0023] S13. The model storage update module determines the new layer based on the tensor feature information of each neural network layer and adds extra storage requirements for it.

[0024] S14. The fine-grained model storage module completes model storage based on the specific layer structure information of the submitted model file and the tensor feature information of the newly added layer.

[0025] Furthermore, the fine-grained model storage module first obtains the model fetch records of the storage system within the past time window, and then uses a linear regression method to predict the call frequency of each model on different storage nodes in the next time window. Subsequently, based on the prediction results, it schedules and updates the positions of tensor files of each layer in the storage system, and updates the global layer file hash index. The specific steps are as follows:

[0026] S141. The model storage submodule deletes the submitted model in the storage service node in step S11 above and temporarily stores the tensor feature information of the newly added layer.

[0027] S142. The model storage submodule uploads the specific layer structure information of the submitted model file to the global model storage index, and adds the index of the new layer to the global layer file storage index according to the tensor feature information of the new layer.

[0028] S143. Set a time window, request the analysis submodule to read the model retrieval records in the previous time window of the storage system, and obtain the prediction results of the call frequency of each model in the storage system on different storage nodes in the next time window through the linear regression method.

[0029] S144. The storage decision submodule reads the node resource status and model file storage information in the storage system, and combines the call frequency prediction results to use the random forest algorithm to obtain the adjustment decision of the real-time storage location of the model file.

[0030] S145. The model storage submodule stores each model file in the system according to the adjustment decision, synchronously updates the global model index and the global layer file storage index, and synchronously indexes them to all nodes in the system to complete the storage of the neural network model.

[0031] Furthermore, the steps for the system to retrieve the deep neural network model include:

[0032] S21. When the service proxy submodule receives the model retrieval request, it reads the global model storage index and the global layer file storage index to determine the storage location information of the model to be retrieved and the layer files it uses.

[0033] S22. The service proxy submodule reconstructs the required model files in the distributed cluster and transmits the reconstructed model to the puller according to the model pull request location.

[0034] The beneficial effects of the technical solution provided by this invention are as follows: This invention provides a deep neural network model storage system, which consists of four modules, including a user module, a model file analysis module, a model storage update module, and a fine-grained model storage module. The user module receives the system submission of the user's deep neural network model, determines the layer structure information and newly added layers of the submitted model through the model analysis module and the model storage module, and stores the model file in the fine-grained model storage module, realizing the storage of the neural network model and making it available for retrieval. It combines the advantages of cloud model file warehouse and distributed file storage, and improves the storage resource utilization efficiency of the storage system and enhances the user's service experience through storage scheduling and fine-grained neural network model file storage. Attached Figure Description

[0035] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0036] Figure 1 This is a schematic diagram of a deep neural network storage system according to an embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the operation flow of a deep neural network storage system according to an embodiment of the present invention. Detailed Implementation

[0038] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0039] An embodiment of the present invention provides a deep neural network storage system.

[0040] Please refer to Figure 1 , Figure 1 This is a schematic diagram of a deep neural network storage system according to an embodiment of the present invention. It consists of four modules, including a user module, a model file analysis module, a model storage update module, and a fine-grained model storage module.

[0041] Please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating the operation flow of a deep neural network storage system according to an embodiment of the present invention. It includes two processes: storage and retrieval.

[0042] The storage procedure for deep neural networks is as follows:

[0043] The first step is for the user to submit a deep neural network model to the storage system through the user module, which is then cached on the storage service node closest to the user.

[0044] The user module implements system submission of user models through the API, including model files, model names, and model versions.

[0045] The second step is to call the model analysis module to process the submitted model and obtain the specific layer structure information of the submitted model file as well as the tensor feature information of each neural network layer.

[0046] The model file analysis module obtains specific layer structure information within the model file by printing model structure information, and uses the MD5 information feature extraction algorithm to obtain tensor feature information of each neural network layer; the neural network model layer structure includes layer type, model topology, and model layer tensor parameters.

[0047] The third step involves the model storage update module determining the new layers based on the tensor feature information of each neural network layer, and adding additional storage for them.

[0048] The system perceives the neural network layer clusters within the model files already stored in the system by using a global model storage index and a global layer file storage index. It compares the tensor features of each layer of the user-submitted model with the layer file index in the storage system, and adds neural network layers that do not exist in the current storage system as new layers, adding extra storage for them; otherwise, the information of the new layers is marked as empty. This invention uses a global model hash index and a global fine-grained layer tensor index for the global model storage index and the global layer file storage index, respectively.

[0049] The fourth step involves the fine-grained model storage module storing the model based on the specific layer structure information of the submitted model file and the tensor feature information of the newly added layers. Specifically:

[0050] First, it should be noted that the fine-grained model storage module is divided into four sub-modules: model request analysis sub-module, storage decision sub-module, model storage sub-module, and service proxy sub-module.

[0051] Step 1: The model storage submodule deletes the submitted model from the storage service node in step S11 above and temporarily stores the tensor feature information of the newly added layer.

[0052] Step 2: The model storage submodule uploads the specific layer structure information of the submitted model file to the global model storage index, and adds the index of the new layer to the global layer file storage index according to the tensor feature information of the new layer.

[0053] Step 3: Set a time window, request the analysis submodule to read the model retrieval records in the previous time window of the storage system, and use the linear regression method to obtain the prediction results of the call frequency of each model in the storage system on different storage nodes in the next time window;

[0054] Step 4: The storage decision submodule reads the node resource status and model file storage information in the storage system, and combines the call frequency prediction results to use the random forest algorithm to obtain the adjustment decision of the real-time storage location of the model file.

[0055] Step 5: The model storage submodule stores each model file in the system according to the adjustment decision, synchronously updates the global model storage index and the global layer file storage index, and synchronously indexes them on all nodes in the system to complete the storage of the neural network model.

[0056] The deep neural network model is stored using a distributed storage system architecture, with fine-grained storage of model files based on the layer structure of the model.

[0057] The retrieval process is as follows:

[0058] The first step is that the service proxy submodule receives the model retrieval request, reads the global model storage index and the global layer file storage index, and determines the storage location information of the model to be retrieved and the layer files it uses.

[0059] The second step is for the service proxy submodule to reconstruct the required model files in the distributed cluster and transmit the reconstructed model to the puller according to the model pull request location.

[0060] The beneficial effects of this invention are as follows: This invention provides a deep neural network model storage system, consisting of four modules: a user module, a model file analysis module, a model storage update module, and a fine-grained model storage module. The user module receives system submissions of deep neural network models from users, determines the layer structure information and newly added layers of the submitted model through the model analysis module and the model storage module, and stores the model file in the fine-grained model storage module, realizing the storage and retrieval of neural network models. It combines the advantages of cloud model file warehouses and distributed file storage, and improves the storage resource utilization efficiency of the storage system and enhances the user's service experience through storage scheduling and fine-grained neural network model file storage methods.

[0061] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A deep neural network model storage system, characterized in that, It consists of four modules, including a user module, a model file analysis module, a model storage and update module, and a fine-grained model storage module; The user module receives the user's neural network model pull request and the system submission of the user's neural network model, including the model file, model name and model version; The model file analysis module is used to analyze the specific layer structure information of the model file and extract the tensor feature information of each neural network layer. The model storage update module is used to determine the new layers in the model file and update the storage index based on the tensor feature information of each neural network layer. The fine-grained model storage module is used for the main storage and retrieval functions of the model storage system; The user module receives system submissions of deep neural network models from users. Through the model analysis module and model storage module, it determines the layer structure information and newly added layers of the submitted model, and stores the model file in the fine-grained model storage module, thus realizing the storage of neural network models and making them available for retrieval.

2. The deep neural network model storage system as described in claim 1, characterized in that, The user module implements system submission of user models through API.

3. The deep neural network model storage system as described in claim 2, characterized in that, The layer structure of a neural network model includes the types of layers, the model topology, and the tensor parameters of the model layers.

4. The deep neural network model storage system as described in claim 3, characterized in that, The model file analysis module obtains the specific layer structure information within the model file by printing the model structure information, and uses the MD5 information feature extraction algorithm to obtain the tensor feature information of each neural network layer.

5. A deep neural network model storage system as described in claim 4, characterized in that, The specific process by which the model storage update module confirms the additional storage needs of model files is as follows: by using the global model storage index and the global layer file storage index to perceive the neural network layer clusters within the model files already stored in the system, the tensor features of each layer of the user-submitted model are compared with the layer file indexes in the storage system. Neural network layers that do not currently exist in the storage system are added as new layers, and additional storage is added for them; otherwise, the information of the new layers is marked as empty. The global model storage index and the global layer file storage index respectively adopt the global model hash index and the global fine-grained layer tensor index.

6. A deep neural network model storage system as described in claim 5, characterized in that, The fine-grained model storage module is divided into four sub-modules: model request analysis sub-module, storage decision sub-module, model storage sub-module, and service proxy sub-module.

7. A deep neural network model storage system as described in claim 6, characterized in that, The steps for storing a deep neural network model in the system include: S11. Users submit deep neural network models to the storage system through the user module, and cache them in the storage service node closest to the user. S12. Call the model analysis module to process the submitted model and obtain the specific layer structure information of the submitted model file and the tensor feature information of each neural network layer. S13. The model storage update module determines the new layer based on the tensor feature information of each neural network layer and adds extra storage requirements for it. S14. The fine-grained model storage module completes model storage based on the specific layer structure information of the submitted model file and the tensor feature information of the newly added layer.

8. A deep neural network model storage system as described in claim 7, characterized in that, Step S14 is as follows: S141. The model storage submodule deletes the submitted model in the storage service node in step S11 above and temporarily stores the tensor feature information of the newly added layer. S142. The model storage submodule uploads the specific layer structure information of the submitted model file to the global model storage index, and adds the index of the new layer to the global layer file storage index according to the tensor feature information of the new layer. S143. Set a time window, request the analysis submodule to read the model retrieval records in the previous time window of the storage system, and obtain the prediction results of the call frequency of each model in the storage system on different storage nodes in the next time window through the linear regression method. S144. The storage decision submodule reads the node resource status and model file storage information in the storage system, and combines the call frequency prediction results to use the random forest algorithm to obtain the adjustment decision of the real-time storage location of the model file. S145. The model storage submodule stores each model file in the system according to the adjustment decision, synchronously updates the global model storage index and the global layer file storage index, and synchronously indexes them to all nodes in the system to complete the storage of the neural network model.

9. A deep neural network model storage system as described in claim 8, characterized in that, The deep neural network model is stored using a distributed storage system architecture, which stores the model files in a fine-grained manner, with the layer structure of the model as the granularity.

10. A deep neural network model storage system as described in claim 9, characterized in that, The steps for the system to retrieve a deep neural network model include: S21. When the service proxy submodule receives the model retrieval request, it reads the global model storage index and the global layer file storage index to determine the storage location information of the model to be retrieved and the layer files it uses. S22. The service proxy submodule reconstructs the required model files in the distributed cluster and transmits the reconstructed model to the puller according to the model pull request location.

Images