Method and system for uniformly managing AI models based on distributed file system

A technology of distributed files and management modules, which is applied in the field of unified management of AI models based on distributed file systems, can solve the problems of model prediction value deviation, cumbersome conversion process, and increase the difficulty of deployment, so as to achieve the effect of optimizing update and convenient use
CN110765077AActive Publication Date: 2020-02-07中电福富信息科技有限公司 +1
6 Cites 5 Cited by

Patent Information

Authority / Receiving Office
CN · China
Current Assignee / Owner
中电福富信息科技有限公司
Publication Date
2020-02-07

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention discloses a method and system for unified management of AI models based on a distributed file system. Based on the distributed file system, a model iteration management module is additionally arranged to extract preset model file information; wherein the model file information comprises information such as a model name, a model version, model creation time, whether a model is onlinepublic and whether the model is dirty, an AI model record is newly added in the metadata table, and the model is stored in a model warehouse according to a preset model storage path to construct an AImodel management system formed by combining the metadata table and the model warehouse. The newly-added model reading module analyzes data input by a user, extracts model information, matches the model information with records in a metadata table, extracts metadata items in the table, checks whether a model is online or not, and if the model is online, extracts complete target model information including a dirty model state from nodes of the distributed file system according to metadata and returns the complete target model information to the user. A user can use the model in real time and can also optimize the model and upload the model again to facilitate optimization and updating of the model.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of artificial intelligence, in particular to a method and system for unified management of AI models based on a distributed file system. Background technique

[0002] Currently, there are various methods applied to generate predictive models. Data scientists and engineers can choose from a variety of languages ​​to build AI predictive models. For example, use the Python language to call the scikit-learn framework to build a prediction model, use Java or Scala language to call the Spark MLlib framework to build a prediction model, and so on. Numerous construction methods result in specialized models for various environments. Recently, with the widespread use of deep learning, frameworks such as TensorFlow and Pytorch support publishing machine learning models through REST or gRPC, but traditional machine learning frameworks, such as scikit-learn, gensim, xgboost, etc., do not yet support it. Also, tensorfl...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More