Model reasoning method and device, electronic equipment and computer readable storage medium

By receiving model service call requests and distinguishing online and offline model inference requests, calling pre-generated packages, online and offline model inference are realized, and the results are monitored, and the deployment of model inference service in the existing technology is solved. The deployment of model inference service is scattered and resource management is solved in the existing technology. Problems, improving management efficiency and controllability.

CN119005346APending Publication Date: 2024-11-22CHINA CITIC BANK CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411034724.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing technology fails to effectively consider offline model service scenarios, and does not effectively monitor the characteristics and inference results of model inference, resulting in the problem of dispersed deployment of model inference services and the inability to manage resources in a unified manner.

Method used

Provide a model inference method and device, which distinguishes online and offline model inference requests by receiving model service call requests, calls pre-generated online and offline inference packages, realizes online and offline model inference, and monitors the inference results.

Benefits of technology

It realizes unified centralized management of all model inference services, solves the problem of dispersed deployment of model inference services and unavailable resources to be managed in a unified manner, and improves the efficiency and controllability of model inference.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119005346A_ABST
    Figure CN119005346A_ABST
Patent Text Reader

Abstract

The invention provides a model reasoning method and device, electronic equipment and a computer readable storage medium, and relates to the technical field of artificial intelligence, in particular to the technical fields of model reasoning, big data and the like. According to the specific implementation scheme, a model service calling request is received, wherein the model service calling request is used for calling model reasoning; the model reasoning comprises online model reasoning and offline model reasoning; and under the condition that the model service calling request is calling online model reasoning, calling a pre-generated online reasoning program package to complete online model reasoning. And under the condition that the model service calling request is calling off-line model reasoning, calling a pre-generated off-line reasoning program package to complete off-line model reasoning.
Need to check novelty before this filing date? Find Prior Art

Citation Information

Cited By

  • Model training reasoning method and device, medium and equipment

    CN121503690A