Model reasoning method and device, electronic equipment and computer readable storage medium
By receiving model service call requests and distinguishing online and offline model inference requests, calling pre-generated packages, online and offline model inference are realized, and the results are monitored, and the deployment of model inference service in the existing technology is solved. The deployment of model inference service is scattered and resource management is solved in the existing technology. Problems, improving management efficiency and controllability.
Patent Information
- Application Number
- CN202411034724.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-22
AI Technical Summary
The existing technology fails to effectively consider offline model service scenarios, and does not effectively monitor the characteristics and inference results of model inference, resulting in the problem of dispersed deployment of model inference services and the inability to manage resources in a unified manner.
Provide a model inference method and device, which distinguishes online and offline model inference requests by receiving model service call requests, calls pre-generated online and offline inference packages, realizes online and offline model inference, and monitors the inference results.
It realizes unified centralized management of all model inference services, solves the problem of dispersed deployment of model inference services and unavailable resources to be managed in a unified manner, and improves the efficiency and controllability of model inference.
Smart Images

Figure CN119005346A_ABST
Abstract
Citation Information
Cited By
Model training reasoning method and device, medium and equipment
CN121503690A
