AI model cross-platform deployment system and method based on uniform interface and intermediate presentation layer

Abstract

The invention provides an AI model cross-platform deployment system and method based on a unified interface and an intermediate presentation layer, and is used for solving the technical problem that the deployment adaptation of an AI model among different cloud platforms is difficult. The system adopts a hierarchical architecture design and comprises a unified interface layer, a middle presentation layer and a platform adaptation layer. Wherein the unified interface layer provides a standardized model deployment interface specification; the middle presentation layer converts the AI model into a platform-independent middle format and manages model dependence and configuration information; and the platform adaptation layer is responsible for converting the intermediate format into a deployment format of a target platform and calling a corresponding platform API to complete deployment. Through the innovative design of the unified interface and the middle presentation layer, the development cost of cross-platform deployment is remarkably reduced, the portability of the model is improved, and the technical problem in the cross-platform deployment process of the model is effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an AI model cross-platform deployment system based on a unified interface and an intermediate representation layer and a deployment method thereof. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, AI models are increasingly widely used in various fields. In order to fully realize the value of AI models, it is necessary to efficiently deploy them from the development environment to various cloud computing platforms for practical application. AI model cross-platform deployment, as a key link connecting model development and practical application, plays an important role in the industrialization process of artificial intelligence technology.

[0003] Currently, there are two technical solutions for AI model cross-platform deployment: one is to develop a special deployment adaptation module for each target platform, and deploy the model to the corresponding platform by writing platform-specific deployment scripts and configuration files; the other is to use the proprietary tool chain provided by cloud service providers. These tool chains usually integrate model conversion, resource configuration and deployment management functions, and can directly deploy the model to the corresponding cloud platform.

[0004] However, the above technical solutions have obvious shortcomings: the solution of developing an adaptation module for each platform not only has a large amount of development work, but also is difficult to maintain and extend, and when a new cloud platform needs to be supported, a complete adaptation module needs to be redeveloped; and the solution of using cloud service provider's proprietary tool chain is likely to cause the technology stack to be locked to a specific platform, making it difficult to achieve flexible migration of models between different cloud platforms, and increasing development and operation and maintenance costs.

[0005] To solve the above problems, the industry has tried to develop a unified model deployment framework. Such frameworks attempt to mask platform differences through abstract interfaces, but due to the lack of an effective intermediate representation mechanism, they often result in overly complex frameworks and lack flexibility when handling platform-specific functions. Although these attempts have improved the cross-platform deployment problem to some extent, they have not fundamentally solved the technical pain points in the model deployment process.

[0006] Therefore, there is an urgent need for a technical solution that can effectively solve the problem of AI model cross-platform deployment, ensuring standardization and normalization of the deployment process, taking into account the characteristics of different platforms, and ensuring good scalability and maintainability of the solution. This has become a key problem that needs to be broken through in the field of artificial intelligence technology. SUMMARY

[0007] The purpose of the present application is to solve the technical problems of high development cost, poor portability and platform lock-in existing in the cross-platform deployment process of existing AI models. In addition, the present application also aims to provide a standardized deployment solution with strong scalability and low maintenance cost.

[0008] To achieve the above-mentioned purpose, the present application provides an AI model cross-platform deployment system and method based on a unified interface and an intermediate representation layer. The system adopts a hierarchical architecture design, including a unified interface layer, an intermediate representation layer and a platform adaptation layer, and realizes cross-platform deployment of models through standardized interface specifications and platform-independent intermediate representations.

[0009] Specifically, the unified interface layer provides standardized model deployment interface specifications, including model import interfaces, configuration management interfaces and deployment control interfaces, for unified management of various aspects of model deployment.

[0010] Further, the intermediate representation layer is responsible for converting AI models into platform-independent intermediate formats, and this layer includes model conversion modules, dependency management modules and state tracking modules, which realize the standardization of model description and the unified management of deployment state.

[0011] Preferably, the platform adaptation layer includes format conversion modules, API calling modules and feedback processing modules, which are responsible for converting intermediate formats into deployment formats of target platforms and completing actual deployment operations through platform-specific API calls.

[0012] Optionally, the system also includes a deployment monitoring module and a log management module for real-time monitoring of deployment status and recording of key information.

[0013] In some embodiments, the intermediate format of the intermediate representation layer includes model structure information, model parameter information, runtime configuration information and resource requirement information, realizing the completeness and standardization of model description.

[0014] In addition, the API calling module of the present application also includes an authentication management unit, a request construction unit and a response parsing unit to ensure safe and reliable interaction with different cloud platforms.

[0015] In a preferred embodiment, the system receives AI models through the unified interface layer, converts them through the intermediate representation layer, and completes deployment to specific cloud platforms through the platform adaptation layer. The entire process is standardized and automated, greatly improving deployment efficiency.

[0016] In another preferred embodiment, the system performs integrity verification on the model to be deployed before performing the deployment operation, checks the availability status of the target platform, and confirms whether the required computing resources for deployment meet the requirements, thereby ensuring the reliability of the deployment process.

[0017] By adopting the above scheme, the application has the following beneficial effects: 1. By designing a unified interface layer, the development cost of cross-platform deployment is significantly reduced, and developers only need to connect to the standard interface to realize multi-platform deployment. 2. The intermediate representation layer mechanism is adopted to improve the portability of the model, so that the model can be easily migrated between different platforms. 3. The modular design of the platform adaptation layer makes the system have good scalability, and only needs to develop the corresponding adapter when supporting new platforms. 4. The standardization management of the deployment process is realized, and the deployment efficiency and reliability are improved. 5. Avoiding the technical dependence on a specific cloud platform, enhancing the user's platform selection freedom.

[0018] In summary, the application effectively solves the technical problem of AI model cross-platform deployment through innovative hierarchical architecture design, providing important support for the large-scale application of AI technology. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0020] Figure 1 is a system overall architecture diagram. The diagram adopts a top-down hierarchical structure, clearly shows the relationship between the unified interface layer (110), the intermediate representation layer (120) and the platform adaptation layer (130) three-layer architecture and its internal components, and distinguishes the function modules of different levels by different colors.

[0021] Figure 2 is a unified interface layer interaction sequence diagram. The diagram uses a sequence diagram to show the interaction process between the model import interface (210), the configuration management interface (220) and the deployment control interface (230) and the client, including the complete process of model uploading, configuration submission, deployment request and state query.

[0022] Figure 3 is an intermediate representation layer structure diagram. The diagram adopts a left-right layout to show the internal components of the model conversion module (310), the dependency management module (320) and the state tracking module (330) and the data flow relationship between them, and each module contains specific functional units.

[0023] Figure 4is a platform adaptation layer state diagram. The diagram uses the form of a state diagram to show the state transition process of the format conversion module (410), the API calling module (420), and the feedback processing module (430), as well as the processing flow and state changes within each module.

[0024] Figure 5 is a monitoring and log management system flow diagram. The diagram shows the data flow conversion process of the deployment monitoring module (510) and the log management module (520), distinguishes the two main modules by color, and shows the complete flow of event collection, processing, analysis, and log management.

[0025] Figure 6 is a deployment flow and verification mechanism diagram. The diagram shows the complete flow of the model verification phase (610), the resource inspection phase (620), and the deployment execution phase (630), including the specific steps and state transitions of each phase, as well as the processing flow in the success / failure cases. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0027] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the indicated device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application.

[0028] Embodiment 1: Basic system architecture embodiment

[0029] The present embodiment provides a basic architecture implementation of an AI model cross-platform deployment system based on a unified interface and an intermediate representation layer. As shown in Figure 1 , the system uses a hierarchical architecture design, including a unified interface layer 110, an intermediate representation layer 120, and a platform adaptation layer 130, which are connected through standardized data exchange interfaces.

[0030] The unified interface layer 110 is located at the top of the system and serves as a unified access portal for the system. This layer uses standardized interface definition specifications to separate the implementation details of the lower layer from the calling logic of the upper layer through interface abstraction. The unified interface layer is designed using the RESTful API architecture style, all interfaces follow a unified request-response mode, and supports JSON format data exchange. The access rights of the interface are controlled through the OAuth 2.0 protocol to ensure the security of system calls.

[0031] The intermediate representation layer 120 is the core layer of the system, responsible for implementing the standardized representation and conversion of models. This layer uses a graph structure (G) to represent AI models, where G = (V, E), V represents the set of computation nodes, and E represents the set of data flow edges. Each node contains attribute information such as operation type and parameter configuration, and the edge describes the data dependency relationship between nodes. This representation has sufficient expressiveness and can describe common deep learning model structures. The intermediate representation layer converts the input model into a standardized intermediate format through a model parser and maintains the state information of the model during deployment.

[0032] The platform adaptation layer 130 is responsible for converting the intermediate representation into the specific deployment format required by the target platform. This layer implements corresponding adapters for different target platforms, each adapter contains specific format conversion logic and API calling modules. The adapter uses a plug-in design, supports dynamic loading and extension, making the system flexible to support new deployment platforms. The platform adaptation layer also implements a unified error handling mechanism to standardize the handling of abnormal situations during deployment.

[0033] In the specific workflow, when a model deployment request is received, the unified interface layer first verifies the legality of the request and parses the request parameters. Then, the intermediate representation layer receives the model file and converts it into a standardized intermediate format through a model parser. The system selects the corresponding platform adapter according to the target deployment platform, and performs format conversion and deployment operations. The state information of the entire deployment process is recorded in real time in the intermediate representation layer, and feedback is provided to the caller through the unified interface layer.

[0034] The hierarchical architecture mode proposed in this embodiment innovatively solves the problem of AI model cross-platform deployment. Unlike traditional point-to-point adaptation solutions, this architecture introduces an intermediate representation layer as the core abstraction layer, fundamentally changing the paradigm of model deployment. This innovation is not a simple combination of technologies, but a redefinition of the deployment architecture: the unified interface layer realizes the standardization of the deployment entry, the intermediate representation layer creatively solves the platform difference problem, and the platform adaptation layer ensures the scalability of the deployment process. This architecture innovation enables cross-platform deployment to change from "customized development" to "standardized adaptation", embodying significant architectural pattern innovation value.

[0035] It should be noted that the above embodiments are only one of the preferred specific implementations of the present application, and those skilled in the art can make appropriate adjustments and changes to the system architecture without deviating from the design idea of the present application. For example, a cache mechanism can be added between each layer to improve performance, or additional functional modules can be added to meet specific deployment requirements.

[0036] Embodiment 2: Specific implementation of the unified interface layer

[0037] This embodiment, based on embodiment 1, details the specific implementation of the unified interface layer. As shown in Figure 2 The unified interface layer includes three core functional modules: model import interface 210, configuration management interface 220, and deployment control interface 230.

[0038] The model import interface 210 is responsible for handling the uploading and parsing of model files. This interface supports AI model files in multiple standard formats, including ONNX, TensorFlow SavedModel, and PyTorch TorchScript. The request format definition of the model import interface is as follows:

[0039] The configuration management interface 220 is used to manage the configuration information of model deployment. The configuration information is designed in a hierarchical structure, supporting the combination of global configuration and specific platform configuration. The configuration data structure is defined as follows:

[0040] The deployment control interface 230 is responsible for managing the life cycle of model deployment, including deployment start, state query, update, and termination operations. This interface implements an asynchronous operation mode, tracking the deployment progress through task ID. The deployment task state transition can be represented as:

[0041] The unified interface layer adopts the REST architectural style, and all interfaces follow the standard HTTP method semantics. Interface security is achieved through a multi-layer authentication mechanism, including API key authentication (request signature using the HMAC-SHA256 algorithm), access token authentication (token verification based on the OAuth 2.0 protocol), and request timestamp verification (to prevent replay attacks).

[0042] To ensure the reliability of interface calls, the unified interface layer implements a complete error handling mechanism. The error response format is defined as follows:

[0043] The exception handling during interface calls adopts a hierarchical strategy. Request verification exceptions are directly returned as error responses; business processing exceptions are logged and error information is returned; system-level exceptions trigger alarms and perform fault recovery.

[0044] The unified interface layer design of the present embodiment achieves unified management of the model deployment process through standardized interface definitions and normalized data structures. This design not only ensures the reliability and security of the interface, but also provides good scalability, supporting the extension and optimization of subsequent functions.

[0045] It should be noted that the above interface implementation methods can be adjusted according to actual needs. For example, a version control mechanism for the interface can be added, or the granularity of configuration management can be expanded to adapt to different deployment scenario requirements. In addition, the specific implementation of the interface can use other mainstream Web service frameworks, as long as the basic principles and data structure design described in the present embodiment are followed.

[0046] Embodiment 3: Implementation method of intermediate representation layer and design of intermediate format

[0047] The present embodiment, based on the foregoing embodiments, details the specific implementation method of the intermediate representation layer. As shown in Figure 3 , the intermediate representation layer includes a model conversion module 310, a dependency management module 320, and a state tracking module 330, which achieve unified representation of models through standardized intermediate formats.

[0048] The model conversion module 310 is responsible for converting AI models of different formats into a unified intermediate representation format. The conversion process first constructs the topology of the computation graph, and then extracts operator and parameter information. The formal definition of the computation graph is as follows: G compute =(V op ,E data ,φ,ψ) Where: V op ={v i |v idenotes a computation operator E data = {e ij | e ij denotes a data flow φ: V op → Attrs defines operator attribute mapping ψ: E data → Types defines data type mapping

[0049] The intermediate format of the model adopts a hierarchical design, including four levels of model structure, computation graph, parameter information, and runtime configuration:

[0050] The dependency management module 320 is responsible for handling the dependency relationship of the model runtime. This module maintains a directed acyclic graph (DAG) of dependencies to describe the dependency relationship between components: G dep = (V comp , E dep ) where V comp represents a set of dependent components, and E dep represents a set of dependency edges. The dependency resolution algorithm uses topological sorting to ensure the correct loading order of dependencies.

[0051] The state tracking module 330 implements state management throughout the life cycle of model deployment. State information is maintained using an event-driven model, and the state transition function is defined as: S next = f(S current , E trigger ) where S represents a set of states, and E trigger represents triggering events. State changes are pushed to related components in real time through an event bus.

[0052] The intermediate representation layer implements data persistence through a unified serialization mechanism. The serialization format uses ProtocolBuffers, supporting both binary and text formats. The design of the serialized data structure ensures cross-language and cross-platform interoperability.

[0053] The core innovation of the intermediate representation layer is the design of a complete model representation system, which guarantees the completeness and consistency of the representation through formal mathematical models. This design not only supports model representation of mainstream deep learning frameworks, but also has good extensibility to adapt to the needs of new model structures.

[0054] It's important to note that the intermediate representation format can be tailored or expanded based on specific application scenarios. For example, it might be possible to add representation for model quantization information or expand the set of runtime configuration parameters. Furthermore, the serialization mechanism can also adopt other efficient serialization schemes, as long as they ensure data integrity and interoperability.

[0055] Example 4: Implementation of the platform adaptation layer and adapter design

[0056] This embodiment, based on the above embodiment, describes in detail the specific implementation of the platform adaptation layer. Figure 4 As shown, the platform adaptation layer includes a format conversion module 410, an API calling module 420 and a feedback processing module 430, and supports different target platforms through an extensible adapter architecture.

[0057] The format conversion module 410 is responsible for converting the intermediate representation into the proprietary format of the target platform. The conversion process is based on a bidirectional mapping mechanism to ensure the accurate conversion of model structure and parameters. The formal definition of format conversion is as follows: T:Model IR →Model target The conversion function T can be expressed as:

[0058] The API call module 420 implements the interactive interface with the target platform. This module adopts the adapter mode to provide a unified calling interface for each target platform. The adapter interface is defined as follows:

[0059] The authentication management unit implements a unified package of multiple authentication mechanisms and supports authentication methods based on keys, tokens, and certificates. The authentication context is defined as:

[0060] Feedback processing module 430 is responsible for processing the deployment results and status information returned by the platform. This module implements a standardized result parsing and error handling mechanism. The feedback information processing flow can be expressed as follows: Where φ represents the result parsing function, ψ represents the error handling function, Represents function composition.

[0061] The platform adaptation layer adopts a plug-in design and supports dynamic loading of platform adapters. The adapter registry maintains the mapping relationship between platform identifiers and adapter implementations: Registry = {(id i ,adapteri ) for i in PlatformSet

[0062] The life cycle of an adapter is uniformly managed by the container, including initialization, runtime configuration update, and resource recycling phases. The container provides the system services required by the adapter through dependency injection.

[0063] The core innovation of the platform adaptation layer lies in the design of an extensible adapter architecture, which realizes seamless support for different deployment platforms through unified interface abstraction and standardized conversion mechanisms. This design not only simplifies the development process of platform adaptation, but also ensures the reliability and consistency of deployment operations.

[0064] It should be noted that the specific implementation of the platform adaptation layer can be optimized according to actual needs. For example, a hot plug mechanism for adapters can be added, or an automatic discovery and registration function for adapters can be implemented. In addition, other design patterns can also be used for the implementation of adapters as long as they meet the requirements of extensibility and maintainability.

[0065] Embodiment 5: Implementation method of monitoring and log management system

[0066] This embodiment is based on the foregoing embodiments and details the implementation method of the system's monitoring and log management. As shown in Figure 5 The monitoring and log management system includes a deployment monitoring module 510 and a log management module 520, which realizes monitoring and tracking of the entire model deployment process through real-time data collection and analysis.

[0067] The deployment monitoring module 510 realizes real-time monitoring of the model deployment process. This module uses an event-driven architecture and captures the state changes of each component in the system through the observer pattern. The formal definition of a monitoring event is as follows: type: event type, source: event source, Event = {timestamp: timestamp,} payload: event data, context: context information

[0068] The collection and processing flow of monitoring data can be represented as: collect: E(t) → D(t) M(t) = {process: D(t) → S(t)} analyze: S(t) → A(t) Where E(t) represents the original event stream, D(t) represents the collected data, S(t) represents the processed state information, and A(t) represents the analysis result.

[0069] The log management module 520 is responsible for the collection, storage, and analysis of system operation logs. The log recording adopts a hierarchical structure design, containing multiple log levels:

[0070] The log storage adopts a distributed architecture, supporting real-time writing and querying. The storage strategy is defined as follows:

[0071] The monitoring system implements an adaptive alarm mechanism, and the alarm rules are based on a state machine model: Alert = (States, Rules, Actions) Where: States = {Normal, Warning, Critical} Rules: Metric x Threshold → State Actions: State x Policy → Operation

[0072] The system supports multi-dimensional monitoring index aggregation analysis, and the aggregation function is defined as:

[0073] The core innovation of the monitoring and log management system lies in the implementation of a complete observability solution. Through the event-driven monitoring architecture and the hierarchical log management mechanism, accurate monitoring and problem diagnosis of the model deployment process are achieved. This design not only provides rich monitoring dimensions, but also supports flexible alarm policy configuration.

[0074] It should be noted that the specific implementation of the monitoring and log management system can be adjusted according to actual needs. For example, more monitoring index types can be added, or a more complex alarm rule engine can be implemented. In addition, the storage strategy can also choose different storage schemes according to actual scenarios, as long as it can meet the performance and reliability requirements.

[0075] Embodiment 6: Implementation method of complete deployment process and verification mechanism

[0076] This embodiment is based on the previous embodiments and details the complete deployment process and verification mechanism of the system. As shown in Figure 6 The complete deployment process includes a model verification phase 610, a resource inspection phase 620, and a deployment execution phase 630, which ensures the reliability of the deployment process through strict verification and inspection mechanisms.

[0077] The model verification phase 610 first performs integrity verification on the model to be deployed. The verification process adopts a multi-level check mechanism, and the verification function is defined as follows: Where the model integrity score is calculated as: Score integrity = α · V s + β · V p + γ · V d

[0078] Here α, β, γ are weight coefficients, V s , V p , V d represent the verification results of structure, parameters and dependencies respectively.

[0079] The resource checking phase 620 is responsible for evaluating the resource status of the target platform. The resource checking model is defined as:

[0080] The resource matching degree calculation adopts a vector distance model: Where w i is the resource item weight, r i and a i represent the demand and available quantity respectively.

[0081] The deployment execution phase 630 implements complete deployment flow control. The execution process state transition is defined as:

[0082] The transaction management of the deployment process adopts a compensation mechanism:

[0083] The system implements a full-process exception handling mechanism, and the exception handling strategy is defined as:

[0084] The core innovation of this embodiment is to design a complete deployment flow control system, which realizes the reliability guarantee of the deployment process through formal verification model and resource evaluation mechanism. This design not only ensures the accuracy of the deployment operation, but also provides perfect exception handling and rollback mechanism.

[0085] It should be noted that the specific implementation of the deployment process can be adjusted according to actual needs. For example, more verification dimensions can be added, or more complex resource scheduling strategies can be implemented. In addition, the exception handling mechanism can also be customized according to specific scenarios, as long as it can guarantee the reliability and recoverability of the deployment process.

Claims

1. An AI model cross-platform deployment system, characterized by: include: (a) Unified interface layer, which provides standardized model deployment interface specifications; (b) Intermediate representation layer, used to convert AI models into a platform-independent intermediate format; (c) a platform adaptation layer, configured to convert the intermediate format into a deployment format for a specific target platform; Among them, the unified interface layer, intermediate presentation layer and platform adaptation layer constitute a hierarchical model deployment architecture.

2. The system according to claim 1, wherein: The unified interface layer includes: (a) Model import interface, used to receive the AI ​​model to be deployed; (b) Configuration management interface, used to manage the configuration information of model deployment; (c) Deployment control interface, used to control the execution process of model deployment.

3. The system according to claim 1, wherein: The intermediate presentation layer includes: (a) Model conversion module, used to convert AI models into a unified intermediate format; (b) Dependency management module, used to manage the dependencies required for model operation; (c) Status tracking module, used to record the status information of model deployment.

4. The system according to claim 1, wherein: The platform adaptation layer includes: (a) a format conversion module, used to convert the intermediate format into the target platform format; (b) an API calling module, used to call the deployment interface of the target platform; (c) Feedback processing module, used to process the deployment results returned by the platform.

5. The system according to claim 3, wherein: The intermediate format includes: (a) Model structure information, model parameter information, runtime configuration information, and resource requirement information.

6. The system according to claim 4, characterized in that The API calling module also includes: (a) an authentication management unit for managing access credentials to the target platform; (b) a request construction unit, used to construct an API request that complies with the target platform specification; (c) A response parsing unit, used to parse the API response of the target platform.

7. The system according to claim 1, wherein: Also includes: (a) Deployment monitoring module, used to monitor the model deployment status in real time; (b) Log management module, used to record key information of the deployment process.

8. A cross-platform deployment method for an AI model based on the system according to any one of claims 1 to 7, characterized in that: include: (a) Receive the AI ​​model to be deployed through the unified interface layer; (b) converting the AI ​​model into a unified format for an intermediate representation layer; (c) Call the corresponding platform adapter according to the target platform type; (d) Converting the unified format into a target platform format and completing the deployment.

9. The method according to claim 8, characterized in that The method further comprises: (a) Monitor the deployment status in real time during the deployment process; (b) Record key log information of the deployment process; (c) Feedback deployment results to users.

10. The method according to claim 8, characterized in that The method further comprises: (a) Verify the integrity of the model to be deployed; (b) Check the availability of the target platform; (c) Confirm whether the resources required for deployment meet the requirements.

Images