Resource packaging and deploying method, product, equipment and medium

Abstract

The invention discloses a resource packaging and deploying method, a product, equipment and a medium, and relates to the technical field of cloud computing and artificial intelligence, and the method comprises the following steps: converting a structured resource depended by an AI application into a target structured resource in a standardized YAML format, storing a block path of an unstructured resource depended by the AI application, and generating a metadata index; constructing a resource dependency graph according to the metadata information of the target structured resource, the block path information in the metadata index and the resource dependency relationship between the unstructured resource; packaging each block file, the metadata information, the metadata index and the resource dependence graph formed by block compression of the target structured resource and the unstructured resource into a single resource packet; and converting the single resource packet into a target cloud platform instruction through an interface translation engine, so as to perform resource deployment on the current target cloud platform based on the target cloud platform instruction. And unified packaging and automatic cross-cloud deployment of the full-stack heterogeneous resources are realized.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing and artificial intelligence technologies, and in particular to a resource packaging and deployment method, product, device, and medium. Background Art

[0002] With the deep integration of cloud computing and artificial intelligence technologies, enterprise applications are gradually shifting to AI (Artificial Intelligence) architectures. These applications rely on heterogeneous cloud resources, encompassing infrastructure such as cloud hosts and container clusters, as well as Platform as a Service (PaaS) services like databases and middleware. They also require the integration of unstructured data such as large models and knowledge bases. However, mainstream infrastructure-as-code tools (such as Terraform) can define resource configuration templates but cannot package large model files. Container technologies (such as Docker) are limited to encapsulating application runtime environments and do not cover cloud hosts and PaaS service instances. Model hosting platforms (such as HuggingFace) lack the ability to manage and coordinate with underlying computing resources (such as GPU (Graphics Processing Unit) nodes). This results in the need for repetitive scripting for cross-environment deployment, which is inefficient, error-prone, and difficult to ensure data consistency. Existing solutions require a combination of multiple tools (infrastructure-as-code tools + containers + model repositories), failing to achieve unified packaging and one-click migration of full-stack resources, severely hindering the scalability of AI applications.

[0003] It can be seen that how to achieve unified packaging and automated cross-cloud deployment of full-stack heterogeneous resources for AI applications, and eliminate configuration fragmentation and high multi-cloud adaptation costs caused by tool chain fragmentation, is a problem that technical personnel in this field need to solve. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a resource packaging and deployment method, product, device and medium, which can realize the unified packaging and automated cross-cloud deployment of full-stack heterogeneous resources of AI applications, eliminating the configuration fragmentation and high cost of multi-cloud adaptation caused by the separation of tool chains.

[0005] To solve the above technical problems, an embodiment of the present invention provides a resource packaging and deployment method, including: Convert the structured resources that AI applications rely on into target structured resources in the standardized YAML format, store the block paths of the unstructured resources that AI applications rely on, and generate metadata indexes; Construct a resource dependency graph based on the metadata information of the target structured resource, the block path information in the metadata index, and the resource dependency relationship between the unstructured resources; Packing the target structured resources and unstructured resources into a single resource package, including the block files, metadata information, metadata index, and resource dependency graph formed after block compression; The interface translation engine converts the single resource package into target cloud platform instructions, so as to deploy resources on the current target cloud platform based on the target cloud platform instructions.

[0006] Optionally, convert the structured resources that the AI application depends on into target structured resources in the standardized YAML format, store the block paths of the unstructured resources that the AI application depends on, and generate metadata indexes, including: Structural division of the full-stack AI application dependencies into structured and unstructured resources. Convert structured resources into target structured resources with metadata in standard YAML format; The resource path of the unstructured resources is identified to obtain the target unstructured resource list and the corresponding metadata index.

[0007] Optionally, before packaging the target structured resources and unstructured resources into a single resource package, the following steps may be included: Determine whether the total resource size of the unstructured resources exceeds a preset file size threshold; If so, the unstructured resource is divided into blocks according to a preset block size to obtain block resources; Compress each block resource based on a preset compression algorithm and according to preset compression parameters to obtain each block file; Perform hash operations on each block file to obtain standard verification information of each block file; Added standard checksum information and preset compression parameters to the metadata index.

[0008] Optionally, the process of packaging the target structured resources and unstructured resources into a single resource package, including the compressed block files, metadata information, metadata indexes, and resource dependency graph, further includes: Perform hash operation on each block file to obtain the verification information to be verified for the corresponding block file; Extract the standard verification information of each block file from the metadata index, and check whether the standard verification information is consistent with the verification information to be verified; If they are consistent, the block file passes the integrity check and the packaging process is started; If they are inconsistent, an error will be reported directly and the packaging process will be stopped.

[0009] Optionally, the interface translation engine is used to convert a single resource package into target cloud platform instructions, including: The platform parameters of the target structured resources in a single resource package are mapped to the target platform interface parameters under the target cloud platform through the interface translation engine.

[0010] Optionally, the platform parameters of the target structured resources in the single resource package are mapped to the target platform interface parameters under the target cloud platform through the interface translation engine, including: Extracting a node attribute set of a target structured resource from a resource dependency graph through an interface translation engine; wherein the node attribute set includes a resource type identifier, associated dependency relationships, and configuration constraints; A predefined platform parameter rule library is queried based on the node attribute set to map the platform parameters of the target structured resource to the target platform interface parameters under the target cloud platform according to the matching target mapping rules.

[0011] Optionally, after querying the predefined platform parameter rule base based on the node attribute set, it also includes: If there are conflicting items in the matching results of resource type identifiers, associated dependencies, and configuration constraints, the optimal matching result is selected based on the resource priority in the resource dependency graph to execute the corresponding platform parameter mapping step; If there is no match, the self-healing mechanism is triggered to generate a degradation plan and update the resource dependency graph.

[0012] Optional resource packaging and deployment methods also include: Update resource node identifiers based on the resource dependency graph; The resource dependency graph is reconstructed based on the updated resource node identifier to obtain a reconstructed resource dependency graph.

[0013] Optionally, deploy resources on the current target cloud platform based on the target cloud platform's instructions, including: Execute resource deployment operations for the full stack of AI application dependencies on the current target cloud platform based on the target platform interface parameters, resource node identifiers, and the reconstructed resource dependency graph.

[0014] Optionally, based on the target platform interface parameters, resource node identifiers, and the reconstructed resource dependency graph, perform resource deployment operations for the full-stack dependent resources of the AI application on the current target cloud platform, including: Generate the creation sequence of full-stack dependent resources for the AI application adapted to the current target cloud platform based on the reconstructed resource dependency graph; Execute resource deployment operations for the AI application full-stack dependent resources on the current target cloud platform based on the target platform interface parameters, resource node identifiers, and the order in which the AI application full-stack dependent resources are created.

[0015] Optionally, the structured resources of AI applications include infrastructure as a service resources and platform as a service components; the unstructured resources of AI applications include AI model files and knowledge base datasets.

[0016] Optional resource packaging and deployment methods also include: Capture the return information of the target cloud platform interface during resource deployment; If there is a timeout error in the returned information, it will automatically retry and wait with exponential backoff; If there are configuration conflict errors in the returned information, the associated dependencies are checked according to the resource dependency graph and batch rollback is performed; If the returned information contains an unavailable error, switch to the backup area and reconstruct the resource node identifier; Mark successful or failed resources on the resource dependency graph to generate a visual resource deployment progress topology.

[0017] In a second aspect, an embodiment of the present invention provides a computer program product, including a computer program / instruction, which implements the steps of the aforementioned disclosed resource packaging and deployment method when executed by a processor.

[0018] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the aforementioned disclosed resource packaging and deployment method.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the resource packaging and deployment method disclosed above are implemented.

[0020] An embodiment of the present invention discloses a resource packaging and deployment method, including: converting structured resources that an AI application depends on into target structured resources in a standardized YAML format, storing the block paths of unstructured resources that the AI application depends on and generating a metadata index; constructing a resource dependency graph based on the metadata information of the target structured resources, the block path information in the metadata index, and the resource dependency relationship between the unstructured resources; packaging the target structured resources and the unstructured resources into a single resource package after the block files, metadata information, metadata index, and resource dependency graph are compressed; and converting the single resource package into a target cloud platform instruction through an interface translation engine, so as to deploy resources on the current target cloud platform based on the target cloud platform instruction.

[0021] As can be seen from the above technical solution, by converting structured resources into a YAML format and storing unstructured resources in blocks, and generating metadata indexes, structured and unstructured data can be integrated into the same packaging and processing workflow framework. A resource dependency graph is constructed based on metadata information and block paths, resolving deployment order issues caused by missing dependencies, ensuring consistent resource configuration, and providing a topological basis for multi-cloud adaptation, eliminating the need for manual scripting of resource creation sequences. By packaging structured resources, block files, metadata, and a dependency graph into a single resource package, all AI application dependencies, from infrastructure to model parameters, are integrated into a single entity, enabling one-click migration and eliminating the risk of version mismatches caused by the patchwork of multiple tools in traditional solutions. An interface translation engine converts resource packages into target cloud platform instructions and executes deployment, eliminating manual adaptation costs. This standardizes the packaging of full-stack AI application resources and automates multi-cloud deployment, achieving the core goal of "package once, deliver everywhere" and completely eliminating the manual intervention costs and cross-cloud configuration inconsistencies caused by fragmented tool chains. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 A flow chart of a resource packaging and deployment method provided by an embodiment of the present invention; Figure 2 A schematic diagram of a multi-cloud adapter compatible with multi-cloud vendor API architecture provided by an embodiment of the present invention; Figure 3 A flow chart of a method for resource packaging and deployment at a user end provided by an embodiment of the present invention; Figure 4 A structural diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] The terms "including" and "having," as used in the present description and accompanying drawings, and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements and may include steps or elements that are not listed.

[0026] In order to enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0027] With the deep integration of cloud computing and artificial intelligence technologies, enterprise applications are gradually shifting to AI-based architectures. These applications rely on heterogeneous cloud resources, encompassing infrastructure such as cloud hosts and container clusters, as well as Platform-as-a-Service (PaaS) services like databases and middleware. They also require the integration of unstructured data such as large models and knowledge bases. However, mainstream infrastructure-as-code tools (such as Terraform) can define resource configuration templates but cannot package large model files. Container technologies (such as Docker) are limited to encapsulating application runtime environments and do not extend to cloud hosts and PaaS service instances. Model hosting platforms (such as HuggingFace) lack the ability to manage and integrate with underlying computing resources (such as GPU nodes). This results in the need for repetitive scripting for cross-environment deployment, which is inefficient, error-prone, and difficult to ensure data consistency. Existing solutions require a combination of multiple tools (IaC + containers + model repositories), preventing unified packaging and one-click migration of full-stack resources, severely hindering the scalability of AI applications.

[0028] To this end, the present invention provides a resource packaging and deployment solution that can achieve unified packaging and automated cross-cloud deployment of full-stack heterogeneous resources for AI applications, eliminating configuration fragmentation and high multi-cloud adaptation costs caused by tool chain fragmentation.

[0029] Reference Figure 1 As shown, the present invention provides a resource packaging and deployment method, including: Step S11: Convert the structured resources that the AI application depends on into target structured resources in a standardized YAML format, store the block paths of the unstructured resources that the AI application depends on, and generate metadata indexes.

[0030] In this embodiment, the structured resources of an AI application are Infrastructure as a Service resources and Platform as a Service components; the unstructured resources of an AI application are AI model files and knowledge base datasets. It is understood that the structured resources of an AI application specifically include cloud host specifications (e.g., ecs.g6.4xlarge), container configurations, IaaS, PaaS, network topology, etc. Unstructured resources, on the other hand, are AI model files and knowledge base datasets. Specifically, if the current AI application is an AI-based risk control system, it is necessary to migrate this risk control system from Alibaba Cloud to AWS (Amazon Web Services) to achieve automated deployment of the full stack of AI resources and seamless migration between the two platforms in minutes. First, the structured and unstructured resources involved in the current AI-based risk control system are obtained. Structured resources (e.g., cloud host configurations and database parameters) are typically described in standardized YAML / JSON, while unstructured resources (e.g., model files and log data) lack a unified format. This creates a technical gap between the two, necessitating the use of unified modeling information to establish semantic associations between resources.

[0031] In this embodiment, the AI application full stack dependent resources are structured to obtain structured resources and unstructured resources; the structured resources are converted into target structured resources with metadata information in the standard YAML format; the resource path of the unstructured resources is identified to obtain a list of target unstructured resources and the corresponding metadata index. It can be understood that the steps for obtaining structured resources and hybrid modeling of unstructured resources are as follows: Obtain cloud platform API metadata through interfaces, such as cloud host configuration (partial structured resources) in Alibaba Cloud, AWS, and OpenStack clouds.

[0032] Obtain model files, container images, knowledge base files, and other data (partial structured resources and unstructured resources) through the cloud platform storage interface, such as calling the S3 (Amazon Simple Storage Service) docking storage interface to obtain the corresponding files.

[0033] The structured resources are uniformly converted into target structured resources in the standard YAML format. Furthermore, in the structured resource modeling phase, cloud host configurations, PaaS component parameters, network topology, etc. are converted into standardized YAML format files, where the YAML format files define metadata information such as resource types, versions, and dependencies in detail. These metadata information clearly shows the internal and mutual association logic of structured resources, such as the specific PaaS component version that a cloud host configuration depends on. Therefore, the metadata information of structured resources and the dependencies between structured resources are extracted in the target structured resources converted into the standard YAML format. The metadata information includes the dependencies between structured resources, which is used to subsequently define the dependency chain of cloud host configurations, container services, and large model files.

[0034] For the unstructured resources, since they cannot be converted into YAML format files, and the unstructured resources in this solution are model files and knowledge base files, both of which are large files, in order to achieve minute-level deployment in subsequent deployment, large files need to be segmented, packaged, and transmitted in parallel. It should be noted that pre-segmentation is performed when modeling the current unstructured resources. In this way, each pre-segment corresponds to a resource segment path, and a corresponding metadata index for the corresponding pre-segment is generated, where the metadata index contains the segment path information of the corresponding pre-segment. Through these metadata indexes, the characteristics of each segment of the unstructured resources and the relationship between them and the unstructured resources can be clarified.

[0035] Step S12: construct a resource dependency graph based on the metadata information of the target structured resource, the block path information in the metadata index, and the resource dependency relationship between the unstructured resources.

[0036] In this embodiment, the target structured resources, metadata information, unstructured resources, and metadata indexes obtained above are subjected to cross-resource resource dependency extraction to construct a resource dependency graph. Currently, in the process of constructing the resource dependency graph, since the resource dependency relationships (metadata information and metadata indexes) between the target structured resources and unstructured resources are extracted, these resource dependencies are used to describe the dependency relationships between structured resources and unstructured resources based on a directed acyclic graph to form a global dependency graph, that is, a resource dependency graph.

[0037] Step S13: Packing the target structured resources and unstructured resources into a single resource package, including the block files, metadata information, metadata index, and resource dependency graph formed by block compression.

[0038] In this embodiment, before packaging the target structured resources and unstructured resources into a single resource package, the block files, metadata information, metadata index, and resource dependency map formed by block compression, also include: determining whether the total resource size of the unstructured resources exceeds a preset file size threshold; if so, performing block processing on the unstructured resources according to the preset block size to obtain each block resource; compressing each block resource based on a preset compression algorithm and according to preset compression parameters to obtain each block file; performing a hash operation on each block file to obtain standard verification information of each block file; and adding the standard verification information and preset compression parameters to the metadata index. Among them, the process of packaging the block files, metadata information, metadata index, and resource dependency map formed by compressing the target structured resources and unstructured resources into a single resource package also includes: performing hash operations on each block file to obtain the verification information to be verified of the corresponding block file; extracting the standard verification information of each block file from the metadata index, and detecting whether the standard verification information is consistent with the verification information to be verified; if they are consistent, the block file passes the integrity check and the packaging process is started; if they are inconsistent, an error is directly reported and the packaging process is stopped.

[0039] It can be understood that the resource size of the current unstructured resources is verified. Since the unstructured resources are large files as mentioned in the previous embodiment, in order to avoid the block operation on small files, the resource size of the unstructured resources is verified before packaging, that is, whether the total resource size of the unstructured resources exceeds the preset file size threshold. For example: the AI model file in the structured resources is 300GB, and the preset file size threshold is 5GB, then it is obvious that the current AI model file exceeds the preset file size threshold. Correspondingly, the file block process is triggered, and the file block processing is performed according to the 1GB block size to obtain each block resource; then the Zstandard algorithm and compression parameters are used to compress the block resources for each block resource, merge the YAML configuration and the block index table, generate a single file, embed the global resource topology map, and generate separate standard verification information for each block to facilitate subsequent file user integrity verification.

[0040] It should be noted that before packaging, each block resource needs to be verified. Specifically, the hash algorithm used to calculate the standard verification information is used to perform a hash operation on each block resource to obtain the verification information to be verified. The verification information to be verified is then compared with the standard verification information. If they are completely consistent, it indicates that the block file has passed the integrity check and the packaging process is started. If they are inconsistent, an error is directly reported, the packaging process is stopped, and an error is reported so that the operation and maintenance personnel can handle the error information in time.

[0041] In this embodiment, the steps of performing the packaging process are as follows: performing unified metadata processing on the metadata information and metadata index, thereby determining the packaging structure, and generating a compressed package directory structure. The specific target structure example is shown below: ├── META / │ ├── manifest.yaml # Global metadata (resource type, version) │ ├── dependency_graph.json # Dependency topology graph ├── STRUCTURED / │ ├── ecs.yaml # Cloud host configuration │ └── rds.yaml # Database parameters └── UNSTRUCTURED / ├── model_blocks / # Large model blocks │ ├── block_001.zst │ └── index.json # Block hash index.

[0042] The main directory is planned as follows: the META directory describes the metadata and dependencies of the entire package; the STRUCTURED directory describes the various resource configurations in the package, including cloud host configuration, container configuration, PaaS configuration, large model configuration, knowledge base configuration, etc.; the UNSTRUCTURED directory stores various data files.

[0043] During the resource packaging process, it is necessary to ensure the integrity of the files while taking into account the lightweight installation package, reducing space usage, and improving transmission efficiency. Large file compression is achieved by segmenting, compressing, and managing large files in blocks as described in the above embodiment. The large files mainly involved in the method include unstructured data such as model files and knowledge base files. Among them, after block compression, the metadata content format in the installation package is shown in the following code: UNSTRUCTURED / ├── model_blocks / │ ├── block_001.zst # Block file │ ├── block_002.zst │ └── _meta / # Large file metadata subdirectory │ ├── index.json # Block index (including hash and compression parameters) │ └── dict.zstd # Zstandard dictionary file.

[0044] The above content specifically defines the metadata index of the large installation package file, which specifically includes the relevant path information of the blocks and parameter information such as block information, compression parameter information, and compression algorithm information that are subsequently updated to the metadata index.

[0045] It is important to note that, based on the storage of unstructured resource blocks and the construction of a dependency graph, the access frequency of unstructured resource blocks (such as model layer parameters and knowledge base fragments) is calculated through historical user access logs and AI application runtime call records. These blocks are then marked with a heat level in the metadata index (pre-classified by access frequency, corresponding to core, high-frequency, and low-frequency levels). The resource dependency graph is then pruned based on the heat level. Specifically, for low-frequency blocks, their strong dependencies on core resources are temporarily removed in non-urgent deployment scenarios, retaining only logical references. This allows core and high-frequency blocks to be transferred first during cross-cloud transmission, and low-frequency blocks to be asynchronously loaded after core functionality is enabled. If the deployment environment is resource-constrained, such as if the target cloud platform temporarily runs out of quota, the deployment of low-frequency blocks can be skipped based on the pruned dependency graph, ensuring only the availability of core functionality. Furthermore, before concurrent block transmission, the heat level identifiers in the metadata are read to generate a priority transmission queue. During dependency reconstruction, the dependency constraints of low-frequency blocks are automatically weakened based on pruning rules. Correspondingly, the above steps can solve the problems of slow deployment startup and resource waste caused by large models and too many knowledge base blocks. They also allow core blocks to be available first, shortening the startup time of basic functions of AI applications, further adapting to resource-constrained scenarios, and improving fault tolerance in complex cloud environments.

[0046] Step S14: converting the single resource package into target cloud platform instructions through the interface translation engine, so as to deploy resources on the current target cloud platform based on the target cloud platform instructions.

[0047] In this embodiment, the platform parameters of the target structured resources in a single resource package are mapped to the target platform interface parameters of the target cloud platform through an interface translation engine. The resource node identifiers are updated based on the resource dependency graph; the dependency relationships of the resource dependency graph are reconstructed based on the updated resource node identifiers to obtain a reconstructed resource dependency graph. It can be understood that the API translation engine converts a single resource package into an executable instruction set (target platform interface parameters) for the target cloud platform to achieve cross-platform deployment, specifically including: 1. Parameter mapping: Map the platform parameters in the target structured resource (such as compute_type: gpu) to the API parameters of the target cloud platform (such as AWS InstanceType: p4d.24xlarge); 2. Dependency Reconstruction: Update the resource node identifiers in the resource dependency graph according to the resource identification rules of the target platform (such as the VPC ID format); 3. Instruction generation and execution: Outputs a deployment instruction set (such as a Terraform template) adapted to the target cloud platform and executes the deployment according to the resource dependency graph. Combining pre-checking, block-based concurrent transmission, and self-healing mechanisms ensures the success rate of deployment.

[0048] Specifically, for the parameter mapping step, first, the node attribute set of the target structured resource is extracted from the resource dependency graph through the interface translation engine; wherein the node attribute set includes resource type identification, associated dependencies, and configuration constraints; based on the node attribute set, the predefined platform parameter rule library is queried to map the platform parameters of the target structured resource to the target platform interface parameters under the target cloud platform according to the matching target mapping rules. It can be understood that the resource type identification, such as: GPU instance / vector database; associated dependencies, such as: bound container service; configuration constraints, such as CUDA 11.8 / FP16 precision. Figure 2 As shown, the interface translation engine (multi-cloud adapter) pre-maps various metadata to different cloud platforms, forming several mapping rules between metadata and cloud platforms. For example, metadata can specifically include cloud host metadata, container metadata, PaaS metadata, etc.; and various cloud platforms specifically include Alibaba Cloud, AWS, OpenStack, etc.

[0049] Parameter translation maps the resource configuration in the package file to the target cloud platform API parameters. The following is the mapping rule for mapping the resource configuration of the cloud host to the AWS resource type: #Cloud host → AWS EC2 instance type mapping rules ecs.g6.4xlarge→ec2.m5.8xlarge essd.cloud_disk→gp3.volume_type.

[0050] Among them, after querying the predefined platform parameter rule base based on the node attribute set, it also includes: if there are conflicting items in the matching results of the resource type identifier, the matching results of the associated dependencies, and the matching results of the configuration constraints, then the optimal matching result is selected according to the resource priority in the resource dependency graph to execute the corresponding platform parameter mapping step; if there is no matching item, the self-healing mechanism is triggered to generate a degradation plan and update the resource dependency graph. It can be understood that if there are conflicting items, the configuration conflict is resolved based on the graph priority. For example: when the GPU model conflicts with the container driver version, the core constraints are retained according to the dependency chain priority; if there is no matching item, the platform differences are dynamically adapted, for example: if the target platform does not have an FP16 GPU instance, it is automatically downgraded to FP32 and the model accuracy mark is updated.

[0051] In this embodiment, resource deployment is performed on the current target cloud platform based on the target cloud platform instruction, including: executing the resource deployment operation of the full stack of AI application dependent resources on the current target cloud platform based on the target platform interface parameters, resource node identifiers, and the reconstructed resource dependency graph. Specifically, the creation order of the full stack of AI application dependent resources adapted to the current target cloud platform is generated based on the reconstructed resource dependency graph; the resource deployment operation of the full stack of AI application dependent resources is executed on the current target cloud platform according to the target platform interface parameters, resource node identifiers, and the creation order of the full stack of AI application dependent resources. It can be understood that before finding resource deployment, for the block resources in the resources, the model blocks are distributed to the target cloud object storage, downloaded in parallel, and reorganized according to the index table. Among them, since the original AI resources are applied on the previous cloud platform, the dependency relationship will be updated when deployed to another cloud platform, which will cause the resource dependency graph to be reconstructed. Further, the corresponding resource deployment is performed according to the resource creation order in the updated resource dependency graph.

[0052] In this embodiment, after resource deployment, the return information from the target cloud platform interface during the resource deployment process is captured. If a timeout error is found in the return information, an automatic retry is performed with exponential backoff. If a configuration conflict error is found in the return information, the associated dependencies are checked based on the resource dependency graph and batch rollback is performed. If an unavailable error is found in the return information, the backup region is switched and the resource node identifiers are reconstructed. Successful or failed resources are marked on the resource dependency graph to generate a visual resource deployment progress topology. It is understood that the hierarchical self-healing strategy and resource status tracking in the event of deployment failure specifically monitor the deployment status (return information) in real time. If the return information contains various corresponding deployment error messages, the corresponding hierarchical self-healing mechanism is triggered and the corresponding self-healing behavior is executed.

[0053] like Figure 3As shown in the figure, if the current user uses the user interaction module to package and deploy AI full-stack resources, then the specific scenario is that a large bank needs to migrate its AI-based risk control system (which relies on cloud hosts, databases, TensorFlow models, and knowledge bases) from Alibaba Cloud to AWS. Correspondingly, the user clicks or selects buttons or keys on the interactive interface through the user interaction module, and then the resource discovery module obtains various cloud resources related to the risk control system (structured resources and non-structured resources) and performs resource modeling. For example: Cloud host configuration: # ecs.yaml (Alibaba Cloud) instance_type: ecs.g6.4xlarge # GPU instance vpc_id: vpc-123456 security_group_ids: [sg-abcdef].

[0054] Database configuration: # rds.yaml engine: mysql version: 8.0 storage_encrypted: true.

[0055] PaaS components: # kafka.yaml brokers: 3 topic_partitions: 100.

[0056] The above is the resource modeling of structured resources in the risk control system.

[0057] For the unstructured resources of the risk control system, the resource modeling is as follows: TensorFlow model file: Original file: risk_model_v1.h5 (12GB).

[0058] Dynamic blocking: block model_blocks / block_001.zst and block_002.zst are divided into 5GB blocks.

[0059] Metadata Record: { "blocks": [ {"hash": "sha256:abc123", "size": 5368709120} ], "compression": "zstd" }.

[0060] Knowledge base data: Historical transaction data (Parquet format, 3 GB); The blocks are compressed and stored in data_blocks / , generating data_index.json.

[0061] Dependency Modeling: Dependency graph: { "nodes": [ {"id": "vpc-123456", "type": "vpc"}, {"id": "gpu-instance", "type": "compute", "dependsOn": ["vpc-123456"]}, {"id": "model-service", "type": "model", "dependsOn": ["gpu-instance"]} ], "edges": [ {"from": "vpc-123456", "to": "gpu-instance"} ] }.

[0062] Dependency constraints: Model services must be deployed on GPU instances; The database and Kafka must be in the same VPC.

[0063] Then, the structured and unstructured resources of the risk control system are encapsulated through the encapsulation and verification modules. The specific steps are as follows: Structured resource packaging: Compress ecs.yaml, rds.yaml, and kafka.yaml into structured_resources.tar.gz.

[0064] Unstructured resource encapsulation: Block compression: The total compression ratio of the model file after segmentation is increased by 30% (from 12GB to 8.5GB); The database backup file is compressed into data_backup.zst (compression ratio 4:1).

[0065] Integrity check: Generate the index.json record block hash value (SHA256); Furthermore, resource packages are synthesized based on the storage and transmission module. Specifically, resources are packaged and synthesized according to the packaging structure disclosed in the above embodiment to obtain a single resource package.

[0066] Finally, the multi-cloud management module automatically converts the Alibaba Cloud resource package into an AWS-compatible format and completes the deployment. The specific steps are as follows: 1. API translation engine mapping: Parameter conversion: #Alibaba Cloud field → AWS field mapping table; compute_type: ecs.g6.4xlarge→InstanceType: p4d.24xlarge; vpc_id: vpc-123456→VpcId: vpc-aws-987654; Semantic adaptation: Alibaba Cloud's CreateInstance API → AWS's RunInstances API; Automatically handle parameter differences (e.g., Alibaba Cloud's SecurityGroupId → AWS's SecurityGroupIds array).

[0067] 2. Dependency remapping: The AWS VPC ID format is different from that of Alibaba Cloud, and the node ID in the dependency graph is automatically updated.

[0068] / / Alibaba Cloud dependency graph {"from": "vpc-123456", "to": "gpu-instance"} / / After converting to AWS {"from": "vpc-aws-987654", "to": "i-0abcd1234efgh5678"}.

[0069] 3. Cross-platform deployment and execution: Blocked concurrent transmission: Upload via AWS S3 multipart upload (aws s3 cp --recursive --multipart); Resume: After a transfer is interrupted, the transfer is continued based on the hash value recorded in index.json.

[0070] Dependency-driven deployment: First create an AWS VPC (vpc-aws-987654), and then deploy an EC2 instance (i-0abcd1234efgh5678).

[0071] Self-healing mechanism: If the GPU instance fails to start (such as an incompatible AMI version), it will automatically roll back and replace it with an alternative AMI. If the verification fails (such as a block hash mismatch), it will trigger an automatic re-download.

[0072] The functions of the above modules are as follows: User interaction module: user operation interface and API interface.

[0073] Resource discovery module: used to discover various cloud resources, such as required cloud hosts, containers, PaaS, large models, etc.

[0074] Packaging and verification module: compress large files into blocks and form a unified installation package format, etc.

[0075] Multi-cloud management module: Multi-cloud adaptation, installing interfaces supported by various cloud platforms, and calling corresponding cloud platform interfaces to create resources.

[0076] Storage and transmission module: file storage based on various cloud platforms, etc.

[0077] In this way, the intelligent packaging and automated deployment method of heterogeneous resources for AI applications, by uniformly encapsulating the full stack of AI application dependencies (from IaaS to model files), achieves one-time packaging and delivery everywhere, breaking through the fragmented problem of heterogeneous resource migration and providing infrastructure support for the large-scale implementation of AI. Specifically, the complete dependency chain from IaaS resources to model files is uniformly packaged, and the time for manual intervention in deployment and configuration is reduced from weeks to hours. Standardized delivery: Supports one-time packaging and delivery everywhere, greatly reducing the time for cross-cloud and cross-region replication of enterprise AI applications. One-click cross-cloud adaptation is achieved through the manufacturer's API translation engine, supporting mainstream cloud platforms and reducing code development costs.

[0078] By converting structured resources into a YAML format and storing unstructured resources in blocks, and generating metadata indexes, this approach integrates structured and unstructured data into a unified packaging and processing framework. A resource dependency graph is constructed based on metadata and block paths, resolving deployment order issues caused by missing dependencies, ensuring consistent resource configuration, and providing a topological basis for multi-cloud adaptation, eliminating the need for manual scripting of resource creation sequences. By packaging structured resources, block files, metadata, and a dependency graph into a single resource package, all AI application dependencies, from infrastructure to model parameters, are integrated into a single entity, enabling one-click migration and eliminating the risk of version mismatches caused by the patchwork of multiple tools in traditional solutions. An interface translation engine converts resource packages into target cloud platform instructions and executes deployment, eliminating manual adaptation costs. This standardizes the packaging of full-stack AI application resources and automates multi-cloud deployment, achieving the core goal of "package once, deliver everywhere" and completely eliminating the manual intervention costs and cross-cloud configuration inconsistencies caused by fragmented tool chains.

[0079] Furthermore, the embodiment of the present application also discloses an electronic device, Figure 4 is a structural diagram of an electronic device according to an exemplary embodiment. Figure 4 The content in the description should not be considered as any limitation on the scope of use of this application. The electronic device may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the resource packaging and deployment method disclosed in any of the aforementioned embodiments. In addition, the electronic device in this embodiment may specifically be an electronic computer.

[0080] In this embodiment, the power supply 23 is used to provide operating voltage for various hardware devices on the electronic device; the communication interface 24 can create a data transmission channel between the electronic device and external devices. The communication protocol it follows is any communication protocol that can be applied to the technical solution of this application and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world. Its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0081] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or CD, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0082] The operating system 221 is used to manage and control the hardware devices on the electronic device, as well as the computer program 222, which can be Windows Server, NetWare, Unix, Linux, etc. In addition to including computer programs capable of implementing the resource packaging and deployment method performed by the electronic device disclosed in any of the aforementioned embodiments, the computer program 222 can further include computer programs capable of completing other specific tasks.

[0083] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when executed by a processor, the computer program implements the resource packaging and deployment method disclosed above. The specific steps of this method can be referred to the corresponding content disclosed in the above embodiments and will not be repeated here.

[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. Reference can be made to the descriptions of the identical or similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple, and the relevant parts can be referred to the descriptions of the methods.

[0085] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0086] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0087] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0088] The above is a detailed introduction to the technical solution provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the ideas of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A resource packaging and deployment method, characterized in that: include: Convert the structured resources that AI applications rely on into target structured resources in the standardized YAML format, store the block paths of the unstructured resources that AI applications rely on, and generate metadata indexes; Constructing a resource dependency graph according to the metadata information of the target structured resource, the block path information in the metadata index, and the resource dependency relationship between the unstructured resource; Packing the target structured resource, each block file formed by block compression of the unstructured resource, the metadata information, the metadata index, and the resource dependency graph into a single resource package; The single resource package is converted into a target cloud platform instruction through an interface translation engine, so as to perform resource deployment on the current target cloud platform based on the target cloud platform instruction.

2. The resource packaging and deployment method according to claim 1, characterized in that: The process of converting the structured resources that the AI application depends on into target structured resources in a standardized YAML format, storing the block paths of the unstructured resources that the AI application depends on, and generating metadata indexes includes: Structural division of the full-stack AI application dependencies into structured and unstructured resources. Convert the structured resource into a target structured resource carrying metadata information in a standard YAML format; Resource path identification is performed on the unstructured resources to obtain a target unstructured resource list and corresponding metadata indexes.

3. The resource packaging and deployment method according to claim 1, characterized in that: Before packaging the target structured resource, the block files formed by compressing the unstructured resource blocks, the metadata information, the metadata index, and the resource dependency graph into a single resource package, the method further includes: Determining whether the total resource size of the unstructured resources exceeds a preset file size threshold; If yes, the unstructured resource is divided into blocks according to a preset block size to obtain block resources; Compressing each of the block resources based on a preset compression algorithm and according to preset compression parameters to obtain each block file; Performing a hash operation on each of the block files to obtain standard verification information of each of the block files; The standard verification information and the preset compression parameters are added to the metadata index.

4. The resource packaging and deployment method according to claim 3, characterized in that: The process of packaging the target structured resource, the block files formed by compressing the unstructured resource blocks, the metadata information, the metadata index, and the resource dependency graph into a single resource package further includes: Performing a hash operation on each of the block files to obtain verification information to be verified for the corresponding block file; Extracting standard verification information of each of the block files from the metadata index, and detecting whether the standard verification information is consistent with the verification information to be verified; If they are consistent, the block file passes the integrity check and the packaging process is started; If they are inconsistent, an error will be reported directly and the packaging process will be stopped.

5. The resource packaging and deployment method according to claim 1, characterized in that: The converting of the single resource package into target cloud platform instructions by the interface translation engine includes: The platform parameters of the target structured resources in the single resource package are mapped to target platform interface parameters under the target cloud platform through an interface translation engine.

6. The resource packaging and deployment method according to claim 5, characterized in that: Mapping the platform parameters of the target structured resources in the single resource package to target platform interface parameters under the target cloud platform through the interface translation engine includes: Extracting a node attribute set of a target structured resource from the resource dependency graph through an interface translation engine; wherein the node attribute set includes a resource type identifier, associated dependency relationships, and configuration constraints; A predefined platform parameter rule library is queried based on the node attribute set, so as to map the platform parameters of the target structured resource to target platform interface parameters under the target cloud platform according to the matching target mapping rules.

7. The resource packaging and deployment method according to claim 6, characterized in that: After querying a predefined platform parameter rule base based on the node attribute set, the method further includes: If there are conflicting items among the matching results of the resource type identifier, the matching results of the associated dependency, and the matching results of the configuration constraint, the optimal matching result is selected according to the resource priority in the resource dependency graph to execute the corresponding platform parameter mapping step; If there is no match, the self-healing mechanism is triggered to generate a degradation plan and update the resource dependency graph.

8. The resource packaging and deployment method according to claim 6, characterized in that: Also includes: Updating resource node identifiers based on the resource dependency graph; The resource dependency graph is reconstructed based on the updated resource node identifiers to obtain a reconstructed resource dependency graph.

9. The resource packaging and deployment method according to claim 8, characterized in that: The deploying resources on the current target cloud platform based on the target cloud platform instruction includes: Based on the target platform interface parameters, the resource node identifier, and the reconstructed resource dependency graph, a resource deployment operation of the full-stack dependent resources of the AI application is performed on the current target cloud platform.

10. The resource packaging and deployment method according to claim 9, characterized in that: The resource deployment operation of executing the full-stack dependent resources of the AI application on the current target cloud platform based on the target platform interface parameters, the resource node identifier, and the reconstructed resource dependency graph includes: Generate a creation sequence of full-stack dependent resources of the AI application adapted to the current target cloud platform based on the reconstructed resource dependency graph; Execute resource deployment operations for the AI application full-stack dependent resources on the current target cloud platform according to the target platform interface parameters, the resource node identifier, and the creation order of the AI application full-stack dependent resources.

11. The resource packaging and deployment method according to claim 1, characterized in that: The structured resources of the AI application are infrastructure as a service resources and platform as a service components; the unstructured resources of the AI application are AI model files and knowledge base data sets.

12. The resource packaging and deployment method according to any one of claims 1 to 11, characterized in that: Also includes: Capture the return information of the target cloud platform interface during resource deployment; If there is a timeout error in the returned information, the process will automatically retry and wait with exponential backoff; If there is a configuration conflict error in the returned information, check the associated dependencies according to the resource dependency graph and roll back in batches; If an unavailable error exists in the returned information, switching to a backup area and reconstructing the resource node identifier; Successful or failed resources are marked on the resource dependency graph to generate a visual resource deployment progress topology graph.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the resource packaging and deployment method according to any one of claims 1 to 12 are implemented.

14. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the resource packaging and deployment method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the resource packaging and deployment method according to any one of claims 1 to 12.

Images