Application deployment method and device based on cloud computing power and storage medium

Abstract

The invention discloses an application deployment method and device based on cloud computing power and a storage medium, and relates to the technical field of digital data processing, the application deployment method based on cloud computing power comprises the following steps: in response to a demand form and session data, extracting demand features in the demand form and the session data; matching a training and pushing model and training and pushing data according to the demand features; deploying a development environment based on the training and pushing model and the training and pushing data; in response to the training and pushing operation of the development environment, calling the appointment computing power of a user corresponding to the demand form, generating a special model, and packaging the special model into a model encryption deployment package; and deploying the model encryption deployment package to front and rear end applications. The technical effect of accelerating landing of the artificial intelligence application can be achieved.

Description

Technical Field

[0001] This application relates to the technical field of digital data processing, and particularly to an application deployment method, device, and storage medium based on cloud computing power. Background Art

[0002] With the rapid development of artificial intelligence technology, the deep integration of AI (Artificial Intelligence) and hardware has become an important trend in digital transformation. From AI health assistants that continuously monitor physiological data, to intelligent devices in industrial scenarios that collect high-precision sensor information, to AI material generation in the creative field and round-the-clock AI customer service systems, AI applications are gradually penetrating into vertical fields such as healthcare, manufacturing, and entertainment.

[0003] However, there is a gap in the collaborative adaptation between algorithms and hardware. For example, AI models need to optimize computing power allocation for specific hardware. Edge computing devices, for instance, are limited by power consumption and are difficult to support large neural networks. At the same time, hardware iteration often leads to algorithm compatibility issues. Changes in the data formats of new sensors may cause model failures, forcing developers to repeatedly adjust the model architecture.

[0004] Therefore, the software development cycle of AI applications is significantly longer than that of traditional software, making it difficult to quickly form competitiveness. Summary of the Invention

[0005] The main objective of this application is to provide an application deployment method, device, and storage medium based on cloud computing power, aiming to solve the technical problem that the software development cycle of AI applications is significantly longer than that of traditional software and it is difficult to quickly form competitiveness.

[0006] To achieve the above objective, this application provides an application deployment method based on cloud computing power, which is applied to a computing power leasing platform. The application deployment method based on cloud computing power includes: Responding to a requirement form and negotiation materials, and extracting requirement features from the requirement form and the negotiation materials; Matching a training and inference model and training and inference data according to the requirement features; Deploying a development environment based on the training and inference model and the training and inference data; Responding to the training and inference operation of the development environment, invoking the reserved computing power of the user corresponding to the requirement form, generating a dedicated model, and packaging it into a model encryption deployment package; Deploying the model encryption deployment package to the front-end and back-end applications.

[0007] In one embodiment, the step of responding to a requirement form and negotiation materials and extracting requirement features from the requirement form and the negotiation materials includes: Perform speech recognition on the meeting materials to generate unstructured text data; Through a pre-trained multimodal alignment model, perform cross-modal semantic association on the unstructured text data and the structured field data of the requirement form, where the multimodal alignment model uses a bidirectional attention mechanism to calculate the association matrix between form fields and dialogue texts; According to the association matrix, screen key semantic segments to generate the requirement features including the hardware type identifier, computing power requirement level, and data security requirements, where the hardware type identifier includes the processor architecture, memory capacity, and interface protocol feature encoding of the target device.

[0008] In one embodiment, before the step of performing cross-modal semantic association on the unstructured text data and the structured field data of the requirement form through a pre-trained multimodal alignment model, it includes: Construct a labeled hardware requirement corpus, where the training data of the hardware requirement corpus includes a triple of form fields, dialogue texts, and corresponding hardware type identifiers; Adopt a contrastive learning loss function to constrain the cosine similarity of form-dialogue pairs of the same hardware type in the feature space to be higher than that of heterogeneous samples; Based on an adversarial sample generation enhancement model, enhance the parsing robustness of fuzzy expressions to obtain the multimodal alignment model.

[0009] In one embodiment, the step of matching the training and inference model and the training and inference data according to the requirement features includes: Parse the hardware type identifier in the requirement features, and screen candidate training and inference models based on the processor architecture compatibility map, where the map pre-stores the mapping relationship of the support status of neural network operators on different hardware platforms; Perform dynamic pruning on the candidate training and inference models according to the computing power requirement level to generate the training and inference model that meets the memory capacity constraint of the target device; Based on the data security requirements, construct a privacy computing channel, extract initial data matching the requirement scenario label from the distributed data source, and generate the training and inference data protected by differential privacy through the homomorphic encryption algorithm.

[0010] In one embodiment, the step of deploying the development environment based on the training and inference model and the training and inference data includes: According to the framework type of the training and inference model, match the corresponding version of the runtime dependency library from the preset container template library, and construct a basic container image including a framework interpreter and a model conversion tool chain; Parse the interface protocol feature encoding of the hardware type identifier in the requirement features, and inject the target hardware driver layer into the basic container image to generate a training and inference execution environment adapted to the hardware. Mount the training and inference data with differential privacy protection to the container storage volume, and automatically generate a data preprocessing script that matches the input and output formats of the training and inference model; Configure environment variable parameters, including the model service port number, computing power resource quota, and monitoring probe collection frequency, to complete the development environment deployment.

[0011] In one embodiment, the step of, in response to the training and inference operation of the development environment, invoking the demand form to allocate the reserved computing power for the corresponding user, generating a dedicated model, and encapsulating it into a model encryption deployment package includes: Based on the container runtime environment of the development environment, parse the dynamic scheduling policy of the reserved computing power, and generate resource allocation parameters and computing power constraint conditions; According to the resource allocation parameters, dynamically adjust the training batch size and video memory occupancy ratio through a resource allocation algorithm driven by reinforcement learning, and output gradient update parameters that adapt to the target hardware computing power threshold; Based on the gradient update parameters and the data privacy requirements marked in the demand form, dynamically allocate the differential privacy budget for the training data set through the privacy computing gateway, and generate a sequence of feature tensors with privacy protection labels; Adopt a homomorphic encryption algorithm to perform layer-by-layer encryption processing on the model weights corresponding to the sequence of feature tensors, where the encryption key is generated by mixing two factors using the user identity identifier and the hardware device fingerprint in the demand form; Inject the encrypted model weights into the hardware acceleration module based on the trusted execution environment, and integrate it with the inference driver library corresponding to the gradient update parameters to generate an initial deployment package; Perform a runtime environment simulation on the initial deployment package according to the computing power constraint conditions, and generate the model encryption deployment package containing the national cryptography algorithm verification code.

[0012] In one embodiment, the step of deploying the model encryption deployment package to the front-end and back-end applications includes: Parse the integrity verification code in the model encryption deployment package, verify the integrity of the deployment package through the national cryptography algorithm, and synchronously verify the consistency between the hardware device fingerprint and the registered device information in the demand form; Based on the hardware acceleration module of the trusted execution environment, create an isolated model running sandbox on the target device, and load the encrypted model weights and the inference driver library, where the sandbox is configured to only allow access to the training and inference data through the privacy computing channel; Dynamically select the training and inference model according to the characteristics of the front-end application interface protocol, where the training and inference model includes an operator acceleration library that matches the target processor architecture, and inject the model weights into the pre-allocated video memory area through the memory mapping technology; Deploy a gateway in the backend service container, configure an input data format converter and an output result encryption module, and the converter automatically adjusts the concurrent thread number of batch inference according to the computing power requirement level in the requirement characteristics.

[0013] In one embodiment, before the step of, in response to the training and inference operation in the development environment, calling the requirement form to reserve the computing power for the user, generating a dedicated model and encapsulating it into a model encryption deployment package, it includes: Analyze the computing power requirement level, task concurrency peak and model complexity parameters in the requirement form to generate initial computing power resource allocation parameters; Based on the computing power consumption records in the historical training database, calculate the computing power requirement baseline value of the target scenario through a time series prediction model, where the baseline value includes three-dimensional constraint indicators of video memory occupancy rate, floating-point operation volume and memory bandwidth utilization rate; According to the hardware architecture identifier in the requirement form, match the processor instruction set support map, and filter the candidate computing power types that meet the computing power requirement baseline value; Based on the reinforcement learning algorithm, perform Pareto optimization on the resource utilization rate and energy consumption ratio of the candidate computing power types to generate a dynamic scheduling strategy including a computing power threshold interval, a time window constraint and an elastic scaling rule; Bind the dynamic scheduling strategy to the target hardware resource pool to generate a timestamped reserved computing power identifier to identify the reserved computing power.

[0014] In addition, to achieve the above object, the present application also provides an application deployment device based on cloud computing power, and the application deployment device based on cloud computing power includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the application deployment method based on cloud computing power as described above.

[0015] In addition, to achieve the above object, the present application also provides a storage medium, the storage medium is a computer-readable storage medium, and a program for implementing the application deployment method based on cloud computing power is stored on the computer-readable storage medium, and the program for implementing the application deployment method based on cloud computing power is executed by a processor to implement the steps of the application deployment method based on cloud computing power as described above.

[0016] The present application provides a method for deploying an application based on cloud computing power. First, the present application extracts the requirement features in the requirement form and the negotiation materials by responding to the requirement form and the negotiation materials; matches the training and inference model and the training and inference data according to the requirement features; deploys the development environment based on the training and inference model and the training and inference data; in response to the training and inference operation of the development environment, calls the requirement form to allocate the reserved computing power of the user, generates a dedicated model and encapsulates it into a model encryption deployment package; deploys the model encryption deployment package to the front-end and back-end applications, solves the technical problem in the related art that the software development cycle of AI applications is greatly extended compared with traditional software and it is difficult to quickly form competitiveness, and realizes the technical effect of accelerating the implementation of AI applications. Description of the Drawings

[0017] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a schematic flowchart provided for Embodiment 1 of the method for deploying an application based on cloud computing power of the present application; Figure 2 It is a schematic flowchart provided for Embodiment 5 of the method for deploying an application based on cloud computing power of the present application; Figure 3 It is a schematic flowchart provided for Embodiment 6 of the method for deploying an application based on cloud computing power of the present application; Figure 4 It is a schematic hardware structure diagram related to Embodiment of the device for deploying an application based on cloud computing power of the present application.

[0020] The realization of the purpose, the functional features and the advantages of the present application will be further described in conjunction with the embodiments with reference to the drawings. Detailed Embodiments

[0021] It should be understood that the specific embodiments described here are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0022] In order to better understand the technical solutions of the present application, the following will be described in detail in conjunction with the specification drawings and the specific embodiments.

[0023] Currently, there is a gap in the co - adaptation of algorithms and hardware. For example, AI models need to optimize computing power allocation for specific hardware. Edge computing devices, limited by power consumption, are difficult to support large - scale neural networks. At the same time, hardware iteration often leads to algorithm compatibility problems. Changes in the data format of new sensors may cause model failure, forcing developers to repeatedly adjust the model architecture. Therefore, the software development cycle of AI applications is greatly extended compared to traditional software, making it difficult to quickly form competitiveness.

[0024] The main solution of this application is: in response to a requirement form and negotiation materials, extract the requirement features in the requirement form and the negotiation materials; match a training and inference model and training and inference data according to the requirement features; deploy a development environment based on the training and inference model and the training and inference data; in response to the training and inference operation of the development environment, call the reserved computing power of the user corresponding to the requirement form, generate a dedicated model and package it into a model encryption deployment package; deploy the model encryption deployment package to the front - end and back - end applications. The technical effect of accelerating the implementation of AI applications is achieved.

[0025] It should be noted that the execution subject of this embodiment can be a computing power leasing platform, or a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an application deployment device based on cloud computing power that can implement the above functions. This embodiment does not make specific limitations in this regard. Hereinafter, taking the computing power leasing platform as the execution subject as an example, this embodiment and the following embodiments will be described.

[0026] Based on this, Embodiment 1 of this application proposes an application deployment method based on cloud computing power. Please refer to Figure 1 The application deployment method based on cloud computing power includes steps S10 - S50: Step S10, in response to a requirement form and negotiation materials, extract the requirement features in the requirement form and the negotiation materials.

[0027] In this embodiment, the requirement form refers to the structured data filled in by the user on the computing power leasing platform, including fields such as hardware type, computing power requirement level, data security level, etc. The negotiation materials refer to the technical communication records between the user and the platform, including unstructured data in the form of voice, text, or video.

[0028] As an alternative implementation, the conversation materials are converted into text through speech recognition technology (e.g., real-time transcription based on WebRTC (Web Real-Time Communication)), and then the structured form fields and unstructured text of the requirement form are cross-modally semantically associated through a pre-trained multi-modal alignment model such as a CLIP (Contrastive Language-Image Pre-training) variant. For example, when the user orally mentions "a GPU (Graphics Processing Unit) that supports AVX512 (Advanced Vector Extensions 512)", the system will automatically map to the "processor architecture" field of the form and generate a coding identifier.

[0029] Step S20: Match the training and inference model and the training and inference data according to the requirement features.

[0030] In this embodiment, the training and inference model refers to a machine learning model adapted to the target hardware architecture, such as TensorFlowLite or ONNX Runtime (Open Neural Network Exchange Runtime), and the training and inference data refers to a training data set protected by privacy, which needs to meet the requirements of differential privacy or homomorphic encryption.

[0031] As an alternative implementation, based on the hardware type identifiers in the requirement features, such as the number of NVIDIA CUDA (Compute Unified Device Architecture) cores and the support status of the ARM NEON (Advanced SIMD extension) instruction set, candidate models are screened from a pre-built processor architecture compatibility map. For example, if the requirement features include "FP16 (16-bit Floating Point) precision support", the model versions that only support FP32 (32-bit Floating Point) are automatically excluded, and the model layers are dynamically pruned to adapt to the memory capacity of the target device.

[0032] Step S30: Deploy a development environment based on the training and inference model and the training and inference data.

[0033] In this embodiment, the development environment includes a containerized runtime environment, a hardware driver layer, and a data preprocessing toolchain.

[0034] As an alternative implementation, pull the base image that matches the training and inference model framework from the container template library, such as PyTorch 2.1 + CUDA 11.7, and inject the target hardware driver. By mounting an encrypted storage volume, for example, loading differential privacy data using a block device encrypted with LUKS (Linux Unified Key Setup), and automatically generating a data preprocessing script, such as normalizing image data into a tensor format.

[0035] Step S40: In response to the training and inference operation of the development environment, call the demand form to allocate the reserved computing power for the user, generate a dedicated model, and package it into a model encryption deployment package.

[0036] In this embodiment, the reserved computing power refers to the cloud computing resource quota pre-applied by the user, including GPU video memory, floating-point operation volume, and time window constraints. The model encryption deployment package includes an executable file with encrypted model weights, an inference driver library, and an integrity verification code.

[0037] As an alternative implementation, dynamically adjust the training batch size through a reinforcement learning algorithm such as PPO (Proximal Policy Optimization) to ensure that the video memory occupancy rate does not exceed 80% of the reserved computing power. Use a homomorphic encryption algorithm such as CKKS (Cheon-Kim-Kim-Song) to encrypt the model weights layer by layer. The encryption key is generated by mixing the user ID (Identifier) and the device fingerprint, such as the TPM (Trusted Platform Module) chip identifier. The final deployment package generates a verification code through the SM3 (commercial cryptographic algorithm standard) national cryptographic algorithm to ensure transmission integrity.

[0038] Step S50: Deploy the model encryption deployment package to the front-end and back-end applications.

[0039] In this embodiment, the front-end application refers to the user interface, which needs to be adapted to a lightweight inference engine. The back-end application refers to the model inference service on the cloud server, which needs to support high concurrency and elastic scaling.

[0040] As an alternative implementation, a lightweight inference engine subset is dynamically loaded at the front end, such as TensorFlow.js or Core ML (Core Machine Learning). Encrypted weights are injected into the pre-allocated video memory area through memory mapping technology. At the back end, a RESTful API (Representational State Transfer Application Programming Interface) gateway is deployed, a batch inference thread pool is configured, the concurrency is automatically adjusted according to the computing power requirement level, and an exception circuit breaker module is integrated to monitor the inference latency and resource occupancy in real time.

[0041] Exemplarily, in a deployment scenario of a smart medical imaging analysis system The user fills out a structured form, selects the hardware type as "NVIDIA A100 GPU cluster" (supporting AVX512 instruction set), the computing power requirement level as "high-performance computing", and the data security requirement as "HIPAA (Health Insurance Portability and Accountability Act) compliance". The platform technicians discuss with the user the data scale record: a 200TB lung CT image dataset, the model requirements (the 3D U-Net model needs to support FP16 precision), and the interface protocol (RESTful API latency ≤ 200ms), extract the key parameters through a speech recognition engine, and associate the form fields to generate a "hardware architecture identifier: CUDA 11.8 + TensorRT (Tensor Runtime) 8.6".

[0042] Based on LSTM (Long Short-Term Memory) prediction, 80 A100 graphics cards need to be reserved. A resource allocation plan is generated through reinforcement learning, with 64 cards in the training phase and elastically scaled down to 16 cards in the inference phase, and the energy consumption ratio is optimized by 23%. Deploy a PyTorch 2.1 + CUDA 11.8 container image, configure the NVLink (NVIDIA high-speed interconnection technology) driver, and achieve cross-node gradient synchronization through the RDMA (Remote Direct Memory Access) network.

[0043] Use a two-factor key to encrypt the model weights and inject the TEE (Trusted Execution Environment) to isolate the medical data access path. The backend Kubernetes cluster is configured with an automatic scaling policy (CPU / GPU utilization threshold of 80%), and the front end integrates TensorFlow.js to achieve medical digital imaging and communication preview on the browser side. The final requirement analysis time is shortened to 15 minutes, the GPU fragmentation rate is reduced to 3.2%, and it passes the HIPAA audit.

[0044] In this embodiment, by responding to the requirement form and the meeting materials, extract the requirement features in the requirement form and the meeting materials; match the training and inference model and the training and inference data according to the requirement features; deploy the development environment based on the training and inference model and the training and inference data; in response to the training and inference operation of the development environment, call the reserved computing power of the user corresponding to the requirement form, generate a dedicated model and package it into a model encryption deployment package; deploy the model encryption deployment package to the front-end and back-end applications, significantly reducing the development time cost of the AI application.

[0045] Based on any of the above embodiments, in the second embodiment of the present application, step S10 includes: Step S11, perform speech recognition on the meeting materials to generate unstructured text data.

[0046] In this embodiment, the meeting materials refer to the real-time communication recording between the user and the technical staff of the computing power leasing platform, which needs to be converted into a processable text format. Unstructured text data refers to the content of the conversation without a predefined format, including requirement information described in natural language.

[0047] As an optional implementation, use a Whisper-based speech recognition engine to transcribe the recording into text in real time, and use natural language processing (NLP) technology to remove redundant tone words and extract structured fields such as "FP16 precision support" and "200ms response latency".

[0048] Step S12, through a pre-trained multimodal alignment model, perform cross-modal semantic association on the unstructured text data and the structured field data of the requirement form, where the multimodal alignment model uses a bidirectional attention mechanism to calculate the association matrix between the form fields and the conversation text.

[0049] In this embodiment, the multimodal alignment model refers to a deep learning model that integrates structured form fields, such as "hardware type" and "computing power requirement level", with unstructured conversation text. The bidirectional attention mechanism means that the model calculates the correlation matrix between the form fields and the conversation text to achieve bidirectional semantic alignment. For example, the weight matching between the "processor architecture" field in the form and the description of "requiring AVX512 instruction set" in the conversation.

[0050] As an alternative implementation, in the feature space, calculate the attention weights for the "memory capacity" field in the form and descriptions such as "at least 64GB video memory" in the conversation to generate a correlation matrix, and filter out semantic segments with a matching degree higher than 0.8.

[0051] Step S13: Filter key semantic segments according to the correlation matrix to generate the requirement features including the hardware type identifier, computing power requirement level, and data security requirements, where the hardware type identifier includes the processor architecture, memory capacity, and interface protocol feature encoding of the target device.

[0052] In this embodiment, the hardware type identifier includes the encoding combination of the processor architecture, memory capacity, and interface protocol of the target device. The data security requirement refers to the privacy protection level selected by the user.

[0053] As an alternative implementation, perform encoding mapping on the filtered semantic segments. For example, encode "NVIDIA A100 GPU" as "CUDA11.8 + TensorRT 8.6", and generate a requirement feature file in JSON format in combination with the computing power requirement level such as HPC level.

[0054] This embodiment significantly shortens the requirement analysis cycle through automatic speech recognition and intelligent association of structured and unstructured data, reduces the risk of resource misallocation caused by manual misjudgment, and at the same time meets the privacy compliance requirements of high-security scenarios such as medical and financial fields.

[0055] Optionally, before step S12, it includes: Step A10: Construct a labeled hardware requirement corpus, and the training data of the hardware requirement corpus includes a triple of form fields, conversation text, and the corresponding hardware type identifier.

[0056] In this embodiment, each piece of training data in the hardware requirement corpus consists of three parts, namely: form fields: structured data filled in by users, such as "Processor Architecture: NVIDIA A100". Dialogue text: unstructured communication records between users and platform technicians, such as the speech-to-text "GPU that needs to support CUDA 11.8". Hardware type identifier: a code that uniquely identifies the hardware configuration, such as "NV-A100-80G-PCIe4". Triplet relationship: the three need to strictly correspond. For example, "Memory Capacity: 64GB" in the form needs to be associated with "At least 64GB video memory" in the dialogue to the same hardware identifier.

[0057] As an alternative implementation, a semi-automated annotation process is adopted. First, form field keywords are extracted through regular expressions, then hardware parameters are extracted from the dialogue text in combination with NLP technology, and finally triples are generated by expert review.

[0058] For example, a piece of training data can be expressed as: json { "form fields": {"Processor Architecture": "NVIDIA A100", "Memory Capacity": "64GB"}, "dialogue text": "Need a graphics card that supports CUDA 11.8, and the video memory is not less than 64G", "hardware type identifier": "NV-A100-64G-CUDA11.8" } Step A20, adopt a contrastive learning loss function to constrain the cosine similarity of form-dialogue pairs of the same hardware type in the feature space to be higher than that of heterogeneous samples.

[0059] In this embodiment, for the contrastive learning objective, the cosine similarity between the form fields and the dialogue text of the same hardware type in the feature space needs to be greater than 0.85, and the similarity between different hardware types needs to be less than 0.3. For the loss function design, Triplet Loss is used, and the margin is set to 0.5 to ensure that the distance between similar samples is closer and the distance between heterogeneous samples is farther.

[0060] As an alternative implementation, taking the hardware identifier of "NVIDIA A100" as an example, select its corresponding form field such as "Processor Architecture: A100" as the anchor, the positive sample is the associated dialogue text such as "Need an A100 graphics card", and the negative sample is the dialogue text of other hardware types such as "Need an AMD MI250". During model training, minimize the distance between the anchor and the positive sample, and at the same time maximize the distance between the anchor and the negative sample.

[0061] Step A30, enhance the parsing robustness of the fuzzy expressions based on the adversarial sample generation enhanced model to obtain the multimodal alignment model.

[0062] In this embodiment, the adversarial samples include fuzzy expressions and noise interference. The adversarial training method is to add the adversarial samples to the training set and iteratively optimize the model parameters so that it can still accurately associate semantics in a noisy environment.

[0063] As an alternative implementation, the fast gradient sign method is used to generate adversarial samples. For example, add perturbations to the original dialogue text "64GB video memory is required" to generate "64GB video memory is needed" or "About 64G video memory is required", forcing the model to learn to ignore spelling mistakes or fuzzy quantifiers. After adversarial training, the parsing accuracy of the model for fuzzy expressions is increased from 78% to 95%.

[0064] This embodiment significantly reduces the manual intervention in the requirement parsing process through structured triple data, contrastive learning optimization, and adversarial training.

[0065] Based on any of the above embodiments, in the third embodiment of the present application, the steps of matching the training and inference model and the training and inference data according to the requirement characteristics include: Step S21, parse the hardware type identifier in the requirement characteristics, and screen the candidate training and inference models based on the processor architecture compatibility map, where the map pre-stores the mapping relationship of the support status of neural network operators on different hardware platforms.

[0066] In this embodiment, the hardware type identifier is composed of parameters such as processor architecture encoding, memory capacity, and interface protocol, and is used to uniquely identify the hardware configuration of the target device. The processor architecture compatibility map pre-stores the support status of neural network operators on different hardware platforms. For example, NVIDIA GPUs support CUDA operators, Huawei Ascend supports CANN operators, and ARM architectures support models accelerated by the Neon instruction set.

[0067] As an alternative implementation, extract the hardware type identifier field from the requirement form and parse out sub-fields such as processor architecture, memory capacity, and interface protocol. Match these sub-fields with the processor architecture compatibility map to screen out a list of hardware platforms that support target operators such as FP16 mixed precision. Filter out models that do not meet the conditions according to hardware parameters such as memory capacity ≥ 64GB, and finally generate a list of candidate training and inference models.

[0068] Maintain the compatibility map through a dynamic update mechanism. For example, when it is detected that a certain hardware platform newly supports TensorRT8.6, automatically add the ResNet-50-TRT8.6 model to the candidate list. When screening, give priority to matching the computing power requirement level, such as "HPC level" corresponding to models that support mixed precision computing.

[0069] Step S22: Dynamically prune the candidate training and inference models according to the computing power requirement level to generate the training and inference models that meet the memory capacity constraint of the target device.

[0070] In this embodiment, the computing power requirement levels are divided into "edge computing level" (parameter quantity ≤ 5 million, memory ≤ 512 MB), "general computing level" (parameter quantity ≤ 100 million, memory ≤ 8 GB), and "HPC level" (no limit on parameter quantity). Dynamic pruning reduces the model complexity through knowledge distillation, quantization compression, and channel pruning, and verifies whether the peak memory occupancy of the pruned model on the target device meets the standard through a memory simulator.

[0071] As an alternative implementation, receive the candidate training and inference model list and the computing power requirement level as inputs. Perform static analysis on these models, calculate their floating-point operation amounts and peak memory occupancies, and set pruning thresholds according to the computing power level. Remove redundant modules such as the Dropout layer in BERT, and apply quantization techniques such as INT8 quantization to further compress the model volume. Finally, generate a set of lightweight models adapted to the target device.

[0072] For edge devices, adopt a hierarchical pruning strategy. First, remove the attention heads in the BERT-large model that are irrelevant to the downstream tasks, and then perform 8-bit quantization on the fully connected layer. Finally, reduce the model memory occupancy to meet the memory constraint of the edge device.

[0073] Step S23: Build a privacy computing channel based on data security requirements, extract the initial data that matches the demand scenario label from the distributed data sources, and generate the training and inference data protected by differential privacy through a homomorphic encryption algorithm.

[0074] In this embodiment, the privacy computing channel is designed based on the zero-trust architecture, and an encrypted link is established with distributed data sources such as medical image libraries and financial transaction libraries through TLS mutual authentication to ensure the integrity of the data transmission process. Differential privacy protection uses the Paillier homomorphic encryption algorithm to encrypt sensitive fields such as patient IDs, and adds Laplace noise λ = 0.1 during the data aggregation stage, so that individual records cannot be reverse-inferred.

[0075] As an alternative implementation, extract the original data such as medical images and financial transaction records that match the demand scenario label from the distributed database, and establish a secure data transmission channel through TLS 1.3 encryption. Apply the Paillier homomorphic encryption to sensitive fields such as patient IDs and transaction amounts to ensure the confidentiality of the data during storage and transmission. Add Gaussian noise during the data aggregation process to achieve differential privacy protection and meet the GDPR compliance requirements. Finally, generate an encrypted training and inference dataset.

[0076] Extract initial data that matches the demand scenario tags from distributed nodes, perform data alignment through the federated learning framework, and generate a differentially private dataset. During the training process, the decrease in the model accuracy is at a relatively low level, meeting the compliance requirements.

[0077] In this embodiment, through the triple technology linkage of hardware adaptation, model lightweighting, and privacy computing, the balance between the end-to-end efficiency and security of the training and inference system is achieved, and the deployment cost in high-sensitivity scenarios is reduced.

[0078] Based on any of the above embodiments, in the fourth embodiment of the present application, the steps of deploying a development environment based on the training and inference model and the training and inference data include: Step S31, according to the framework type of the training and inference model, match the corresponding version of the runtime dependency library from a preset container template library, and construct a basic container image including a framework interpreter and a model conversion toolchain.

[0079] In this embodiment, the framework type of the training and inference model refers to the deep learning framework based on the model submitted by the user, including but not limited to TensorFlow, PyTorch, ONNX, etc. The framework type determines the computing libraries, toolchains, and compatibility requirements required for model operation. Container template library: Pre-configured standardized runtime environment images for multiple AI frameworks, including dependency libraries such as corresponding versions of CUDA, cuDNN, NCCL, etc., and components such as framework interpreters like Python interpreters and model conversion toolchains like ONNXRuntime, TensorRT, etc., for quickly generating container images adapted to the target framework. Basic container image: A Docker image containing a framework interpreter, a model conversion toolchain, and runtime dependency libraries, serving as the underlying operation basis for the training and inference environment, and supporting model loading, inference, and conversion operations.

[0080] As an alternative implementation, extract the framework type field of the training and inference model from the demand form, parse out the framework name and version number; at the same time, query the metadata index of the container template library to obtain a list of image templates that match the framework type. According to the framework version number, filter the image templates that support this version from the template library. Dynamically add the runtime dependency libraries and toolchains required by the target framework to the base image. Perform multi-stage construction on the image, remove unnecessary debug files and intermediate products, and finally generate a lightweight basic container image. Generate a basic container image containing a framework interpreter, a model conversion toolchain, and runtime dependency libraries, and store it in a private image repository for subsequent deployment of the training and inference environment.

[0081] Through the preset container template library and the automated matching mechanism, the construction time of the basic container image is shortened, and the image volume is reduced, significantly improving the initialization efficiency of the training and inference environment, and at the same time ensuring the compatibility between the framework version and the hardware platform.

[0082] Step S32: Analyze the interface protocol feature encoding of the hardware type identifier in the demand feature, inject the target hardware driver layer into the base container image, and generate a training and inference execution environment adapted to the hardware.

[0083] In this embodiment, the hardware type identifier: includes hardware feature information such as processor architecture, interface protocol, and memory capacity, and is used to describe the computing power and connection characteristics of the target device. The target hardware driver layer: supports the driver programs and operator libraries for specific hardware to ensure that the model can efficiently utilize the hardware resources.

[0084] As an alternative implementation, analyze the hardware type identifier field in the demand form, extract sub-fields such as processor architecture, interface protocol, and memory capacity. Determine the driver requirements of the target hardware according to the interface protocol feature encoding. Inject the driver library of the target hardware into the base container image through the dynamic library loading mechanism, and configure the environment variable to point to the driver path. Run the hardware compatibility test script to verify the adaptability of the driver layer and the base image, and ensure that the model can normally call the hardware acceleration function. Generate a training and inference execution environment adapted to the target hardware, including a complete driver layer and an optimized base image, to support hardware-accelerated inference.

[0085] Through the dynamic driver injection mechanism, the adaptability success rate of the training and inference execution environment for heterogeneous hardware such as GPUs and NPUs is improved, and the inference performance is improved compared to the general image.

[0086] Step S33: Mount the training and inference data protected by differential privacy to the container storage volume, and automatically generate a data preprocessing script that matches the input and output formats of the training and inference model.

[0087] In this embodiment, the differentially private protected data: the training data set protected by homomorphic encryption or noise injection technology to ensure the privacy of the data during transmission and calculation. The data preprocessing script: an automated script that converts encrypted data into the model input format, supporting automated feature extraction, normalization, and other operations.

[0088] As an alternative implementation, mount the encrypted training and inference data volume from the distributed storage system, and analyze the encryption format and storage structure of the data. Load the encrypted data through the decryption module in the container and verify the integrity of the data. Automatically generate a preprocessing script according to the input layer configuration of the training and inference model, including operation logics such as data decryption, normalization, and bucketing. Perform differential privacy verification on the decrypted data to ensure the indistinguishability of individual records. Generate a training and inference data set and a preprocessing script that match the model input format, supporting direct use for model training and inference.

[0089] Through the automated data preprocessing mechanism, the data loading and conversion efficiency is improved, while meeting the privacy compliance requirements and reducing the risk of data leakage.

[0090] Step S34, configure the environment variable parameters, including the model service port number, computing power resource quota, and monitoring probe collection frequency, to complete the deployment of the development environment.

[0091] In this embodiment, the computing power resource quota: limits the upper limit of CPU / GPU resource usage through a container orchestration tool to ensure that the model runs within the specified hardware resource range. The monitoring probe: collects metrics such as inference latency and video memory occupancy for real-time monitoring of the model running status.

[0092] As an alternative implementation, extract parameters such as the computing power requirement level, service port, and monitoring frequency from the requirements form. Set the resource quota of the container, such as GPU video memory ≤ 32GB and CPU cores ≤ 20. Inject the monitoring module, configure the sampling frequency and the metric reporting path. Verify the legality of the environment variables to ensure correct configuration. Complete the deployment of the development environment. The container has the capabilities of resource isolation, performance monitoring, and service exposure, supporting the stable operation of training and inference tasks.

[0093] Through refined resource management and monitoring mechanisms, the computing power utilization rate is improved, the inference latency is reduced, and at the same time, real-time performance analysis and fault troubleshooting are supported.

[0094] Based on any of the above embodiments, in the fifth embodiment of the present application, referring to Figure 2 , the steps of, in response to the training and inference operation of the development environment, calling the reserved computing power of the user corresponding to the requirements form, generating a dedicated model, and encapsulating it into a model encryption deployment package include: Step S41, based on the container runtime environment of the development environment, parse the dynamic scheduling policy of the reserved computing power to generate resource allocation parameters and computing power constraint conditions.

[0095] In this embodiment, the container runtime environment of the development environment: It is a lightweight virtualization technology that can package an application program and its dependencies into independent containers and maintain a consistent running state in different environments. Common container runtime environments such as the Docker runtime provide the basic capabilities for operations such as creating, starting, and stopping containers. The dynamic scheduling policy of the reserved computing power: This is a policy that flexibly adjusts the allocation of computing power resources according to different application scenarios and time requirements. For example, allocate more computing power during peak business periods and reduce computing power allocation during low periods to improve resource utilization. Resource allocation parameters: refer to the specific parameters based on which computing power resources are scheduled, such as the number of CPU cores, the number of GPUs, and the memory size, used to clarify the amount of resources that each task can use. Computing power constraint conditions: The restrictive conditions for the use of computing power, such as the maximum allowable power consumption and specific hardware model requirements, to ensure that the use of computing power complies with specific specifications and requirements.

[0096] As an alternative implementation, first, in the container runtime environment of the development environment, obtain the current resource status information through container management tools such as Kubernetes. Then, parse the dynamic scheduling policy for the user's reserved computing power in the requirement form, which may exist in the form of a configuration file or code. According to the parsing result and combined with the current resource status, generate specific resource allocation parameters, such as allocating 4 CPU cores and 1 GPU for a certain training task. At the same time, determine the computing power constraint conditions according to the user's requirements and hardware characteristics, such as the maximum power consumption cannot exceed 200 watts.

[0097] Step S42: According to the resource allocation parameters, dynamically adjust the training batch size and the video memory occupancy ratio through a resource allocation algorithm driven by reinforcement learning, and output gradient update parameters adapted to the target hardware computing power threshold.

[0098] In this embodiment, the resource allocation algorithm driven by reinforcement learning: Reinforcement learning is a machine learning method in which an agent interacts with the environment and continuously tries different behaviors to maximize the cumulative reward. In the resource allocation scenario, the agent dynamically adjusts the resource allocation policy according to the current resource status and task requirements to achieve the optimal resource utilization effect. Training batch size: In the process of deep learning training, the number of samples selected from the training dataset each time. An appropriate batch size can affect the training speed and the performance of the model. Video memory occupancy ratio: The ratio of the GPU video memory occupied by the model during training. Reasonably controlling the video memory occupancy ratio can avoid video memory overflow and improve the training efficiency. Gradient update parameters: In deep learning, gradient update is a key step in optimizing model parameters. The gradient update parameters determine the amplitude and direction of the model parameter update in each iteration. Target hardware computing power threshold: The maximum computing power load that the target hardware can withstand. Exceeding this threshold may cause a decline in hardware performance or a failure.

[0099] As an alternative implementation, input the generated resource allocation parameters into the resource allocation algorithm driven by reinforcement learning. This algorithm uses the training batch size and the video memory occupancy ratio as adjustable actions, and uses the training speed, accuracy, and resource utilization rate of the model as the reward function. By continuously interacting with the training environment, the algorithm dynamically adjusts the training batch size and the video memory occupancy ratio so that the training process of the model can adapt to the computing power threshold of the target hardware. Finally, output the optimal gradient update parameters under the current resource allocation for subsequent model training.

[0100] Step S43: Based on the gradient update parameters and the data privacy requirements marked in the requirement form, dynamically allocate the differential privacy budget for the training dataset through the privacy computing gateway, and generate a sequence of feature tensors with privacy protection labels.

[0101] In this embodiment, data privacy requirements refer to the requirements for the privacy protection of training data clearly put forward by users in the requirement form, such as the degree of data anonymization, data access rights, etc. A privacy computing gateway is a middleware for protecting data privacy. It can encrypt, desensitize, etc. data, and at the same time support data calculation and analysis under the premise of privacy protection. Differential privacy budget: Differential privacy is a technology for protecting data privacy. The differential privacy budget is used to control the amount of noise that can be introduced during data publication to balance data availability and privacy. Feature tensor sequence: In deep learning, data is usually represented in the form of tensors. A feature tensor sequence refers to a tensor sequence formed by arranging the features in the training dataset in a certain order. Privacy protection label: A label added to each feature tensor to identify whether the tensor has undergone privacy protection processing and the processing method.

[0102] As an alternative implementation, according to the obtained gradient update parameters and the data privacy requirements marked in the requirement form, the privacy computing gateway first analyzes the training dataset to determine the sensitivity of each data sample. Then, based on the principle of differential privacy, the differential privacy budget is dynamically allocated to introduce different degrees of noise to data samples with different sensitivities. During the processing, a privacy protection label is added to each feature tensor to indicate what kind of privacy protection processing the tensor has undergone. Finally, a feature tensor sequence with privacy protection labels is generated for subsequent model training.

[0103] Step S44, layer-by-layer encrypt the model weights corresponding to the feature tensor sequence using a homomorphic encryption algorithm, where the encryption key is generated by two-factor mixing of the user identity identifier in the requirement form and the hardware device fingerprint.

[0104] In this embodiment, homomorphic encryption algorithm: A special encryption algorithm that allows specific calculations to be performed on encrypted data without decrypting the data first. The calculation result is the same as the result of performing the same calculation on the plaintext data after decryption. Model weights: In a deep learning model, weights are the core parameters of the model, which determine the learning ability and performance of the model. User identity identifier: Information used to uniquely identify a user, such as a username, user ID, etc. Hardware device fingerprint: A unique identifier composed of various characteristic information of the hardware device, such as CPU model, MAC address, etc. Two-factor mixing generation: Using the user identity identifier and the hardware device fingerprint as two factors, and mixing them through a specific algorithm to generate an encryption key to increase the security of the key.

[0105] As an alternative implementation, for the model weights corresponding to the generated sequence of feature tensors with privacy protection tags, a homomorphic encryption algorithm is used for layer-by-layer encryption. During the encryption process, first, the user identity identifier is obtained from the requirement form, and at the same time, the hardware device fingerprint is obtained through a hardware information collection tool. Then, these two factors are input into a pre-designed key generation algorithm to generate an encryption key. The encryption key is used to encrypt each layer of the model weights to ensure the security of the model weights during transmission and storage.

[0106] Step S45: Inject the encrypted model weights into the hardware acceleration module based on the trusted execution environment and integrate them with the inference driver library corresponding to the gradient update parameters to generate an initial deployment package. In this embodiment, the trusted execution environment is a hardware security technology that provides an isolated execution environment where the code and data running in it are protected from external malware attacks. The hardware acceleration module is a hardware device specifically used to accelerate computing tasks, such as GPUs, FPGAs, etc. The inference driver library is a software library used to drive the hardware acceleration module for model inference, which provides interfaces and functions for interacting with the hardware device. The initial deployment package is a data packet containing the encrypted model weights, the inference driver library, and related configuration information for subsequent model deployment.

[0107] As an alternative implementation, the encrypted model weights are injected into the hardware acceleration module based on the trusted execution environment. This hardware acceleration module provides a secure storage and execution environment to ensure that the encrypted model weights are not leaked. At the same time, the inference driver library corresponding to the gradient update parameters obtained in step S42 is integrated with the encrypted model weights. The inference driver library is responsible for interacting with the hardware acceleration module to implement the model inference function. Finally, the encrypted model weights, the inference driver library, and related configuration information are packaged into an initial deployment package to prepare for subsequent deployment.

[0108] Step S46: Perform a runtime environment simulation on the initial deployment package according to the computing power constraint conditions to generate the model encrypted deployment package containing the national cryptography algorithm verification code.

[0109] In this embodiment, the runtime environment simulation is to test and verify the initial deployment package by simulating the actual runtime environment to ensure that it can work properly in actual operation. The national cryptography algorithm verification code: The national cryptography algorithm is a cryptography algorithm independently developed in our country, such as SM2, SM3, SM4, etc. The national cryptography algorithm verification code is the verification value obtained by performing a hash calculation on the initial deployment package using the national cryptography algorithm, which is used to verify the integrity and authenticity of the deployment package. The model encrypted deployment package is a deployment package that has been encrypted and contains the national cryptography algorithm verification code for secure deployment to the target environment.

[0110] As an alternative implementation, according to the generated computing power constraint conditions, use a runtime environment simulation tool to simulate the initial deployment package. During the simulation process, monitor the resource usage of the initial deployment package to ensure that it runs within the computing power constraint conditions. If the simulation runs successfully, use a national cryptography algorithm such as the SM3 algorithm to calculate the hash of the initial deployment package to generate a national cryptography algorithm verification code. Add the national cryptography algorithm verification code to the initial deployment package to finally generate a model encrypted deployment package containing the national cryptography algorithm verification code. This deployment package can be safely deployed to the target environment to ensure the security and integrity of the model.

[0111] Based on any of the above embodiments, in Embodiment 6 of the present application, referring to Figure 3 , the steps of deploying the model encrypted deployment package to the front-end and back-end applications include: Step S51, parse the integrity verification code in the model encrypted deployment package, verify the integrity of the deployment package through the national cryptography algorithm, and synchronously verify the consistency between the hardware device fingerprint and the registered device information in the requirement form.

[0112] In this embodiment, the model encrypted deployment package: a data packet containing encrypted model weights, inference driver libraries, and related configuration information for safely deploying the model. Integrity verification code: a fixed-length value calculated using a specific algorithm such as the national cryptography algorithm for the content of the deployment package to verify whether the deployment package has been tampered with during transmission or storage. National cryptography algorithm: a series of cryptography algorithms independently developed in China, such as SM2, SM3, SM4, etc., which have high security and reliability and are used to ensure the confidentiality, integrity, and authenticity of information. Hardware device fingerprint: a unique identifier composed of various characteristic information of the hardware device, such as CPU model, MAC address, hard disk serial number, etc., for identifying and verifying the identity of the device. Registered device information in the requirement form: device-related information pre-registered by the user in the requirement form, including device model, hardware configuration, etc., for comparison and verification with the actual deployed device.

[0113] As an alternative implementation, when receiving the model encrypted deployment package, first extract the integrity verification code from the deployment package. Then, use the same national cryptography algorithm such as the SM3 algorithm to recalculate the content of the deployment package to obtain a new verification code. Compare the newly calculated verification code with the extracted integrity verification code. If the two are consistent, it indicates that the integrity of the deployment package has not been damaged. At the same time, obtain the hardware device fingerprint of the current target device through a hardware information collection tool. Compare the obtained hardware device fingerprint with the registered device information in the requirement form to check whether the two are consistent. If the hardware device fingerprint matches the registered device information, subsequent deployment operations can continue; otherwise, terminate the deployment and issue a security warning.

[0114] Step S52: Based on the hardware acceleration module in the trusted execution environment, create an isolated model running sandbox on the target device, and load the encrypted model weights and the inference driver library, where the sandbox is configured to only allow access to the training and inference data through the privacy computing channel.

[0115] In this embodiment, the hardware acceleration module: a hardware device dedicated to accelerating computing tasks, such as GPU, FPGA, etc., which can significantly improve the inference speed of the model. The model running sandbox: an isolated running environment used to restrict the access rights of the model, ensure that the model runs in a secure environment, and prevent damage to the target device. The encrypted model weights: the model parameters after encryption processing to ensure the security of the model during transmission and storage. The inference driver library: a software library used to drive the hardware acceleration module for model inference, providing interfaces and functions for interacting with the hardware device. The privacy computing channel: a secure communication channel used for data transmission and calculation while protecting data privacy, such as using technologies like homomorphic encryption and differential privacy.

[0116] As an alternative implementation, use the hardware acceleration module based on the trusted execution environment on the target device to create an isolated model running sandbox. This sandbox is an independent running environment isolated from other parts of the target device to ensure that the operation of the model will not affect other functions of the device. In the sandbox, load the encrypted model weights and the integrated inference driver library. During the loading process, ensure that the decryption operation of the model weights is performed in the trusted execution environment to guarantee data security.

[0117] Configure the sandbox to only allow access to the training and inference data through the privacy computing channel. This can ensure the privacy of the training and inference data during transmission and use, and prevent data leakage. For example, homomorphic encryption technology can be used to encrypt the training and inference data, and the data does not need to be decrypted during the calculation in the sandbox, thus achieving privacy protection.

[0118] Step S53: Dynamically select the training and inference model according to the characteristics of the front-end application interface protocol, where the training and inference model includes an operator acceleration library matching the target processor architecture, and inject the model weights into the pre-allocated video memory area through memory mapping technology.

[0119] In this embodiment, the front-end application interface protocol features: the characteristics of the protocol followed for communication between the front-end application and the back-end service, including data format, request method, response format, etc., which are used to determine a suitable training and inference model. Target processor architecture: the architecture of the processor adopted by the target device, such as x86, ARM, etc. Different processor architectures have an impact on the running efficiency of the model. Operator acceleration library: a function library optimized for a specific processor architecture, containing efficient implementations of various computing operators, which can accelerate the inference process of the model. Memory mapping technology: a technology that maps the physical memory of a disk file or other device to the virtual address space of a process. Through this technology, disk files or device memory can be directly accessed, improving data transfer efficiency. Video memory pre-allocation area: a pre-allocated area in the GPU video memory, which is used to store model weights and intermediate calculation results, so as to reduce the overhead of video memory allocation and release.

[0120] As an alternative implementation, analyze the front-end application interface protocol features, and dynamically select the most suitable model from the available training and inference models according to these features. For example, if the front-end application requires real-time response, a model with a faster inference speed may need to be selected; if the front-end application processes image data, a model suitable for image recognition may need to be selected. The selected training and inference model should include an operator acceleration library that matches the target processor architecture. In this way, the performance of the target processor can be fully utilized to improve the inference efficiency of the model. For example, for a processor based on the x86 architecture, an operator acceleration library optimized for x86 can be selected; for a processor based on the ARM architecture, an operator acceleration library optimized for ARM can be selected. Use the memory mapping technology to inject the encrypted model weights decrypted in the sandbox into the video memory pre-allocation area. Through memory mapping, the model weights can be directly mapped from the disk or other storage devices to the video memory, avoiding the overhead of data copying and transmission and improving the data access speed. At the same time, the use of the video memory pre-allocation area can reduce the time for video memory allocation and release, further improving the inference performance of the model.

[0121] Step S54, deploy a gateway in the back-end service container, configure an input data format converter and an output result encryption module, and the converter automatically adjusts the concurrent threads of batch inference according to the computing power requirement level in the requirement characteristics.

[0122] In this embodiment, the backend service container: uses containerization technology such as Docker to package the backend service into an independent running unit, facilitating deployment and management. Gateway: serves as the entry point for the backend service, responsible for receiving requests from the front-end application and forwarding the requests to the corresponding service modules. It can also perform operations such as request filtering, routing, and traffic limiting. Input data format converter: used to convert the input data sent by the front-end application into a format that the training and inference model can process. For example, it can convert image data in different formats into a unified format. Output result encryption module: encrypts the output result of the training and inference model to protect the privacy and security of the data. For example, it uses a symmetric encryption algorithm to encrypt the result. The computing power requirement level in the requirement characteristics: the level of computing power requirements proposed by the user in the requirement form, such as high, medium, low, etc., which is used to guide the allocation and adjustment of resources. The number of concurrent threads for batch inference: the number of threads running simultaneously during batch inference. Reasonably adjusting the number of concurrent threads can improve the inference efficiency.

[0123] As an alternative implementation, the gateway is deployed in the backend service container as the unified entry point for the entire backend service. The gateway can be implemented using open-source gateway frameworks such as Nginx, Kong, etc. Configure the routing rules of the gateway to accurately forward the requests from the front-end application to the corresponding training and inference model service modules. Configure the input data format converter in the gateway. This converter can perform corresponding conversion operations according to the data format sent by the front-end application and the data format required by the training and inference model. For example, if the front-end application sends JPEG format image data and the training and inference model requires PNG format image data, the converter can convert the JPEG image to a PNG image. At the same time, configure the output result encryption module in the gateway. When the training and inference model completes the inference, the output result encryption module uses a pre-configured encryption algorithm (such as the AES algorithm) to encrypt the output result to ensure the security of the result during transmission.

[0124] According to the computing power requirement level in the requirement characteristics, the input data format converter automatically adjusts the number of concurrent threads for batch inference. If the computing power requirement level is high, the number of concurrent threads can be increased to make full use of the hardware resources and improve the inference speed; if the computing power requirement level is low, the number of concurrent threads can be reduced to reduce energy consumption and resource occupancy. For example, by monitoring the usage of hardware resources, the number of concurrent threads can be dynamically adjusted to achieve optimal resource configuration.

[0125] Based on any of the above embodiments, in the seventh embodiment of this application, before the step of calling the reserved computing power of the user for the application in the requirement form, generating a dedicated model and encapsulating it into a model encryption deployment package in response to the training and inference operation in the development environment, it includes: Step B10: Analyze the computing power requirement level, task concurrency peak value, and model complexity parameters in the requirement form to generate initial computing power resource allocation parameters.

[0126] In this embodiment, the computing power requirement level is the level divided by the user according to their own business needs for the required computing power, such as high, medium, and low levels, which reflects the general requirements for computing speed and processing power. The task concurrency peak value is the maximum number of tasks that the system needs to process simultaneously within a specific time period, which reflects the processing pressure of the system under peak load. The model complexity parameter is a related parameter that describes the complexity of the model, such as the number of layers, the number of neurons, and the number of parameters of the model, which affects the computing resources required for model training and inference. The initial computing power resource allocation parameter is a parameter initially determined for allocating computing power resources based on the information obtained from the above analysis, such as the number of CPU cores, the number of GPUs, and the memory size.

[0127] As an alternative implementation, first, extract the computing power requirement level, task concurrency peak value, and model complexity parameters from the requirement form. Tools such as regular expressions and data parsing libraries can be used to parse the text or structured data in the requirement form. For example, if the requirement form is a JSON file, the Python json library can be used for parsing. Based on the extracted parameters, combine preset rules or empirical formulas to generate initial computing power resource allocation parameters. For example, if the computing power requirement level is high, the task concurrency peak value is large, and the model complexity is high, then more CPU cores, GPUs, and larger memory can be allocated. A parameter mapping table can be established to determine the corresponding initial computing power resource allocation parameters according to different parameter combinations.

[0128] Step B20: Based on the computing power consumption records in the historical training database, calculate the computing power requirement baseline value of the target scenario through a time series prediction model, where the baseline value includes three-dimensional constraint indicators of video memory occupancy rate, floating-point operation volume, and memory bandwidth utilization rate.

[0129] In this embodiment, the historical training database is a database that stores data related to computing power consumption during previous model training processes, including the computing power usage of different tasks and different models during training. The computing power consumption record records the usage of various computing power indicators during each model training process, such as the change data of video memory occupancy, floating-point operation volume, and memory bandwidth utilization rate over time. The time series prediction model is a model used to analyze time series data and make predictions, such as ARIMA, LSTM, etc., which predicts future computing power requirements by learning historical data. The target scenario is the specific business scenario where the user currently needs to perform model training or inference. The computing power requirement baseline value is a reasonable reference value for computing power requirements in the target scenario, which consists of three-dimensional constraint indicators of video memory occupancy rate, floating-point operation volume, and memory bandwidth utilization rate.

[0130] As an alternative implementation, extract the computing power consumption records related to the target scenario from the historical training database. The extraction can be filtered according to conditions such as task type, model type, and hardware environment. Organize the extracted computing power consumption records into time series data and use a suitable time series prediction model for training. For example, use the ARIMA model in the statsmodels library of Python or the LSTM model in the tensorflow library. After training the model, input the relevant information of the target scenario to predict the baseline values of the video memory occupancy rate, floating-point operation volume, and memory bandwidth utilization rate under the target scenario, which are used as references for computing power requirements.

[0131] Step B30: According to the hardware architecture identifier in the requirement form, match the processor instruction set support map and filter the candidate computing power types that meet the computing power requirement baseline value.

[0132] In this embodiment, the hardware architecture identifier: information used to uniquely identify the hardware architecture type, such as x86, ARM, GPU architecture, etc., which reflects the basic characteristics of the hardware and the instruction set support situation. The processor instruction set support map: a map that records the instruction set information supported by different processor architectures, showing the correspondence between various instruction sets and hardware architectures. Candidate computing power types: different types of computing resources that meet the computing power requirement baseline value, such as different models of CPUs, GPUs, etc.

[0133] As an alternative implementation, obtain the hardware architecture identifier from the requirement form. According to this identifier, search for the corresponding instruction set information in the processor instruction set support map. Combine the obtained computing power requirement baseline value to filter the existing various computing power types. Check whether the hardware architecture of each computing power type supports the required instruction set and whether it can meet the baseline value requirements of the video memory occupancy rate, floating-point operation volume, and memory bandwidth utilization rate. Filter out the candidate computing power types that meet the conditions.

[0134] Step B40: Based on the reinforcement learning algorithm, perform Pareto optimization on the resource utilization rate and energy consumption ratio of the candidate computing power types to generate a dynamic scheduling policy that includes a computing power threshold interval, a time window constraint, and an elastic scaling rule.

[0135] In this embodiment, the reinforcement learning algorithm: a machine learning algorithm that interacts with the environment through an agent, continuously tries different behaviors to maximize the cumulative reward, such as Q-learning, A3C, etc. Resource utilization rate and energy consumption ratio: an indicator that measures the relationship between the use efficiency of computing resources and energy consumption, reflecting the proportion of computing resources that can be effectively utilized under a certain amount of energy consumption. Pareto optimization: an optimization method that cannot improve a certain goal without reducing other goals, used to find the optimal balance between resource utilization rate and energy consumption ratio. Computing power threshold range: the allowable range of computing power usage, specifying the minimum and maximum computing power usage to ensure that the system operates within a reasonable computing power range. Time window constraint: the restriction on the time of computing power usage, such as the maximum computing power that can be used within certain time periods, to meet the requirements of business needs and resource management. Elastic scaling rule: a rule for dynamically adjusting computing power resources according to the change of business load, such as increasing computing power during peak business hours and reducing it during off-peak hours. Dynamic scheduling strategy: a strategy for dynamically adjusting computing power allocation according to different business scenarios and resource states, including computing power threshold range, time window constraint, and elastic scaling rule.

[0136] As an alternative implementation, the candidate computing power types are used as optional actions in the reinforcement learning algorithm, and the resource utilization rate and energy consumption ratio are used as the reward function. The environmental state is defined as information such as the current business load and resource usage. The reinforcement learning algorithm is used for training. The agent selects different candidate computing power types under different environmental states and maximizes the reward function by continuously interacting with the environment. During the training process, the idea of Pareto optimization is adopted to minimize energy consumption while improving resource utilization rate. According to the training results, a dynamic scheduling strategy including computing power threshold range, time window constraint, and elastic scaling rule is generated. For example, according to the peak and off-peak times of the business, the computing power threshold range for different time periods is determined; according to the change rate of the business load, the elastic scaling rule is formulated.

[0137] Step B50, bind the dynamic scheduling strategy to the target hardware resource pool to generate a timestamped reserved computing power identifier to identify the reserved computing power.

[0138] In this embodiment, the target hardware resource pool: a set of computable resources available for allocation, including various types of hardware devices, such as servers, GPU clusters, etc. Timestamp: records the specific time information of the occurrence of an event, used to identify the generation time of the reserved computing power. Reserved computing power identifier: an identifier used to uniquely identify the reserved computing power resource, including information such as the dynamic scheduling strategy and generation time.

[0139] As an alternative implementation, bind the dynamic scheduling policy generated in step four to the target hardware resource pool. The hardware resources can be allocated and used according to the dynamic scheduling policy by configuring corresponding scheduling rules in the management system of the hardware resource pool. Generate a timestamped identifier for the reserved computing power resources. A unique identifier can be generated using the UUID algorithm, and the current timestamp information can be added to the identifier. Associate the identifier with the reserved computing power resources for subsequent resource management and tracking. For example, record the identifier and the corresponding computing power resource information in the database of the computing power leasing platform for convenient user query and use.

[0140] The present application provides an application deployment device based on cloud computing power. The application deployment device based on cloud computing power includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the application deployment method based on cloud computing power in Embodiment 1 above.

[0141] Refer to the following Figure 4 , which shows a schematic structural diagram of an application deployment device based on cloud computing power suitable for implementing the embodiments of the present application. The application deployment device based on cloud computing power in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers, in-vehicle terminals, etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The application deployment device based on cloud computing power shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0142] As shown in Figure 4As shown, the application deployment device based on cloud computing power may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. In the random access memory 1004, various programs and data required for the operation of the application deployment device based on cloud computing power are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the application deployment device based on cloud computing power to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows an application deployment device based on cloud computing power with various systems, it should be understood that it is not required to implement or have all the shown systems. More or fewer systems may be implemented or had alternatively.

[0143] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart may be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device, or installed from the storage device 1003, or installed from the ROM 1002. When the computer program is executed by the processing device 1001, the above functions defined in the method of the embodiments disclosed in the present application are executed.

[0144] The application deployment device based on cloud computing power provided by the present application adopts the application deployment method based on cloud computing power in the above embodiments to solve the technical problem that the software development cycle of AI applications is greatly extended compared with traditional software, making it difficult to quickly form competitiveness. Compared with the prior art, the beneficial effects of the application deployment device based on cloud computing power provided by the present application are the same as those of the application deployment device based on cloud computing power provided by the above embodiments, and other technical features in the application deployment device based on cloud computing power are the same as those disclosed in the method of the previous embodiment, and will not be elaborated here.

[0145] It should be understood that each part disclosed in this application can be implemented by hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0146] As described above, it is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application, and all should be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

[0147] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the method for cloud computing power-based application deployment in the above embodiments.

[0148] The computer-readable storage medium provided by this application can be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, radio frequency (RF), etc., or any suitable combination of the above.

[0149] The above computer-readable storage medium can be included in the cloud computing power-based application deployment device; it can also exist separately and not be assembled into the cloud computing power-based application deployment device.

[0150] The above computer-readable storage medium carries one or more programs, which, when executed by an application deployment device based on cloud computing power, cause the application deployment device based on cloud computing power to: extract demand characteristics from the demand form and the negotiation materials in response to the demand form and the negotiation materials; match a training and inference model and training and inference data according to the demand characteristics; deploy a development environment based on the training and inference model and the training and inference data; call the reserved computing power of the application user for the demand form in response to the training and inference operation of the development environment, generate a dedicated model and package it into a model encryption deployment package; and deploy the model encryption deployment package to the front-end and back-end applications.

[0151] Computer program code for performing the operations of this application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, execute as a stand-alone software package, execute partially on the user's computer and partially on a remote computer, or execute entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0152] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0153] The modules involved in the embodiments of the present application can be implemented in software or in hardware. In some cases, the name of the module does not constitute a limitation on the unit itself.

[0154] The readable storage medium provided by the present application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned application deployment method based on cloud computing power, which can solve the technical problem that the software development cycle of AI applications is greatly extended compared with traditional software and it is difficult to quickly form competitiveness. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as those of the application deployment method based on cloud computing power provided by the above embodiments, and will not be elaborated here.

[0155] An embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the steps of the application deployment method based on cloud computing power as described above.

[0156] The computer program product provided by the present application can solve the technical problem that the software development cycle of AI applications is greatly extended compared with traditional software and it is difficult to quickly form competitiveness. Compared with the prior art, the beneficial effects of the computer program product provided by the embodiments of the present application are the same as those of the application deployment method based on cloud computing power provided by the above embodiments, and will not be elaborated here.

[0157] The above are only the preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present application, or directly or indirectly applied to other related technical fields, shall be included in the patent scope of the present application by the same token.

Claims

1. An application deployment method based on cloud computing power, characterized in that, Applied to a computing power leasing platform, the application deployment method based on cloud computing power includes: In response to a requirement form and negotiation materials, extract the requirement features in the requirement form and the negotiation materials; Match a training and inference model and training and inference data according to the requirement features; Deploy a development environment based on the training and inference model and the training and inference data; In response to the training and inference operation of the development environment, call the reserved computing power of the user corresponding to the requirement form, generate a dedicated model and package it into a model encryption deployment package; Deploy the model encryption deployment package to the front-end and back-end applications.

2. The method according to claim 1, wherein The step of, in response to a requirement form and negotiation materials, extracting the requirement features in the requirement form and the negotiation materials includes: Perform speech recognition on the negotiation materials to generate unstructured text data; Through a pre-trained multimodal alignment model, perform cross-modal semantic association on the unstructured text data and the structured field data of the requirement form, where the multimodal alignment model uses a bidirectional attention mechanism to calculate the association matrix between form fields and dialogue texts; Screen key semantic segments according to the association matrix to generate the requirement features including a hardware type identifier, a computing power requirement level, and a data security requirement, where the hardware type identifier includes the processor architecture, memory capacity, and interface protocol feature encoding of the target device.

3. The method according to claim 2, wherein Before the step of, through a pre-trained multimodal alignment model, performing cross-modal semantic association on the unstructured text data and the structured field data of the requirement form, includes: Construct an annotated hardware requirement corpus, and the training data of the hardware requirement corpus includes a triple of form fields, dialogue texts, and corresponding hardware type identifiers; Adopt a contrastive learning loss function to constrain the cosine similarity of form-dialogue pairs of the same hardware type in the feature space to be higher than that of heterogeneous samples; Based on an adversarial sample generation enhancement model, improve the parsing robustness of fuzzy expressions to obtain the multimodal alignment model.

4. The method according to claim 1, wherein The step of, according to the requirement features, matching a training and inference model and training and inference data includes: Parse the hardware type identifier in the requirement features, and screen candidate training and inference models based on a processor architecture compatibility map, where the map pre-stores the mapping relationship of the support status of neural network operators on different hardware platforms; Perform dynamic pruning on the candidate training and inference models according to the computing power requirement level to generate the training and inference model that meets the memory capacity constraint of the target device; Construct a privacy computing channel based on the data security requirement, extract initial data matching the requirement scenario label from distributed data sources, and generate the training and inference data protected by differential privacy through a homomorphic encryption algorithm.

5. The method according to claim 1, characterized in that The step of, based on the training and inference model and the training and inference data, deploying a development environment includes: According to the framework type of the training and inference model, match the corresponding version of the runtime dependency library from a preset container template library, and construct a basic container image including a framework interpreter and a model conversion tool chain; Parse the interface protocol feature encoding of the hardware type identifier in the requirement features, and inject a target hardware driver layer into the basic container image to generate a training and inference execution environment adapted to the hardware. Mount the training and inference data with differential privacy protection to the container storage volume, and automatically generate a data preprocessing script that matches the input and output formats of the training and inference model; Configure the environment variable parameters, including the model service port number, computing power resource quota, and monitoring probe collection frequency, to complete the development environment deployment.

6. The method according to claim 1, characterized in that The step of, in response to the training and inference operation of the development environment, invoking the demand form to reserve the computing power of the corresponding user, generating a dedicated model, and packaging it into a model encryption deployment package includes: Based on the container runtime environment of the development environment, parse the dynamic scheduling policy of the reserved computing power to generate resource allocation parameters and computing power constraint conditions; According to the resource allocation parameters, dynamically adjust the training batch size and video memory occupancy ratio through a resource allocation algorithm driven by reinforcement learning, and output gradient update parameters that adapt to the target hardware computing power threshold; Based on the gradient update parameters and the data privacy requirements marked in the demand form, dynamically allocate the differential privacy budget for the training dataset through the privacy computing gateway to generate a sequence of feature tensors with privacy protection labels; Adopt a homomorphic encryption algorithm to perform layer-by-layer encryption processing on the model weights corresponding to the sequence of feature tensors, where the encryption key is generated by mixing two factors: the user identity identifier in the demand form and the hardware device fingerprint; Inject the encrypted model weights into the hardware acceleration module based on the trusted execution environment, and integrate them with the inference driver library corresponding to the gradient update parameters to generate an initial deployment package; Perform a runtime environment simulation on the initial deployment package according to the computing power constraint conditions to generate the model encryption deployment package containing the national cryptography algorithm verification code.

7. The method according to claim 1, characterized in that The step of deploying the model encryption deployment package to the front-end and back-end applications includes: Parse the integrity verification code in the model encryption deployment package, verify the integrity of the deployment package through the national cryptography algorithm, and synchronously verify the consistency between the hardware device fingerprint and the registered device information in the demand form; Based on the hardware acceleration module of the trusted execution environment, create an isolated model running sandbox on the target device, load the encrypted model weights and the inference driver library, where the sandbox is configured to only allow access to the training and inference data through the privacy computing channel; Dynamically select the training and inference model according to the characteristics of the front-end application interface protocol, where the training and inference model includes an operator acceleration library that matches the target processor architecture, and inject the model weights into the pre-allocated video memory area through the memory mapping technology; Deploy a gateway in the back-end service container, configure an input data format converter and an output result encryption module, and the converter automatically adjusts the concurrent thread number of batch inference according to the computing power demand level in the demand characteristics.

8. The method according to claim 1, wherein Before the step of, in response to the training and inference operation of the development environment, invoking the demand form to reserve the computing power of the corresponding user, generating a dedicated model, and packaging it into a model encryption deployment package, includes: Parse the computing power demand level, task concurrency peak, and model complexity parameters in the demand form to generate initial computing power resource allocation parameters; Based on the computing power consumption records in the historical training database, calculate the baseline value of the computing power demand for the target scenario through a time series prediction model, where the baseline value includes three-dimensional constraint indicators of video memory occupancy rate, floating-point operation volume, and memory bandwidth utilization rate; According to the hardware architecture identifier in the demand form, match the processor instruction set support map, and screen the candidate computing power types that meet the baseline value of the computing power demand; Based on the reinforcement learning algorithm, perform Pareto optimization on the resource utilization rate and energy consumption ratio of the candidate computing power types, and generate a dynamic scheduling strategy including a computing power threshold interval, a time window constraint, and an elastic scaling rule; Bind the dynamic scheduling strategy to the target hardware resource pool to generate a timestamped reserved computing power identifier to identify the reserved computing power.

9. An application deployment device based on cloud computing power, characterized in that, The application deployment device based on cloud computing power includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the application deployment method based on cloud computing power according to any one of claims 1 to 8.

10. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the application deployment method based on cloud computing power according to any one of claims 1 to 8 are implemented.

Images