A heterogeneous model deployment method and system based on semantic parsing and dependency solving

Abstract

The application belongs to the field of model deployment, and particularly relates to a heterogeneous model deployment method and system based on semantic analysis and dependency solving, comprising a semantic access, a dependency solving, a conversion gateway and a scheduling center. The semantic access module analyzes a model to extract input and output semantics and bottom layer operator constraints, and generates a semantic feature vector; the dependency solving module constructs a cross-layer dependency graph, dynamically relaxes solving with operator reservation as a target, generates a compatible environment and outputs an operator capability contract; the conversion gateway compares user semantics and model semantics, dynamically generates an adapter script based on the capability contract, executes reasoning after sandbox verification, and is encapsulated and returned; and the scheduling center predicts a request amount and expands in advance. The application realizes automatic encapsulation, transparent calling and elastic running of heterogeneous models, and effectively solves dependency conflict, adaptation complexity and scheduling lag problems.

Description

Technical Field

[0001] This invention belongs to the field of model deployment, specifically relating to a heterogeneous model deployment method and system based on semantic parsing and dependency solving. Background Technology

[0002] With the widespread application of artificial intelligence technology across various industries, enterprise-level AI platforms face the significant challenge of providing unified services for heterogeneous models. Existing model deployment solutions suffer from the following main shortcomings: (1) Rigid encapsulation and frequent dependency conflicts. Traditional containerized deployment solutions are usually based on fixed base image templates and adapt the model through simple file mounting or dependency installation. However, there are often complex version conflicts between dependency libraries of different frameworks (such as TensorFlow and PyTorch) and different versions. Existing static matching technology cannot automatically resolve deep dependency conflicts, often leading to encapsulation failure or runtime crashes, i.e., the "dependency hell" problem.

[0003] (2) The cost of adapting the calling interface is high. Existing solutions usually require the client to strictly follow the original input format of the model. Since the input and output formats of heterogeneous models vary greatly, the caller needs to write specific preprocessing code for each model. The lack of semantic-level adaptive conversion mechanism leads to high integration complexity and maintenance difficulties.

[0004] (3) Resource scheduling is lagging and lacks predictability. Existing container scaling strategies are mostly based on passive triggering of real-time monitoring thresholds. In the face of sudden traffic surges, this passive response is significantly lagging and cannot achieve a dynamic balance between resource preloading and optimal cost.

[0005] (4) Semantic definitions are detached from the runtime environment. Most existing semantic definitions are static tag-based descriptions that are not associated with the actual operator support capabilities of the underlying dependency libraries, resulting in the defined semantics being unable to be executed under specific dependency versions.

[0006] (5) Dependency solving lacks operator awareness. Existing dependency solving only focuses on graph theory solving at the software package version level, without considering the operator-level dependencies unique to AI models. When version conflicts occur, the crude version upgrades and downgrades often cause specific operators embedded in the model to fail, resulting in "silent errors" at runtime.

[0007] (6) The conversion logic is static and lacks a verification mechanism. Existing semantic conversions are mostly based on preset hard-coded rules or static execution flows. When changes in the underlying dependent environment lead to missing operators, or when there are anomalies in the input data (such as long-tail distribution or a lot of noise), the lack of dynamic generation of conversion logic and pre-verification before execution can easily lead to uncontrollable model inference results or even service crashes. Summary of the Invention

[0008] The technical problem this invention aims to solve is to provide a heterogeneous model deployment method and system based on semantic parsing and dependency solving. This method dynamically generates adaptation code by directly receiving the target semantics and operator capability contracts, and then pre-validates it in a sandbox. This prior-execution mechanism addresses the transformation failures caused by high noise, long tails, and missing context in heterogeneous input data.

[0009] A heterogeneous model deployment system based on semantic parsing and dependency solving includes a model semantic access module, a dependency solving encapsulation engine, a semantic transformation service gateway, and a predictive resource scheduling center. The model semantic access module is used to receive model files uploaded by users, parse the model files to extract model input semantics, input preprocessing semantics, model output semantics, output postprocessing semantics, and the set of underlying operator constraints required to execute the input preprocessing semantics, and generate and store model semantic feature vectors based on the extraction results. The dependency solving encapsulation engine communicates with the model semantic access module and is used to receive the model semantic feature vector, extract the underlying operator constraint set, obtain global software package metadata, construct a cross-layer dependency graph containing software package nodes, operator nodes, and the constraint relationships between them, perform conflict detection on the cross-layer dependency graph, and if a conflict is detected, perform dynamic relaxation solving with the goal of operator preservation to generate a compatible software package version set; if the solution fails, send a relaxation request to the model semantic access module to trigger a downgrade adjustment of the input preprocessing semantics until the solution is successful; construct a standardized runtime environment based on the successfully solved software package version set, encapsulate the standardized runtime environment into a container image and push it to the container orchestration cluster, and perform operator scanning on the standardized runtime environment to generate operator capability contracts; The semantic transformation service gateway is communicatively connected to the model semantic access module and the dependency solving encapsulation engine, respectively. It receives the model semantic feature vector and the operator capability contract, and receives user-uploaded raw inference data from the business side. It extracts user data semantics from the user-uploaded raw inference data, compares the user data semantics with the model input semantics, and generates a semantic difference set. Based on the semantic difference set and the operator capability contract, it dynamically generates an adapter script. It performs trial runs and verification of the adapter script in an isolation sandbox. If the verification fails, it enhances and adjusts the adapter script and re-verifies it until it passes. It processes the user-uploaded raw inference data using the verified adapter script to obtain a standard input tensor. On the container resources pre-allocated by the predictive resource scheduling center for the standardized operating environment, it sends the standard input tensor into the standardized operating environment to execute model inference, obtaining inference output. It standardizes and encapsulates the inference output according to the output post-processing semantics, generating standardized inference response data, which is returned to the requester as the response result of the user-uploaded raw inference data. The predictive resource scheduling center is communicatively connected to the model semantic access module and the container orchestration cluster that deploys the semantic transformation service gateway. It receives the model semantic feature vector and extracts model computational complexity information; collects real-time operational status indicators of each model service; predicts the request volume for future periods; combines the predicted request volume and the model computational complexity information to generate a resource scheduling scheme that satisfies the Service Level Agreement (SLA) constraints. The resource scheduling scheme includes the target number of container instances and the resource quotas corresponding to each instance. When the predicted request volume exceeds a preset security threshold, the resource scheduling scheme is pre-deployed to the container orchestration cluster, triggering the pre-deployment of the container image and the elastic expansion of the number of container instances.

[0010] Optionally, the model semantic access module parses the model file to extract model input semantics, input preprocessing semantics, model output semantics, and output postprocessing semantics, including: The model file is parsed to obtain the model computation graph. The input node features and output node features in the model computation graph are matched and processed by the built-in operator prototype network. The model input semantics, the input preprocessing semantics, the model output semantics, and the output postprocessing semantics are automatically inferred and output.

[0011] Optionally, conflict detection is performed on the cross-layer dependency graph, including: Calculate the dependency conflict coefficient for each pair of software package nodes in the graph; When the dependency conflict coefficient does not exceed a preset threshold, it is determined that no conflict has been detected. When the dependency conflict coefficient exceeds a preset threshold, a conflict is detected.

[0012] Optional, dynamic relaxation solutions with operator preservation as the objective include: A multi-objective optimization function is constructed, which includes version stability objective and operator retention loss objective. The optimal solution vector is searched in the constraint space of the cross-layer dependency graph as the set of compatible software package versions. When the multi-objective optimization function diverges, the solution is determined to have failed. The multi-objective optimization function is expressed as:

[0013] in, The total loss value of the multi-objective optimization function. The set of compatible software package versions to be solved. This represents the total number of dependent packages. For the first Stability ratings for each software package version and As weight, For the underlying operator constraint set, The loss function is the operator, and divergence means that the multi-objective optimization function does not have a finite minimum value.

[0014] Optionally, the semantic translation service gateway performs trial runs and verification of the adapter script in an isolation sandbox, specifically including: A sampled slice of the user's original inference data is extracted, and the adapter script is run in the isolation sandbox to perform preprocessing operations on the sampled slice to obtain the output tensor. Verify whether the tensor specification of the output tensor meets preset conditions, including shape consistency conditions and numerical distribution validity conditions; If any preset condition is not met, the verification is deemed to have failed.

[0015] Optionally, the enhancement and adjustment of the adapter script includes: If the tensor specification verification of the output tensor fails, a new data preprocessing operation is added to the semantic difference set. Based on the updated semantic difference set, the adapter script is regenerated, and a trial run verification is performed again in the isolation sandbox. The trial run verification includes: extracting a sample slice of the user's original inference data, running the regenerated adapter script to perform preprocessing operations on the sample slice, obtaining the output tensor, and performing tensor specification verification. If the verification still fails, the steps of adding the data preprocessing operation and regenerating the adapter script are iteratively executed until the verification passes.

[0016] Optionally, the predictive resource scheduling center forecasts request volumes for future periods, including: The system collects request volume, response time, CPU utilization, and memory utilization of each model service in real time to construct multi-dimensional time-series features. An online learning model is then used to predict the request volume for future periods based on these multi-dimensional time-series features within a historical time-series window.

[0017] A method for deploying heterogeneous models based on semantic parsing and dependency solving includes: The system receives model files uploaded by users, parses the model files to extract model input semantics, input preprocessing semantics, model output semantics, output postprocessing semantics, and the set of underlying operator constraints required to execute the input preprocessing semantics, and generates and stores model semantic feature vectors based on the extraction results. Based on the model semantic feature vector, the underlying operator constraint set is extracted; global software package metadata is obtained, and a cross-layer dependency graph containing software package nodes, operator nodes, and the constraint relationships between them is constructed; conflict detection is performed on the cross-layer dependency graph, and if a conflict is detected, dynamic relaxation solution with the goal of operator preservation is executed to generate a compatible software package version set; if the solution fails, a relaxation request is fed back to trigger a downgrade adjustment of the input preprocessing semantics until the solution is successful; a standardized runtime environment is constructed based on the successfully solved software package version set, the standardized runtime environment is packaged into a container image and pushed to the container orchestration cluster, and operator scanning is performed on the standardized runtime environment to generate operator capability contracts; Based on the model semantic feature vector and the operator capability contract, the system receives the user's original inference data uploaded by the business side; extracts the user data semantics from the user's original inference data, compares the user data semantics with the model input semantics, and generates a semantic difference set; dynamically generates an adapter script based on the semantic difference set and the operator capability contract; performs trial operation verification of the adapter script in an isolation sandbox, and if the verification fails, enhances and adjusts the adapter script and re-verifies it until the verification passes; processes the user's original inference data using the verified adapter script to obtain a standard input tensor; sends the standard input tensor into the standardized operating environment on the container resources pre-allocated by the predictive resource scheduling center for the standardized operating environment to execute model inference and obtain inference output; standardizes and encapsulates the inference output according to the output post-processing semantics to generate standardized inference response data, which is returned to the requester as the response result of the user's original inference data. Based on the semantic feature vector of the model, the computational complexity information of the model is extracted; the running status indicators of each model service are collected in real time to predict the request volume in future time periods, and the predicted request volume is obtained; combined with the predicted request volume and the computational complexity information of the model, a resource scheduling scheme that meets the service level agreement constraints is generated, and the resource scheduling scheme includes the target number of container instances and the resource quota corresponding to each instance; when the predicted request volume exceeds the preset security threshold, the resource scheduling scheme is issued to the container orchestration cluster in advance, triggering the pre-deployment of the container image and the elastic expansion of the number of container instances.

[0018] A terminal device includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor loads and executes the computer program, it employs a heterogeneous model deployment method based on semantic parsing and dependency solving.

[0019] A computer-readable storage medium storing a computer program, wherein when the computer program is loaded and executed by a processor, a heterogeneous model deployment method based on semantic parsing and dependency solving is employed.

[0020] The beneficial effects of this invention are: 1. Breaking down the separation between semantics and environment to achieve cross-layer joint solution. For the first time, a cross-layer coupling graph of semantics-operator-dependency is proposed, which not only resolves version conflicts but also avoids the silent error of "version compatibility but operator loss" caused by blindly upgrading or downgrading traditional solvers. Through a dynamic relaxation mechanism to adjust semantics in reverse, true closed-loop adaptive model encapsulation is achieved.

[0021] 2. Dynamic generation of transformation logic and self-healing through sandbox pre-verification. By directly receiving the target semantics and operator capability contract, adaptive code is dynamically generated and pre-verified in a sandbox. This prior-execution mechanism intuitively and thoroughly solves the transformation failure problems caused by high noise, long tails, and missing context in heterogeneous input data.

[0022] 3. The semantic access module parses the model to extract input and output semantics and underlying operator constraints, generating semantic feature vectors; the dependency solving module constructs a cross-layer dependency graph, dynamically relaxes the solution with operator preservation as the goal, generates a compatible environment, and produces operator capability contracts; the conversion gateway compares user semantics with model semantics, dynamically generates adapter scripts based on capability contracts, executes inference after sandbox verification, and encapsulates and returns the results; the scheduling center predicts request volume and expands capacity in advance. This invention achieves automatic encapsulation, transparent invocation, and elastic operation of heterogeneous models, effectively solving the problems of dependency conflicts, cumbersome adaptation, and scheduling lag. Attached Figure Description

[0023] Figure 1This is a schematic diagram of the structure of a heterogeneous model adaptive service system according to the present invention. Detailed Implementation

[0024] A heterogeneous model deployment system based on semantic parsing and dependency solving, such as Figure 1 As shown, the present invention includes: Specifically, heterogeneous models are various AI models that come from different sources, have different technology stacks, different formats, and different operating environment requirements.

[0025] Model semantic access module, dependency solving encapsulation engine, semantic transformation service gateway, and predictive resource scheduling center; The model semantic access module is used to receive model files uploaded by users, parse the model files to extract model input semantics, input preprocessing semantics, model output semantics, output postprocessing semantics, and the set of underlying operator constraints required to perform input preprocessing semantics, and generate and store model semantic feature vectors based on the extraction results. The model semantic access module parses the model file to extract model input semantics, input preprocessing semantics, model output semantics, and output postprocessing semantics, including: The model file is parsed to obtain the model computation graph. The input node features and output node features in the model computation graph are matched and processed by the built-in operator prototype network. The model input semantics, input preprocessing semantics, model output semantics and output postprocessing semantics are automatically inferred and output.

[0026] Specifically, unlike traditional purely manual definitions, the system incorporates an Operator Prototype Network. An Operator Prototype Network is a pre-trained neural network model whose function is to perform pattern recognition and classification matching on the features of nodes in the model's computation graph. When a user uploads a model file (such as in ONNX or PB format), the system automatically parses the model file to obtain the model's computation graph. The model's computation graph is an abstract data structure in deep learning frameworks used to represent the structure of neural networks. It consists of multiple computation nodes (operators) and data flow edges between nodes, completely describing the entire computation process of the model from input to output.

[0027] The system uses an operator prototype network to match the features of input and output nodes in the model's computational graph. Specifically, it compares the data type, tensor dimension, and numerical range of the input nodes with the semantic types stored in the historical semantic database to automatically infer and output the semantic type of the model's input. and model output semantic type and input preprocessing semantics and output post-processing semantics .

[0028] Among them, the model input language This describes the semantic meaning of the input data that the model expects to receive, such as RGB images, grayscale images, text sequences, audio waveforms, etc.; input preprocessing semantics. This describes the sequence of preprocessing operations required to transform the raw data into a standard input tensor acceptable to the model, such as scaling to 640×640 resolution, normalizing to the [0,1] interval, and converting to RGB channel order; the model output semantics... It describes the semantic meaning of the model's inference results, such as image classification labels, object detection box coordinates, semantic segmentation masks, etc.; output post-processing semantics. It describes the processing operations required to convert the raw tensors output by the model into structured data that can be directly used by the business, such as Softmax normalization coordinate inverse calculation of category mapping, etc.

[0029] When generating the semantic feature vector of the model, the system simultaneously extracts the set of underlying operator constraints necessary to achieve the semantics. Low-level operators refer to the fundamental computational functions upon which specific data processing operations are performed, such as the bilinear interpolation operator needed for image scaling, and the Gaussian blur operator needed for image filtering. Operator constraint set That is, perform input preprocessing semantics. The system establishes a binding relationship between semantic vectors and operator constraints, ensuring that semantic definitions are correlated with underlying computational capabilities.

[0030] The semantic feature vector of the model is defined as follows:

[0031] This is the semantic feature vector of the model. This vector will be passed to module 2 for constructing the coupling graph and operator preservation solution, to module 3 for DAG optimization and semantic transformation, and to module 4 for computational complexity-aware scheduling. Input semantic types into the model; Preprocess semantics for input; Output semantic type for the model; For output post-processing semantics; The set of underlying operator constraints required to achieve semantics.

[0032] The system performs a hash operation on the uploaded model file to generate a model hash value. A unique model ID is generated based on the model hash value, and the model ID is associated with the model's semantic feature vector and stored in the meta-database. The meta-database is a relational or non-relational database used to store model metadata information, supporting fast retrieval by model ID.

[0033] The hash verification formula is as follows:

[0034] This is a hash value for model file integrity, used to verify file integrity and uniqueness, prevent damage or duplicate uploads, and ensure the security and traceability of model assets; It is a cryptographic hash algorithm; Model weights / configuration files uploaded by users.

[0035] For example, the development team uploaded an action detection model based on PyTorch 1.12 (a deep learning model for detecting human actions in video or image sequences). The system analyzes the model's computation graph and finds that input nodes are preceded by Resize and Normalize operations. The operator prototype network automatically infers... For images (i.e., data of image type that the model expects to receive), P_in is recommended to be Resize(640) (scaling the image to 640×640 pixels) and Normalize(0.5) (normalizing pixel values ​​to a distribution centered at 0.5). Simultaneously, the bilinear interpolation operator constraints required to perform Resize are extracted. ,and Binding, forming a complete semantic feature vector The system calculates and stores the hash value H of the model file, and establishes a model ID and its relation to the model file. The mapping index.

[0036] The system relies on a solution encapsulation engine and communicates with the model semantic access module to receive model semantic feature vectors and extract the underlying operator constraint set. It acquires global software package metadata and constructs a cross-layer dependency graph containing software package nodes, operator nodes, and the constraint relationships between them. It performs conflict detection on the cross-layer dependency graph; if a conflict is detected, it executes a dynamic relaxation solution with operator preservation as the goal, generating a compatible software package version set. If the solution fails, it sends a relaxation request to the model semantic access module to trigger a downgrade adjustment of the input preprocessing semantics until the solution is successful. Based on the successfully solved software package version set, it constructs a standardized runtime environment, encapsulates the standardized runtime environment into a container image, pushes it to the container orchestration cluster, and performs operator scanning on the standardized runtime environment to generate operator capability contracts. Specifically, the dependency solver encapsulation engine, based on the construction of the package dependency graph, integrates the operator constraint set extracted by the model semantic access module. Introduced as special nodes into the graph to construct a cross-layer coupling graph.

[0037] The graph structure of the cross-layer coupling graph is defined as follows:

[0038] in, A cross-layer coupling map; A collection of package nodes; For the set of operator nodes (i.e. Operator constraints in (the context of the problem). This is the set of version dependency constraint edges between packages; This is the set of support constraint edges for the package pair operator. By introducing... and This graph explicitly models operator constraints, supporting subsequent conflict detection by operator awareness.

[0039] A dependency conflict coefficient is introduced to quantify conflicts, as shown in the following formula:

[0040] For the package With Bag The dependency conflict coefficient; this coefficient is used to quantify the degree of version conflict between nodes in the dependency graph. When the set threshold is exceeded, the constraint solver is triggered to start. For nodes in the dependency graph.

[0041] Conflict detection is performed on cross-layer dependency graphs, including: Calculate the dependency conflict coefficient for each pair of software package nodes in the graph; If the dependency conflict coefficient does not exceed the preset threshold, it is determined that no conflict has been detected. A conflict is detected when the dependency conflict coefficient exceeds a preset threshold.

[0042] When conflict edges in the graph trigger constraint solving, this invention constructs an operator-aware multi-objective optimization function to find the optimal solution vector in the solution space that satisfies all constraints.

[0043] Dynamic relaxation solutions aimed at preserving operators include: A multi-objective optimization function is constructed, which includes version stability objective and operator preservation loss objective. The optimal solution vector is searched in the constraint space of the cross-layer dependency graph as the set of compatible software package versions. When the multi-objective optimization function diverges, the solution is considered to have failed. The multi-objective optimization function is expressed as:

[0044] in, The total loss value of the multi-objective optimization function ， The set of compatible software package versions to be solved. This represents the total number of dependent packages. For the first Stability ratings for each software package version and As weight, For the underlying operator constraint set, For operator loss function, divergence means that the multi-objective optimization function does not have a finite minimum value.

[0045] If the solver determines that there is no solution (i.e.) (If the value is ∞, indicating an absolute conflict between the version and the operator), the system does not directly report an error and roll back; instead, it notifies the model semantic access module to modify the semantic definition. For example, it downgrades the affine transformation preprocessing that depends on a higher version of OpenCV to a proportionally scaled preprocessing that can be supported by a lower version, i.e., it updates... and This achieves a dynamic balance between dependency resolution and semantic definition.

[0046] Based on the optimal solution vector Dynamically generate environment configuration files (such as Dockerfile) and use layered caching techniques to build standardized runtime environments (i.e., container images). The built image not only contains... The corresponding optimal dependency package also includes an operator execution library that matches the environment. Finally, the general semantic adapter script is injected into the image and pushed to the image repository.

[0047] Crucially, this module abstracts the list of all operators actually supported by the current image environment into an Operator Capability Contract (denoted as...). This information is then passed to the semantic transformation service gateway as a hard constraint boundary for subsequent generation of transformation logic. An operator capability contract refers to a list of all operators that are actually available (i.e., installed and normally invoked) in a standardized operating environment.

[0048] For example, dependency resolution revealed a version conflict between NumPy (a Python numerical computing library) and TorchVision (a PyTorch computer vision library). One version of TorchVision required NumPy version ≥ 1.21.0, while another version of NumPy had compatibility issues with other libraries in the system. Forcibly downgrading NumPy would cause the bilinear interpolation operator to be lost due to the underlying library version mismatch. The system then triggered a dynamic relaxation mechanism to... The `Resize_Bilinear(640)` in the code is relaxed to `Resize_Nearest(640)` (scaling using the nearest neighbor interpolation algorithm), and the corresponding operator constraints are... Updated from BilinearInterpolate to NearestInterpolate (nearest neighbor interpolation operator). The solver successfully outputs a conflict-free dependency environment.

[0049] At the same time, the operators actually supported by this environment (such as NearestInterpolate, GaussianBlur, etc.) are encapsulated into operator capability contracts. It is deployed and registered with the semantic transformation service gateway along with the container image.

[0050] The semantic transformation service gateway communicates with both the model semantic access module and the dependency solving encapsulation engine. It receives model semantic feature vectors and operator capability contracts, as well as user-uploaded raw inference data from the business side. It extracts user data semantics from the user-uploaded raw inference data, compares the user data semantics with the model input semantics, and generates a semantic difference set. Based on the semantic difference set and operator capability contracts, it dynamically generates an adapter script. The adapter script is then tested in an isolated sandbox; if the test fails, the adapter script is enhanced, adjusted, and retested until it passes. The tested adapter script is used to process the user-uploaded raw inference data to obtain a standard input tensor. On container resources pre-allocated by the predictive resource scheduling center for the standardized operating environment, the standard input tensor is sent to the standardized operating environment to execute model inference, yielding inference output. The inference output is then standardized and encapsulated according to the output post-processing semantics to generate standardized inference response data, which is returned to the requester as the response result of the user-uploaded raw inference data. Specifically, after receiving the user's raw inference data, the gateway extracts its current data semantic features (such as noisy RGB images, variable sizes, etc.) and compares them with the target model semantics passed from Module 1. In and By comparing the results, the semantic gap difference set ΔS is calculated.

[0051] The semantic difference set ΔS refers to the set of differences between the actual semantic features of the user's original data and the standard input semantic features expected by the model. For example, if the user uploads a noisy JPG image of varying size, and the model expects a 640×640 normalized RGB tensor as input, then ΔS = {decode, denoise, resize to 640×640, normalize to [0,1]}.

[0052] The semantic translation service gateway performs trial runs and verification of the adapter scripts in an isolation sandbox, specifically including: Extract a sample slice of the user's original inference data, run the adapter script in the isolation sandbox to perform preprocessing operations on the sample slice, and obtain the output tensor; Verify whether the tensor specifications of the output tensor meet the preset conditions, which include shape consistency conditions and numerical distribution validity conditions. If any preset condition is not met, the verification is deemed to have failed.

[0053] Before sending real requests to the model, the gateway starts the adapter script in a lightweight isolation sandbox. For the user's original inference data The test run is performed on tiny slices (or low-resolution sampled copies). An isolation sandbox is a resource-constrained, lightweight execution environment isolated from the main process, used to safely test the execution of unknown code without affecting the main service.

[0054] After execution, a tensor specification checker is introduced to verify whether the output tensor meets the model's input requirements (e.g., correct shape, no infinite or non-numeric values ​​(NaN), and no gradient explosion in the distribution). The verification formula is as follows:

[0055] in, The tensor output for the sandbox trial run. and This is the threshold for normal distribution. If... The adaptation was deemed successful. (Target Input Tensor Shape): The shape of the standard input tensor expected by the model, derived from the model's semantic feature vector. Obtained from [the source]. : This represents the dimension information of tensor T. For example, (1,3,640,640) represents a four-dimensional tensor with a batch size of 1, 3 channels (RGB), a height of 640 pixels, and a width of 640 pixels. This means that for any element in tensor T xThis means that all elements in the tensor must satisfy the range of values.

[0056] If Valid(T)=0 (for example, due to excessive noise in the input image, direct nearest neighbor scaling leads to abnormal data distribution), the gateway triggers a semantic enhancement strategy, dynamically injects preprocessing steps such as Gaussian denoising into ΔS, regenerates the adapter script, and verifies it again in the sandbox until the verification passes.

[0057] Finally, the gateway will be officially applied to the full user data through the verified adapter. Output standard input tensor = A_script( The data is fed into the model for inference, and the output is processed according to the post-output semantics. It is then reverse-encapsulated into standard structured data (such as JSON format) and returned to the requester.

[0058] Enhancements and adjustments were made to the adapter script, including: If the output tensor fails tensor specification validation, a new data preprocessing operation is added to the semantic difference set. The adapter script is regenerated based on the updated semantic difference set, and a trial run validation is performed again in the isolation sandbox. The trial run validation includes: extracting a sample slice of the user's original inference data, running the regenerated adapter script to perform preprocessing operations on the sample slice, obtaining the output tensor, and performing tensor specification validation. If the validation still fails, the steps of adding the new data preprocessing operation and regenerating the adapter script are iteratively executed until the validation passes.

[0059] For example: The client uploads a noisy surveillance JPG image (i.e., a JPEG image taken by a surveillance camera with significant noise interference). The gateway extracts the user data semantics as a noisy, variable-size RGB image, and compares it with the target semantics. SEM comparison yields the difference ΔS = {Decode, Resize to 640, Normalize}.

[0060] The gateway reads the operator capability contract C_op of module two and finds that the current environment does not support bilinear interpolation. Therefore, it automatically generates an adapter script A_script that uses nearest neighbor resize.

[0061] Before the actual call, the gateway is used in an isolation sandbox. The image is processed by sampling a slice (i.e., a small pixel region extracted from the original image for quick testing of the adapter script's correctness). The tensor specification validator found NaN (Not a Number, usually generated by illegal calculations such as taking the logarithm of zero or dividing by zero) in a region of the output tensor. Due to image noise and the lack of smoothing in nearest neighbor interpolation, the data distribution is abnormal, and Valid(T) is determined to be 0.

[0062] The gateway triggers a self-healing mechanism, dynamically injecting a Gaussian denoising (an image processing method that uses Gaussian kernels to smooth and filter images to remove noise) pre-process into ΔS, regenerating the adapter script, and performing sandbox testing again. This test passed (the output tensor shape was correct and all element values ​​were within the normal range). The gateway then officially applied the new adapter to the entire image, successfully outputting a standard tensor. Input into model inference The predictive resource scheduling center communicates with both the model semantic access module and the container orchestration cluster that deploys the semantic transformation service gateway. It receives model semantic feature vectors and extracts model computational complexity information; it collects real-time operational status metrics for each model service, predicts request volume for future periods, and generates a predicted request volume. Combining the predicted request volume and model computational complexity information, it generates a resource scheduling scheme that meets service level agreement constraints. This scheme includes the target number of container instances and the resource quotas for each instance. When the predicted request volume exceeds a preset security threshold, it pre-deploys the resource scheduling scheme to the container orchestration cluster, triggering the pre-deployment of container images and the elastic expansion of the number of container instances.

[0063] Predictive resource scheduling centers predict request volumes for future periods, including: The system collects request volume, response time, CPU utilization, and memory utilization of each model service in real time to construct multi-dimensional time-series features. It then uses an online learning model to predict the request volume for future periods based on the multi-dimensional time-series features within the historical time-series window.

[0064] Specifically, the request volume of each model service is collected in real time. Response time CPU utilization Memory utilization The formula for constructing a multidimensional time series dataset is as follows:

[0065] in ， Requests per second (QPS); Response time; CPU utilization; Memory utilization. This is a four-dimensional runtime feature vector collected at time t, containing information on four dimensions: request volume, response time, CPU utilization, and memory utilization.

[0066] A lightweight online learning model is employed to predict the number of requests, Q_{t+Δt}, at future time t+Δt based on historical window data. An online learning model is a machine learning model that can incrementally update its parameters as new data arrives, eliminating the need to retrain on the entire dataset each time, making it suitable for real-time prediction scenarios.

[0067] The prediction formula is as follows:

[0068] in, Set the size of the history window; These are model parameters that are dynamically updated based on online data. If the predicted value... This will trigger capacity expansion ahead of schedule.

[0069] In the scaling decision, the semantic feature vector of the model passed from module one is combined. The computational complexity labels (such as input tensor size and operator type) are dynamically adjusted to construct a fine-grained resource scheduling function:

[0070] in, Resources allocated (number of compute instances, CPU / memory quota, etc.); This represents a fine-grained resource scheduling mapping relationship. :future Predict the request volume in step 2 (from the load trend prediction in step 2). : Model semantic feature vector (from Module 1, containing computational complexity labels such as input tensor size and operator type). If the predicted value If this happens, capacity expansion will be triggered ahead of schedule. The goal is to find the minimum resource cost configuration while meeting the Service Level Agreement (SLA), thereby reducing costs and increasing efficiency.

[0071] For example, the model experiences a surge in call volume during a peak business period. The scheduling center collects time-series characteristics in real time. (Including request volume, response time, CPU utilization, and memory utilization), the online learning prediction model calculates the number of requests after 5 minutes. This will exceed 150% of the single instance's processing capacity (i.e., exceed the preset safety threshold Q_safe). The system immediately triggers pre-scaling, increasing the number of container instances from 1 to 3, and combining... The complexity of the operators (such as nearest neighbor interpolation, which is a computationally intensive operator requiring significant CPU resources) is used to allocate corresponding CPU resources to each instance. When actual peak traffic arrives, all three instances are ready, service response latency remains stable, and there are no timeouts or service denials due to insufficient resources.

[0072] A method for deploying heterogeneous models based on semantic parsing and dependency solving includes: Receive model files uploaded by users, parse the model files to extract model input semantics, input preprocessing semantics, model output semantics, output postprocessing semantics, and the set of underlying operator constraints required to perform input preprocessing semantics, generate model semantic feature vectors based on the extraction results and store them; Based on the model's semantic feature vector, the underlying operator constraint set is extracted; global software package metadata is obtained, and a cross-layer dependency graph containing software package nodes, operator nodes, and the constraint relationships between them is constructed; conflict detection is performed on the cross-layer dependency graph, and if a conflict is detected, dynamic relaxation solution with the goal of operator preservation is executed to generate a set of compatible software package versions; if the solution fails, a relaxation request is fed back to trigger a downgrade adjustment of the input preprocessing semantics until the solution is successful; a standardized runtime environment is constructed based on the successfully solved set of software package versions, the standardized runtime environment is packaged into a container image and pushed to the container orchestration cluster, and operator scanning is performed on the standardized runtime environment to generate operator capability contracts; Based on the model semantic feature vector and operator capability contract, the system receives the user's raw inference data uploaded by the business side; extracts the user data semantics from the user's raw inference data, compares the user data semantics with the model input semantics, and generates a semantic difference set; dynamically generates an adapter script based on the semantic difference set and operator capability contract; performs trial runs and verification of the adapter script in an isolated sandbox; if the verification fails, the adapter script is enhanced, adjusted, and re-verified until it passes; processes the user's raw inference data using the verified adapter script to obtain a standard input tensor; sends the standard input tensor into the standardized operating environment on container resources pre-allocated by the predictive resource scheduling center for the standardized operating environment to execute model inference and obtain inference output; standardizes and encapsulates the inference output according to the output post-processing semantics to generate standardized inference response data, which is returned to the requester as the response result of the user's raw inference data. Based on the model's semantic feature vector, the computational complexity information of the model is extracted; the operational status indicators of each model service are collected in real time to predict the request volume for future periods, thus obtaining the predicted request volume; combining the predicted request volume and the model's computational complexity information, a resource scheduling scheme that meets the service level agreement constraints is generated. The resource scheduling scheme includes the target number of container instances and the resource quota corresponding to each instance; when the predicted request volume exceeds the preset security threshold, the resource scheduling scheme is issued to the container orchestration cluster in advance, triggering the pre-deployment of container images and the elastic expansion of the number of container instances.

[0073] This application also discloses a terminal device, including a memory and a processor. The memory stores a computer program that can run on the processor. When the processor loads and executes the computer program, it adopts a heterogeneous model deployment method based on semantic parsing and dependency solving.

[0074] The terminal device can be a computer device such as a desktop computer, a laptop computer, or a cloud server. The terminal device includes, but is not limited to, a processor and a memory. For example, the terminal device may also include input / output devices, network access devices, and buses.

[0075] The processor can be a central processing unit (CPU). Of course, depending on the actual use, it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), off-the-shelf programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc., and this application does not limit it.

[0076] The memory can be an internal storage unit of the terminal device, such as a hard disk or RAM of the terminal device, or an external storage device of the terminal device, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD), or flash memory card (FC) equipped on the terminal device. Furthermore, the memory can be a combination of internal storage units and external storage devices of the terminal device. The memory is used to store computer programs and other programs and data required by the terminal device. The memory can also be used to temporarily store data that has been output or will be output. This application does not limit this.

[0077] In this terminal device, a heterogeneous model deployment method based on semantic parsing and dependency solving in the above embodiments is stored in the terminal device's memory and loaded and executed on the terminal device's processor for convenient use.

[0078] This application also discloses a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it employs a heterogeneous model deployment method based on semantic parsing and dependency solving as described in the above embodiments.

[0079] The computer program can be stored in a computer-readable medium. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or certain middleware. The computer-readable medium includes any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the computer-readable medium includes, but is not limited to, the above-mentioned components.

[0080] In this computer-readable storage medium, a heterogeneous model deployment method based on semantic parsing and dependency solving in the above embodiments is stored in the computer-readable storage medium and loaded and executed on the processor to facilitate the storage and application of the above method.

[0081] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0082] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

Claims

1. A heterogeneous model deployment system based on semantic parsing and dependency solving, characterized in that, This includes a model semantic access module, a dependency solving encapsulation engine, a semantic transformation service gateway, and a predictive resource scheduling center; The model semantic access module is used to receive model files uploaded by users, parse the model files to extract model input semantics, input preprocessing semantics, model output semantics, output postprocessing semantics, and the set of underlying operator constraints required to execute the input preprocessing semantics, and generate and store model semantic feature vectors based on the extraction results. The dependency solving encapsulation engine is communicatively connected to the model semantic access module. It is used to receive the model semantic feature vector, extract the underlying operator constraint set, obtain global software package metadata, construct a cross-layer dependency graph containing software package nodes, operator nodes and the constraint relationship between them, perform conflict detection on the cross-layer dependency graph, and if a conflict is detected, perform dynamic relaxation solving with the goal of operator preservation to generate a set of compatible software package versions. If the solution fails, a relaxation request is sent to the model semantic access module to trigger a downgrade adjustment of the input preprocessing semantics until the solution is successful. A standardized runtime environment is built based on the set of successfully solved software package versions. The standardized runtime environment is packaged into a container image and pushed to the container orchestration cluster. Operator scanning is performed on the standardized runtime environment to generate operator capability contracts. The semantic conversion service gateway is communicatively connected to the model semantic access module and the dependency solving encapsulation engine, respectively, and is used to receive the model semantic feature vector and the operator capability contract, receive the user's original inference data uploaded by the business side; extract the user data semantics of the user's original inference data, compare the user data semantics with the model input semantics, and generate a semantic difference set. Based on the semantic difference set and the operator capability contract, the adapter script is dynamically generated; The adapter script is tested and verified in an isolated sandbox. If the verification fails, the adapter script is enhanced and adjusted and then re-verified until the verification passes. The verified adapter script is used to process the user's original inference data to obtain a standard input tensor; the standard input tensor is sent to the standardized runtime environment on the container resources pre-allocated by the predictive resource scheduling center to perform model inference and obtain inference output; the inference output is standardized and encapsulated according to the output post-processing semantics to generate standardized inference response data, which is returned to the requester as the response result of the user's original inference data. The predictive resource scheduling center is communicatively connected to the model semantic access module and the container orchestration cluster that deploys the semantic conversion service gateway, respectively, and is used to receive the model semantic feature vector and extract model computational complexity information. The system collects real-time operational status metrics for each model service to predict future request volumes. Combining these predicted request volumes with the model's computational complexity information, it generates a resource scheduling scheme that satisfies the Service Level Agreement (SLA) constraints. This resource scheduling scheme includes the target number of container instances and the resource quotas for each instance. When the predicted request volume exceeds a preset security threshold, the resource scheduling scheme is pre-deployed to the container orchestration cluster, triggering the pre-deployment of the container image and the elastic expansion of the number of container instances.

2. The heterogeneous model deployment system based on semantic parsing and dependency solving according to claim 1, characterized in that, The model semantic access module parses the model file to extract model input semantics, input preprocessing semantics, model output semantics, and output postprocessing semantics, including: The model file is parsed to obtain the model computation graph. The input node features and output node features in the model computation graph are matched and processed by the built-in operator prototype network. The model input semantics, the input preprocessing semantics, the model output semantics, and the output postprocessing semantics are automatically inferred and output.

3. The heterogeneous model deployment system based on semantic parsing and dependency solving according to claim 1, characterized in that, Conflict detection is performed on the cross-layer dependency graph, including: Calculate the dependency conflict coefficient for each pair of software package nodes in the graph; When the dependency conflict coefficient does not exceed a preset threshold, it is determined that no conflict has been detected. When the dependency conflict coefficient exceeds a preset threshold, a conflict is detected.

4. The heterogeneous model deployment system based on semantic parsing and dependency solving according to claim 1, characterized in that, Dynamic relaxation solutions aimed at preserving operators include: A multi-objective optimization function is constructed, which includes version stability objective and operator retention loss objective. The optimal solution vector is searched in the constraint space of the cross-layer dependency graph as the set of compatible software package versions. When the multi-objective optimization function diverges, the solution is determined to have failed. The multi-objective optimization function is expressed as: in, The total loss value of the multi-objective optimization function. The set of compatible software package versions to be solved. This represents the total number of dependent packages. For the first Stability ratings for each software package version and As weight, For the underlying operator constraint set, The loss function is the operator, and divergence means that the multi-objective optimization function does not have a finite minimum value.

5. The heterogeneous model deployment system based on semantic parsing and dependency solving according to claim 1, characterized in that, The semantic translation service gateway performs trial runs and verification of the adapter script in an isolation sandbox, specifically including: A sampled slice of the user's original inference data is extracted, and the adapter script is run in the isolation sandbox to perform preprocessing operations on the sampled slice to obtain the output tensor. Verify whether the tensor specification of the output tensor meets preset conditions, including shape consistency conditions and numerical distribution validity conditions; If any preset condition is not met, the verification is deemed to have failed.

6. The heterogeneous model deployment system based on semantic parsing and dependency solving according to claim 1, characterized in that, The enhancement and adjustment of the adapter script includes: If the tensor specification verification of the output tensor fails, a new data preprocessing operation is added to the semantic difference set. Based on the updated semantic difference set, the adapter script is regenerated, and a trial run verification is performed again in the isolation sandbox. The trial run verification includes: extracting a sample slice of the user's original inference data, running the regenerated adapter script to perform preprocessing operations on the sample slice, obtaining the output tensor, and performing tensor specification verification. If the verification still fails, the steps of adding the new data preprocessing operation and regenerating the adapter script are iteratively executed until the verification passes.

7. The heterogeneous model deployment system based on semantic parsing and dependency solving according to claim 1, characterized in that, Predictive resource scheduling centers predict request volumes for future periods, including: The system collects request volume, response time, CPU utilization, and memory utilization of each model service in real time to construct multi-dimensional time-series features. An online learning model is then used to predict the request volume for future periods based on these multi-dimensional time-series features within a historical time-series window.

8. A method for deploying heterogeneous models based on semantic parsing and dependency solving, characterized in that, include: The system receives model files uploaded by users, parses the model files to extract model input semantics, input preprocessing semantics, model output semantics, output postprocessing semantics, and the set of underlying operator constraints required to execute the input preprocessing semantics, and generates and stores model semantic feature vectors based on the extraction results. Based on the model semantic feature vector, the underlying operator constraint set is extracted; global software package metadata is obtained, and a cross-layer dependency graph containing software package nodes, operator nodes, and the constraint relationships between them is constructed; conflict detection is performed on the cross-layer dependency graph, and if a conflict is detected, dynamic relaxation solution with the goal of operator preservation is executed to generate a compatible software package version set; if the solution fails, a relaxation request is fed back to trigger a downgrade adjustment of the input preprocessing semantics until the solution is successful; a standardized runtime environment is constructed based on the successfully solved software package version set, the standardized runtime environment is packaged into a container image and pushed to the container orchestration cluster, and operator scanning is performed on the standardized runtime environment to generate operator capability contracts; Based on the model semantic feature vector and the operator capability contract, the system receives the user's original inference data uploaded by the business side; extracts the user data semantics from the user's original inference data; compares the user data semantics with the model input semantics; and generates a semantic difference set. Based on the semantic difference set and the operator capability contract, the adapter script is dynamically generated; The adapter script is tested and verified in an isolated sandbox. If the verification fails, the adapter script is enhanced and adjusted and then re-verified until the verification passes. The verified adapter script is used to process the user's original inference data to obtain a standard input tensor; the standard input tensor is sent to the standardized runtime environment on the container resources pre-allocated by the predictive resource scheduling center to perform model inference and obtain inference output; the inference output is standardized and encapsulated according to the output post-processing semantics to generate standardized inference response data, which is returned to the requester as the response result of the user's original inference data. Based on the semantic feature vector of the model, extract the computational complexity information of the model; The system collects real-time operational status metrics for each model service to predict future request volumes. Combining these predicted request volumes with the model's computational complexity information, it generates a resource scheduling scheme that satisfies the Service Level Agreement (SLA) constraints. This resource scheduling scheme includes the target number of container instances and the resource quotas for each instance. When the predicted request volume exceeds a preset security threshold, the resource scheduling scheme is pre-deployed to the container orchestration cluster, triggering the pre-deployment of the container image and the elastic expansion of the number of container instances.

9. A terminal device, comprising a memory and a processor, characterized in that, The memory stores a computer program that can run on a processor, and when the processor loads and executes the computer program, it employs the method described in any one of claims 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is loaded and executed by the processor, it employs the method described in any one of claims 8.

Images