Containerized deployment system and method for ASIC edge equipment

Abstract

The invention provides a containerized deployment system and method for ASIC edge equipment. The containerized deployment system for the ASIC edge equipment comprises a basic mirror image construction module, a running environment list generation and signature module, a product library module, a security and authority module, a deployment agent module, a container engine adaptation module, a runtime detection module and a monitoring and log module. An ASIC operation precondition is abstracted into a machine-readable operation environment contract according to the operation environment list, a deployment agent drives an equipment side, and a binary interface and an equipment node are applied to execute automatic detection and generate deployment guidance information; after the container is started, consistency detection is carried out and a detection report is output when running is executed, and finally a closed-loop process including automatic detection, deployment guidance, detection verification and rollback alarm is formed, so that an application mirror image can stably run on ASIC edge equipment without repeatedly carrying bottom layer dependence.

Description

Technical Field

[0001] This invention relates to the field of container deployment technology, and more specifically to a containerized deployment system and method for ASIC edge devices. Background Technology

[0002] In edge intelligent inference scenarios, applications are typically delivered in containers to enable cross-device software distribution, version management, and automated operation and maintenance. For general-purpose processor platforms such as x86 and ARM, container runtimes (such as Docker and containerd), while adhering to the OCI (Open Container Initiative) image specifications, usually only need to address common issues such as image pulling, dependency layering, process isolation, and network / storage mounting. Business applications have relatively weak dependencies on the underlying hardware, and deployment consistency is mainly determined by the image content and runtime parameters.

[0003] For other hardware architectures such as GPUs and FPGAs, engineering practices typically employ a unified driver execution on the host side and injection of access capabilities on the container side to lower the barrier to entry. For example, in GPU scenarios, vendor-provided container runtime extensions (such as device injection toolchains and runtime library mapping mechanisms) can automatically complete device node mapping, driver compatibility checks, and runtime library mounting during the startup phase, enabling application images to be reused within a relatively unified GPU ecosystem. In FPGA scenarios, the host machine relies more on loading bitstreams and firmware and providing user-space runtime libraries, while the container side only carries upper-layer application logic.

[0004] However, the above solution often fails when edge devices are equipped with ASIC acceleration chips, for the following reasons: 1. Significant differences exist between different ASIC vendors in their driver interfaces, firmware versions, runtime ABIs, device node naming, and permission models; 2. The same product may exhibit different behaviors in different batches / different firmware versions; 3. Edge devices also have characteristics such as being offline or in weak network conditions, having limited storage and computing power margins, strong temperature control and power consumption constraints, and short remote operation and maintenance windows, making the success rate and reproducibility of one-time deployment particularly critical.

[0005] Therefore, containerized deployment of ASIC edge devices is not just about launching the container, but also about automatically determining and guiding hardware reachability, version compatibility, library availability, and minimal self-testing before and after startup.

[0006] When edge devices are equipped with ASIC acceleration chips, containerized deployment typically presents the following problems: 1. Reliance on manual installation and debugging: Deployment personnel often need to manually install or verify components such as drivers, inference runtime libraries, operator libraries, mathematical acceleration libraries, and dynamic link libraries on the device side, and check the paths, symbols, permissions and configuration items one by one. The deployment cycle is long and the repeatability is poor. 2. Inconsistent software and hardware versions cause runtime failures: Issues such as driver version mismatch with runtime library ABI (Application Binary Interface), incorrect device node mapping or insufficient permissions, and missing dynamic libraries causing inference program loading failures occur frequently, resulting in containers that can be launched but are not usable; 3. Consistency of batch deployment of similar equipment is difficult to guarantee: Manual operation by different batches of equipment or different maintenance personnel can cause the operating environment to drift, resulting in different performance of the same image, which creates maintenance risks; 4. Lack of closed-loop verification during container startup: Current practices often consider a container to be usable once it starts successfully, but lack runtime detection mechanisms for ASIC accelerator availability, driver / library consistency, and critical operator self-checks, resulting in problems only being exposed during business operation.

[0007] Therefore, there is an urgent need for a containerized deployment method for edge devices equipped with ASIC acceleration chips to solve the technical problem that deployment relies heavily on experience and is difficult to reproduce in batches. Summary of the Invention

[0008] To address the problems of complex dependencies, inconsistent environments, and unstable operation in the containerized deployment of ASIC edge devices in existing technologies, this invention provides a containerized deployment system and method for ASIC edge devices.

[0009] This invention relates to a containerized deployment system for ASIC edge devices, comprising a base image building module, a runtime environment manifest generation and signing module, an artifact repository module, a security and permissions module, a deployment agent module, a container engine adaptation module, a runtime detection module, and a monitoring and logging module. The base image building module is used to build a base container image adapted to ASIC edge devices; the runtime environment manifest generation and signing module is used to generate a runtime environment manifest, which includes driver version fields required by the base image, library manifest fields, device node requirement fields, probe entry fields, and signature fields, and signs the manifest and the base image; the artifact library module is used to store artifacts, including the base image, runtime environment manifest, and application image. These artifacts provide the deployment agent module with traceable image and manifest sources and support signature verification, version management, and auditing; the security and permissions module is used to control image signature verification, startup template auditing, and privileged capabilities. The module includes: Scope of application deployment and driver loading licensing policies; Deployment agent module: Receives application deployment requests, pulls or locates the base image and runtime environment list, performs self-checks on the runtime environment, conducts compatibility assessments based on the detection results and runtime environment list, and generates deployment guidance information; Container engine adaptation module: Solidifies ASIC edge device access capabilities at the host machine container engine level, including default device node mapping rules, default capability templates, security configuration templates, key environment variable injection rules, and necessary resource limit configurations; Runtime probing module: Performs probing and consistency checks after container startup; Monitoring and logging module: Collects deployment logs, probing results, and container runtime performance metrics.

[0010] Furthermore, it also includes a hardware capability profiling module, which is used to collect the capabilities of the device-side ASIC, establish, maintain and output the hardware capability profile of the device-side ASIC, and is used for the selection of the base image version identifier and the self-adaptation of running parameters.

[0011] Furthermore, it also includes a pre-compiled cache and warm-up module, which manages and loads ASIC-related pre-compiled artifacts and runtime caches, and performs warm-up when the container starts or the service is switched to reduce cold start overhead.

[0012] Furthermore, it also includes an adaptive optimization module, which is used to collect operational metrics during container operation and dynamically adjust concurrency, batch size, precision mode, or operator implementation based on a strategy.

[0013] Furthermore, it also includes a fault isolation and recovery module, which is used to perform container-level isolation and recovery after detecting driver anomalies, memory leaks, operator timeouts, or probe failures.

[0014] This invention also provides a containerized deployment method for ASIC edge devices, implemented based on the aforementioned containerized deployment system for ASIC edge devices, comprising the following steps: S1: Build the base image: Write the dependencies required for ASIC operation into the base image, form a versioned image identifier base_tag, and publish it to the artifact repository; S2: Runtime environment manifest generation and signing: Generate manifest(base_tag) and sign it, then publish it to the artifact repository; S3: Build application image: Build application image based on versioned image identifier base_tag. The application image contains only business logic, inference model and application configuration and is stored in the artifact repository. S4: Automatic detection and deployment guidance: Read the manifest(base_tag), obtain the required device nodes and environment variable rules, and detect the device-side parameters. Determine whether the device-side parameters have reached the compatibility threshold. If not, generate deployment guidance information, output the differences and recommended actions. If yes, generate the startup template run_template. S5: Solidified Device Access Configuration: The container engine adaptation module applies the device node mapping, environment variables, and permission configuration in the startup template run_template to the container creation parameters, enabling the container to access ASICs; S6: Start the container and perform a probe: The runtime probe module performs a self-check within the container and generates a probe report. If the probe report indicates that the probe has passed, the container enters the business operation and the monitoring and logging module continuously collects relevant metrics. If the probe fails, the deployment agent module will stop, roll back to a compatible version, or report an alarm according to the policy.

[0015] Furthermore, a hardware feature fingerprint is constructed for the runtime environment, and a target fingerprint is constructed for the base image list of the artifact library. The hardware feature fingerprint and the target fingerprint include asic_vendor, asic_model, firmware / driver main version, runtime ABI identifier, key device node set and key library summary set. In step S4, the deployment agent module sorts multiple candidate base image versions based on the similarity between the hardware feature fingerprint and the target fingerprint, selects the base_tag corresponding to the base image version with the highest similarity and meeting the preset threshold, and generates the run_template accordingly.

[0016] Furthermore, it also includes the step of establishing a hardware capability profile: collecting the ASIC capabilities on the device side, establishing and maintaining the capability profile of the ASIC on the device side, and outputting the capability profile for the selection of the base image version identifier and the self-adaptation of running parameters.

[0017] It also includes cold start preheating and cache loading steps. After the probe_report indicates that the test has passed, before the service starts, the pre-compiled cache and preheating module loads or generates operator cache and model compilation cache based on the base image version identifier base_tag and hardware capability profile.

[0018] Furthermore, it also includes an adaptive optimization step: during business operation, the adaptive optimization module periodically collects temperature, power consumption, bandwidth and load indicators, and dynamically adjusts the concurrency, batch size, precision mode or operator implementation selection under the constraints of the service level agreement. When a risk of thermal throttling or a trend of power consumption exceeding the limit is detected, a load reduction action is triggered and audit information is recorded.

[0019] Furthermore, it also includes fault isolation and recovery steps: when the monitoring and logging module or runtime probe detects an anomaly during runtime, the fault isolation and recovery module performs controlled recovery on the target container, or triggers a base image version switch or rollback.

[0020] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention constructs a base container image adapted to ASIC and abstracts the ASIC running prerequisites into a machine-readable runtime environment contract using a runtime environment manifest; during the deployment phase, the deployment agent performs automatic detection on the device-side driver, application binary interface and device node and generates deployment guidance information; after the container starts, it performs runtime consistency detection and outputs a structured detection report, and finally forms a closed-loop process of automatic detection - guided deployment - detection verification - rollback alarm, so that the application image can run stably on ASIC edge devices without repeatedly carrying underlying dependencies.

[0021] Furthermore, this invention moves the dependencies of drivers and runtime libraries required for ASIC operation forward and solidifies them: dependency integration and environment variable configuration are completed during the base image building stage, transforming deployment from manual installation and configuration to building and starting based on the base image, avoiding repeated installation and debugging on the device side, and solving the problem that the deployment of existing technologies is highly dependent on deployment personnel; this invention improves deployment success rate and maintainability by establishing post-startup detection and closed-loop control, automatically performing hardware availability detection and version consistency verification after the container starts, outputting clear diagnostic information and triggering rollback or alarms when anomalies occur. Attached Figure Description

[0022] To more clearly illustrate the solutions in this invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram of the containerized deployment system for ASIC edge devices according to the present invention; Figure 2 This is a flowchart of the containerized deployment method for ASIC edge devices according to the present invention. Detailed Implementation

[0024] Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings are used to distinguish different objects, not to describe a particular order.

[0025] In this invention, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment to other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this invention can be combined with other embodiments.

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0027] This invention provides a containerized deployment method and system for ASIC edge devices. It constructs a base container image adapted to ASICs and abstracts the ASIC runtime prerequisites into a machine-readable runtime environment contract using a runtime environment manifest. During the deployment phase, a deployment agent automatically detects device-side drivers, application binary interfaces, and device nodes, generating deployment guidance information. After container startup, runtime consistency detection is performed, and a structured detection report is output. This ultimately forms a closed-loop process of automatic detection—guided deployment—detection verification—rollback alarm, enabling application images to run stably on ASIC edge devices without repeatedly carrying underlying dependencies. ASIC (Application-Specific Integrated Circuit) is used for accelerating specific functions; in this invention, it is used for accelerating artificial intelligence inference. like Figure 1 As shown, the containerized deployment system for ASIC edge devices of the present invention includes a base image building module, a runtime environment manifest generation and signing module, an artifact library module, a security and permission module, a deployment agent module, a container engine adaptation module, a runtime detection module, and a monitoring and logging module. Preferably, in order to improve the stability of performance during operation, the present invention also includes functional modules for optimization during the process, including a hardware capability profiling module, a pre-compilation caching and preheating module, an adaptive optimization module, and a fault isolation and recovery module.

[0028] As an embodiment of the present invention, the base image building module, runtime environment manifest generation and signing module, and artifact library module are set on the building side, while the remaining modules are implemented directly on the device side of the edge device. Of course, the base image building module, runtime environment manifest generation and signing module, and artifact library module can also be built on the device side.

[0029] Specifically, the base image building module in this example is used to build a base container image adapted to ASICs. Within the image, it pre-configures the user-space components, runtime libraries, operator libraries, mathematical acceleration libraries, and necessary system configuration files required for inference execution, and configures environment variables and paths (such as LD_LIBRARY_PATH). This module is the first module on the build side, and its purpose is to collect all relevant driver information from the vendor, vendor software development kit (SDK) information, and the computing unit libraries required by the ASIC chip to build the base image layer.

[0030] The runtime environment manifest generation and signing module in this example generates a runtime environment manifest, which includes at least the following fields: `asic_driver_version` (required by the base image), `lib_list` (required by the library manifest), `required_devices` (required by the device node manifest), `probe_entry` (probe entry point manifest), and `signature` (signature). This module then signs the manifest and the base image to ensure consistency and reliability in delivery. This module connects to the base image building module, listing the drivers, software packages, library nodes, and additional probe flags collected in the first step, and signing the base image of the first module to ensure subsequent consistency and compliance checks.

[0031] The runtime environment manifest in this example can be bound to the base image digest of the Open Container Initiative (OCI Artifact / Attestation) and distributed with the artifact repository, or it can be imported offline through the local artifact repository on the device side. In both cases, manifest_digest is used as the consistency verification key.

[0032] In this example, the artifact repository module stores artifacts, including base images, runtime environment manifests, and application images. This provides the Deploy Agent module with traceable image and manifest sources and supports signature verification, version management, and auditing. This module not only contains base images but also the final application layer application images, as well as a collection of runtime manifest contents recorded by the second module.

[0033] In this example, the security and permissions module controls image signature verification, startup template auditing, the scope of privileged capabilities, and driver loading permission policies to prevent uncontrolled high-privilege containers from posing risks to the system. After the build process is completed on the build side and reaches the device side, the security and permissions module first verifies the signature (Manifest), audits the permission scope of the running template, and the driver loading policy. Only after passing these checks is the process handed over to the deployment agent for further processing.

[0034] In this example, the deployment agent module receives application deployment requests, pulls or locates the base image and runtime environment manifest; performs automatic detection on the device-side driver version, runtime ABI identifier, and device node set; performs compatibility assessment based on the detection results and the manifest and generates deployment guidance information deploy_advice; when compatible, it generates a startup template (run template, run_template) containing device mapping, environment variables, and permission capabilities, and schedules the container runtime to complete container creation and startup; when incompatible, it outputs executable repair suggestions or triggers image selection / rollback policies.

[0035] In this example, the container engine adaptation module is used to solidify ASIC device access capabilities at the host machine's container engine level. This includes at least: default device node mapping rules, default capability templates and security configuration templates (seccomp / profile templates), key environment variable injection rules, and necessary resource limit configurations (ulimit / shared memory configuration), enabling container startup to meet ASIC access requirements without manual parameter passing. In the flowchart, this is represented as follows: the container engine adaptation module automatically injects device node mappings, environment variables, capability / security policies, and resource limits based on the run template, eliminating the need for manual input of container startup parameters (such as `--device`, `--privileged`, etc.).

[0036] The runtime probing module performs probing and consistency checks after the container starts, including at least: device node accessibility checks, driver version reading, critical library loadability checks, minimal inference self-checks or operator self-checks, and outputs standardized probe results (probe_report). In this example, the runtime probing module can be implemented by a self-checking program within the container, or it can be executed by a sidecar / daemon process on the host machine entering the container namespace and outputting results in the same probe_report structure.

[0037] The monitoring and logging module in this example is used to collect deployment logs, probe results, container performance metrics, and provide alerts and rollback basis.

[0038] Preferably, this example also includes a hardware capability profiling module. This module is used to collect and maintain the hardware capability profile of the ASIC on the device side, including at least the ASIC vendor identifier field asic_vendor, the ASIC model identifier field asic_model, firmware / driver version, runtime ABI identifier, available operator set, and available memory and bandwidth characteristics. The hardware capability profile output by this module is used for the selection of the base image version identifier (base_tag) and the self-adaptation of runtime parameters. Since the hardware capability profiling module contains basic information about the ASIC chip, i.e., device parameters, it can be used to subsequently output the selected base image version and the specific parameters selected during container startup, such as memory allocation limits and operator calling rules.

[0039] In this example, the pre-compilation cache and warm-up module manages and loads ASIC-related pre-compiled artifacts and runtime caches, including at least operator / kernel caches, model compilation caches, and dynamic link symbol caches. Warming is performed during container startup or service switching to reduce cold start overhead. Before the container officially connects to services, the pre-compilation cache and warm-up module organizes and loads operator / model compilation caches according to the base image identifier (base_tag), application interface (abi_id), and driver version rule (driver_version_rule), minimizing warm-up for commonly used models to reduce cold start latency.

[0040] In this example, the runtime adaptive optimization module collects metrics such as temperature, power consumption, bandwidth, and load. Based on a strategy, it dynamically adjusts concurrency, batch size, precision mode, and operator implementation to strike a balance between stability and performance. Specifically, during business operation, the monitoring module continuously collects temperature, power consumption, bandwidth, and load. The runtime adaptive optimization module dynamically adjusts concurrency, batch size, precision mode, and operator implementation based on hardware capability profiles, proactively reducing load before approaching hardware limits.

[0041] The fault isolation and recovery module is used to perform container-level isolation and recovery after detecting driver anomalies, memory leaks, operator timeouts, or probe failures. This includes resetting the container context, switching to a compatible base image version, or triggering a controlled degradation strategy. In this example, once a fault such as a driver anomaly, library loading failure, or operator timeout occurs, the fault isolation and recovery module performs a partial reset / reconstruction of the target container and switches or rolls back the base image version when necessary. At the same time, it records the relevant base_tag / manifest_digest, forming an auditable and rollbackable closed loop.

[0042] This invention uses a versioned base image as the runtime environment baseline, a container engine adaptation module as the carrier for solidifying device access capabilities, a runtime environment manifest as a machine-readable compatibility constraint and signature carrier, and runtime probes as a closed-loop verification method after startup. These elements work together to enable the deployment agent to generate a startup template without relying on manual configuration, and to automatically determine device reachability, version consistency, library availability, and minimal self-checks after startup. This achieves rapid containerized deployment and stable operation in ASIC scenarios.

[0043] Preferably, the base image building module of this invention uses traceable version numbers for the driver user-space components and runtime libraries and writes them into the runtime environment manifest; the container engine adaptation module injects device node mappings and environment variables according to fixed rules; the runtime probe module outputs a structured probe report (probe_report) based on deterministic probe steps; and the deployment agent performs consistency determination and rollback decisions based on the fields of the runtime environment manifest and probe_report. These key points ensure that the same base image version identifier (base_tag), the same runtime environment manifest, and the same startup template can repeatedly obtain consistent deployment results on similar devices.

[0044] To ensure that the preconditions are machine-readable and verifiable, the runtime environment manifest in this example may include the set of fields shown in Table 1: Table 1. List of fields for runtime environment Fields meaning base_tag Base image version identifier asic_vendor ASIC manufacturer identification asic_model ASIC Model / Generation Identification driver_version_rule Driver version compatibility rules abi_id Runtime ABI identifier required_devices Required set of device nodes required_caps Minimum set of capabilities required for a container (Capability / Security configuration seccomp / profile) lib_list List of necessary dynamic libraries and verification summary (e.g., library name soname + hash value). probe_entry Detect entry point (command / path / parameters) signature List Signature and Certificate Chain Identifier The deployment guidance information deploy_advice may include at least the following set of fields: The system includes the following fields: compatible (compatibility identifier), reason_codes (reason code set), diff_items (diff item set, representing driver / ABI / device node / library differences), recommended base image version (recommend_base_tags), recommended run_params (recommend run_params, representing device mapping / environment variables / permission templates), fix_actions (executable fix actions), and risk_level (risk level).

[0045] To ensure the deployment guidance information is executable, the reason_codes field and the fix_actions field can adopt a fixed mapping relationship, as shown in Table 2 in one embodiment of the present invention.

[0046] Table 2 Deployment Guidance Information reason_code Triggering conditions fix_actions (Fix Actions) DRIVER_MISMATCH V_d does not satisfy driver_version_rule Choose a compatible base_tag or perform a driver upgrade / downgrade. ABI_MISMATCH A_d and abi_id are inconsistent Switch to the base_tag that matches the ABI, or update the host runtime library / symbolic link. DEV_NODE_MISSING required_devices⊆D_r Output a list of missing device nodes and suggest loading modules / correcting mappings and permission templates. LIB_LOAD_FAIL lib_list loading test failed. Outputs the failure name / error code and suggests completing the library mapping or selecting a compatible base_tag. The probe report in this example can include at least the following set of fields: device_ok (device availability), driver_version (driver version), abi_id (ABI identifier), lib_ok (library loading result), self_test_ok (self-test result), failed_steps (failed steps), error_codes (error codes), timestamp (timestamp), base_tag (base image version identifier), and manifest_digest (manifest summary).

[0047] like Figure 2 As shown, the present invention also provides a containerized deployment method for ASIC edge devices, implemented based on the aforementioned containerized deployment system for ASIC edge devices, comprising the following steps: S1: Build the base image: Write the dependencies required for ASIC operation into the base image, form a versioned image identifier base_tag, and publish it to the artifact repository; S2: Runtime environment manifest generation and signing: Generate manifest(base_tag) and sign it, then publish it to the artifact repository; S3: Build application image: Build application image based on versioned image identifier base_tag. The application image only contains business logic, inference model and application configuration, does not repeatedly install underlying dependencies, and is stored in the artifact repository. S4: Automatic detection and deployment guidance: Read the manifest(base_tag), obtain the required device nodes and environment variable rules, and detect the device-side parameters to determine whether the device-side parameters meet the compatibility threshold. If not, generate deployment guidance information, outputting the differences (such as driver version mismatch, missing device nodes or unloadable libraries) and recommended actions (such as selecting compatible base_tag, installing / upgrading drivers, adjusting permission templates). If yes, generate the startup template run_template. S5: Solidified Device Access Configuration: The container engine adaptation module applies the device node mapping, environment variables, and permission configuration in the startup template run_template to the container creation parameters, enabling the container to access ASICs; S6: Start the container and perform probes: The runtime probe module performs a self-check within the container and generates a probe report. The probe report includes at least the fields device_ok, driver_version, lib_ok, and self_test_ok. If the probe report indicates that the probe passed, the container enters service operation and the monitoring and logging module continuously collects relevant metrics; if the probe fails, the deployment agent module stops the operation, rolls back to a compatible version, or reports an alarm according to the policy.

[0048] Preferably, the present invention further includes a step of establishing a hardware capability profile: collecting the ASIC capabilities on the device side, establishing and maintaining the capability profile of the ASIC on the device side, and outputting the capability profile for selecting the base image version identifier and adapting the running parameters.

[0049] Preferably, this example also includes cold start warm-up and cache loading steps. After the probe_report indicates that the probe has passed, before the service starts, the pre-compiled cache and warm-up module loads or generates operator cache and model compilation cache based on the base image version identifier base_tag and hardware capability profile. It can also perform minimal inference warm-up on commonly used models according to a strategy to reduce the startup latency of the first service request.

[0050] Furthermore, the present invention also includes an adaptive optimization step: during business operation, the adaptive optimization module periodically collects temperature, power consumption, bandwidth and load indicators, and dynamically adjusts the concurrency, batch size, precision mode or operator implementation selection under the condition of meeting the service level agreement constraints. When a risk of thermal throttling or a trend of power consumption exceeding the limit is detected, a load reduction action is triggered and audit information is recorded.

[0051] Furthermore, the present invention also includes fault isolation and recovery steps: when the monitoring and logging module or runtime detection detects an anomaly, the fault isolation and recovery module performs controlled recovery on the target container, such as resetting the inference context, releasing recyclable cache, or rebuilding the container and reusing the same base image, and triggering base image version switching or rollback when necessary to avoid the spread of the impact of a whole machine restart.

[0052] Based on the above process steps, in one embodiment of the present invention, the container runtime environment injection can be completed using a runtime parameter contract approach: during the automatic detection and deployment guidance phase, the deployment agent generates an auditable and reproducible startup template (run template, run_template), instead of performing secondary assembly on the base image or generating a new "device image" and writing it back to the artifact library. The startup template run_template includes at least: a base image version identifier field base_tag, a device node whitelist and permission mode field devices, a driver and runtime library read-only mount point or mapping rule field mounts, a key environment variable set field env, a minimum privilege capability and security policy field caps / seccomp_profile, a resource limit configuration field ulimits / shm_size (shared memory configuration), a core binding and NUMA policy field cpu_affinity / numa_policy, an optional hugepage configuration field hugepage, and a startup template digest field run_template_digest. The container engine adaptation module completes runtime injection according to these fixed rules, making "what to inject, with what privileges to inject, and whether minimum privileges are satisfied" calculable and rollbackable objects.

[0053] In one embodiment, the deployment agent can organize the device-side detection results into a hardware capability profile, profile_d, which includes at least fields such as V_d / A_d / D_r / L_d and asic_vendor / asic_model (as described in Table 2). It then compares the profile with the driver_version_rule / abi_id / required_devices / lib_list fields in the runtime environment manifest, outputting a set of differences, diff_items. Next, it generates reason_codes / fix_actions according to a fixed mapping table and provides executable repair paths, thereby transforming the abstract reasons for unavailability and recommended actions into deterministic field comparisons and rule table outputs.

[0054] In one embodiment, the pre-compilation cache and warm-up module reduces cold start latency while meeting the Service Level Agreement (SLA) by reducing operator and model compilation time and controlling the warm-up scale. The cache namespace is organized by the base image version identifier base_tag and the runtime environment constraint fields abi_id / driver_version_rule to avoid cache pollution between different versions.

[0055] The signature of the runtime environment manifest is used to ensure the consistency and trustworthiness of offline delivery: before generating the startup template, the device verifies the manifest and the base image digest to ensure that the contract on which the deployment is based has not been tampered with and is consistent with the base image version; when the runtime probe fails, manifest_digest / base_image_digest / base_tag is written to the probe report probe_report, so that the failure can be traced back to the specific contract version and support rollback decisions.

[0056] The automatic detection and deployment guidance steps in this example may include the following sub-steps: Driver version detection: Read the device-side driver version V_d and determine compatibility according to driver_version_rule; Application Binary Interface (ABI) identifier detection: Read the runtime ABI identifier A_d and check its consistency with abi_id; Device node detection: Scan the / dev / directory to obtain D_r and determine if required_devices ⊆ D_r and have the required permissions; Critical library loadability testing: Perform dynamic linking and loading tests on lib_list; Recommended generation: Generate structured deployment guidance information (deploy_advice) based on the differences and generate a minimum privilege startup template (run_template) when available.

[0057] To facilitate automatic selection and determination of the deployment agent, this invention can calculate a compatibility determination value based on fields such as driver version rules, application binary interface (ABI) identifier, device node set, and library summary in the runtime environment manifest, and compare it with a preset threshold. When the compatibility determination result meets the threshold requirement, the base image is determined to be compatible with the current device and allowed to start; when the compatibility determination result does not meet the threshold, image selection, alarm, or rollback logic is triggered.

[0058] This invention constructs a hardware feature fingerprint for the device-side operating environment and a target fingerprint for the base image manifest, aiming to address the issue of ASIC chip construction universality. The hardware feature fingerprint and target fingerprint include at least the following fields: asic_vendor, asic_model, firmware / driver major version, runtime ABI identifier, key device node set, and key library summary set. The deployment agent can sort multiple candidate base image versions based on the similarity between the device-side fingerprint and the manifest fingerprint, select the base_tag corresponding to the base image version with the highest similarity and meeting a preset threshold, and generate a startup template (run_template) accordingly. This further extends automatic detection into an integrated closed loop encompassing automatic selection, automatic guidance, and automatic verification.

[0059] The compatibility determination in this example can also use ternary consistency or fingerprint similarity; when both exist, ternary consistency can be used as a hard constraint first, and then fingerprint similarity can be used to sort and select mirror versions.

[0060] In one embodiment, the startup template generation of the present invention adopts a least-privilege synthesis strategy to solve the problem of intensive manual parameter passing: the deployment agent uses the required devices in the runtime environment manifest as the constraint set and the available capabilities detected on the device side as the candidate set to generate a set of least-privilege runtime parameters that meet the constraints, and solidifies it into an auditable run_template_digest to support auditing and rollback reproduction. To ensure that the above closed loop can still be stably implemented under the differences between ASIC vendors and edge constraints, the deployment agent layers the device-side detection logic. The bottom layer uses the adapter to uniformly output abstract fields such as V_d / A_d / D_r / L_d, and the upper layer uses a fixed set of Manifest fields to perform compatibility judgment and difference output, thereby reusing the same closed-loop process across vendors and avoiding falling into the specific vendor interface details.

[0061] Prioritizing step S6 in this example, hierarchical probing is employed to control cold start overhead. First, device node reachability, driver version readability, and critical library loadability are verified. Then, after passing these verifications, a minimal operator / inference self-check is performed. Timeouts and error codes are set for each probing step to ensure stable probe_report output. The pre-compiled cache and model compilation cache use the digest of base_tag / abi_id / driver_version_rule as the namespace key. Before container startup, consistency is verified by the Manifest. When a driver or ABI change is detected, the old cache is automatically invalidated and the system reverts to a compatible path to prevent the old cache from polluting the new environment.

[0062] As can be seen from the above, compared with the prior art, the present invention has the following innovative points and advantages: 1. Move the drivers and runtime library dependencies required for ASIC operation to the front and solidify them: Complete dependency integration and environment variable configuration during the base image building stage to avoid repeated installation and debugging on the device side, and change the deployment from manual installation and configuration to building and starting based on the base image; 2. Solidify device access capabilities on the container engine side: Inject and solidify ASIC device node mapping, default permission templates, and key environment variables into container engine configuration or runtime hooks, so that the container automatically obtains the ability to access ASIC when it starts up, reducing human configuration errors; 3. Establish post-startup detection and closed-loop control: After the container starts, it automatically performs hardware availability detection and version consistency verification. In case of anomalies, it outputs clear diagnostic information and triggers rollback or alarms, thereby improving deployment success rate and maintainability. 4. Improve consistency and auditability of batch deployments: By versioning the base image, generating and signing the runtime environment manifest, and automatically generating startup templates, we can achieve standardized delivery that is traceable and reversible, thereby reducing operational risks. 5. Reduce cold start latency: Reduce the initialization time of container first startup or first inference by using pre-compiled / pre-linked artifacts and pre-startup warm-up mechanisms, such as operator caching, model compilation caching, symbol resolution caching, etc. 6. Improve edge energy efficiency and thermal stability: Under the condition of meeting the service level agreement (SLA), dynamically adjust the concurrency, batch size, frequency or operator selection based on operational feedback such as temperature / power consumption / bandwidth to reduce the risk of thermal throttling and energy consumption fluctuations. 7. Enhance runtime fault self-healing capabilities: Provide container-level isolation and recovery actions for issues such as driver anomalies, library loading failures, and operator timeouts, reducing the probability of system restarts and improving business continuity.

[0063] The specific embodiments described above are preferred embodiments of the present invention and are not intended to limit the specific scope of the present invention. The scope of the present invention includes, but is not limited to, these specific embodiments. All equivalent changes made in accordance with the present invention are within the protection scope of the present invention.

Claims

1. A containerized deployment system for ASIC edge devices, characterized in that: This includes modules for building a base image, generating and signing a runtime environment manifest, an artifact repository, security and permissions, a deployment agent, container engine adaptation, runtime detection, and monitoring and logging. The base image building module is used to build a base container image adapted to ASIC edge devices; the runtime environment manifest generation and signing module is used to generate a runtime environment manifest, which includes driver version fields required by the base image, library manifest fields, device node requirement fields, probe entry fields, and signature fields, and signs the manifest and the base image; the artifact library module is used to store artifacts, including the base image, runtime environment manifest, and application image. These artifacts provide the deployment agent module with traceable image and manifest sources and support signature verification, version management, and auditing; the security and permissions module is used to control image signature verification, startup template auditing, and privileged capabilities. The module includes: Scope of application deployment and driver loading licensing policies; Deployment agent module: Receives application deployment requests, pulls or locates the base image and runtime environment list, performs self-checks on the runtime environment, conducts compatibility assessments based on the detection results and runtime environment list, and generates deployment guidance information; Container engine adaptation module: Solidifies ASIC edge device access capabilities at the host machine container engine level, including default device node mapping rules, default capability templates, security configuration templates, key environment variable injection rules, and necessary resource limit configurations; Runtime probing module: Performs probing and consistency checks after container startup; Monitoring and logging module: Collects deployment logs, probing results, and container runtime performance metrics.

2. The containerized deployment system for ASIC edge devices according to claim 1, characterized in that: It also includes a hardware capability profiling module, which is used to collect the capabilities of the ASIC on the device side, establish, maintain and output the hardware capability profile of the ASIC on the device side, and is used for the selection of the base image version identifier and the self-adaptation of running parameters.

3. The containerized deployment system for ASIC edge devices according to claim 1, characterized in that: It also includes a pre-compiled cache and warm-up module, which manages and loads ASIC-related pre-compiled artifacts and runtime caches, and performs warm-up when the container starts or the business switches to reduce cold start overhead.

4. The containerized deployment system for ASIC edge devices according to claim 1, characterized in that: It also includes an adaptive optimization module, which collects operational metrics during container operation and dynamically adjusts concurrency, batch size, precision mode, or operator implementation based on a strategy.

5. The containerized deployment system for ASIC edge devices according to claim 1, characterized in that: It also includes a fault isolation and recovery module, which is used to perform container-level isolation and recovery after detecting driver anomalies, memory leaks, operator timeouts, or probe failures.

6. A containerized deployment method for ASIC edge devices, implemented based on the containerized deployment system for ASIC edge devices according to any one of claims 1-5, characterized in that, Includes the following steps: S1: Build the base image: Write the dependencies required for ASIC operation into the base image, form a versioned image identifier base_tag, and publish it to the artifact repository; S2: Runtime environment manifest generation and signing: Generate manifest(base_tag) and sign it, then publish it to the artifact repository; S3: Build application image: Build application image based on versioned image identifier base_tag. The application image contains only business logic, inference model and application configuration and is stored in the artifact repository. S4: Automatic detection and deployment guidance: Read the manifest(base_tag), obtain the required device nodes and environment variable rules, and detect the device-side parameters. Determine whether the device-side parameters have reached the compatibility threshold. If not, generate deployment guidance information, output the differences and recommended actions. If yes, generate the startup template run_template. S5: Solidified Device Access Configuration: The container engine adaptation module applies the device node mapping, environment variables, and permission configuration in the startup template run_template to the container creation parameters, enabling the container to access ASICs; S6: Start the container and perform a probe: The runtime probe module performs a self-check within the container and generates a probe report. If the probe report indicates that the probe has passed, the container enters the business operation and the monitoring and logging module continuously collects relevant metrics. If the probe fails, the deployment agent module will stop, roll back to a compatible version, or report an alarm according to the policy.

7. The containerized deployment method for ASIC edge devices according to claim 6, characterized in that: A hardware feature fingerprint is constructed for the runtime environment, and a target fingerprint is constructed for the base image list of the artifact library. The hardware feature fingerprint and the target fingerprint include asic_vendor, asic_model, firmware / driver main version, runtime ABI identifier, key device node set and key library summary set. In step S4, the deployment agent module sorts multiple candidate base image versions based on the similarity between the hardware feature fingerprint and the target fingerprint, selects the base_tag corresponding to the base image version with the highest similarity and meeting the preset threshold, and generates the run_template accordingly.

8. The containerized deployment method for ASIC edge devices according to claim 6, characterized in that: It also includes the step of establishing a hardware capability profile: collecting the ASIC capabilities on the device side, establishing and maintaining the capability profile of the device-side ASIC, and outputting the capability profile for base image version selection and self-adaptation of running parameters. It also includes cold start preheating and cache loading steps. After the probe_report indicates that the test has passed, before the service starts, the pre-compiled cache and preheating module loads or generates operator cache and model compilation cache based on the base image version identifier base_tag and hardware capability profile.

9. The containerized deployment method for ASIC edge devices according to claim 6, characterized in that: It also includes an adaptive optimization step: During business operation, the adaptive optimization module periodically collects temperature, power consumption, bandwidth and load indicators, and dynamically adjusts the concurrency, batch size, precision mode or operator selection under the constraints of the service level agreement. When a risk of thermal throttling or a trend of power consumption exceeding the limit is detected, a load reduction action is triggered and audit information is recorded.

10. The containerized deployment method for ASIC edge devices according to claim 6, characterized in that: It also includes fault isolation and recovery steps: when the monitoring and logging module or runtime probe detects an anomaly during runtime, the fault isolation and recovery module performs controlled recovery on the target container, or triggers a base image version switch or rollback.

Images