An industrial edge-oriented lightweight AI model dynamic loading system

Abstract

The application belongs to the technical field of artificial intelligence and edge computing, and particularly relates to a lightweight AI model dynamic loading system for industrial edges, aiming to solve the problems of rigid model deployment, high inference delay and update lag caused by limited resources and variable working conditions of industrial edge devices. The system decouples the AI model into functional fragments with attribute labels by a model fragment manager, generates a configuration strategy by a context-aware adaptation module in combination with device status and task requirements, determines the optimal fragment combination by a scheduling engine, realizes millisecond-level hot plug by a symbolic analysis and double buffering mechanism through an incremental loading executor, and optimizes key parameters online by a local fine-tuning unit using a small amount of data. The scheme supports on-demand loading, dynamic adaptation and non-intrusive upgrade, significantly reduces memory occupation and energy consumption, improves response speed and model generalization ability, and realizes efficient autonomy of edge intelligent services.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and edge computing technology, specifically relating to a lightweight AI model dynamic loading system for industrial edge computing. Background Technology

[0002] The deep integration of artificial intelligence (AI) and the Internet of Things (IoT) technologies has driven the development of industrial intelligence. Edge computing, as a bridge connecting the physical and digital worlds, plays a crucial role in data preprocessing, real-time decision-making, and local autonomy. In applications such as intelligent manufacturing, predictive maintenance, and industrial visual inspection, higher demands are placed on the real-time performance, accuracy, and energy efficiency of model inference. Traditional centralized AI computing models upload raw data to the cloud for processing, resulting in high network latency, high bandwidth consumption, and high privacy risks, making it difficult to meet the low latency and high reliability requirements of industrial sites. Therefore, deploying AI models on industrial edge devices close to the data source has become an inevitable trend, enabling on-site data processing and rapid response.

[0003] The application of lightweight AI models offers feasibility for resource-constrained edge environments. Through techniques such as model compression, pruning, and quantization, the number of model parameters and computational overhead can be significantly reduced while maintaining inference accuracy. However, industrial production environments are characterized by variable operating conditions, dynamic task switching, and strong equipment heterogeneity. A single, fixed lightweight model is difficult to adapt to diverse business needs. For example, on the same production line, different models with different functions, such as defect detection, dimensional measurement, or character recognition, may need to be switched depending on the product type. At the same time, different models of edge hardware vary significantly in terms of computing power, memory, and accelerator support, requiring models to have good portability and adaptability.

[0004] While existing technologies have enabled the static deployment of lightweight models at the edge, they still have significant shortcomings when facing dynamic task scheduling in complex industrial scenarios: They lack a unified management mechanism for multiple model resources, the model loading process relies on manual intervention and is inefficient; model update and version control capabilities are weak, failing to support remote incremental upgrades and rollbacks; they fail to effectively integrate edge gateway scheduling strategies to achieve on-demand loading and unloading, resulting in persistently high memory consumption; their ability to perceive underlying hardware characteristics is insufficient, making it difficult to automatically match the optimal model with the target platform; furthermore, the lack of integrity verification and fault recovery mechanisms during model switching limits system reliability. These problems are further amplified in large-scale distributed industrial edge environments, severely impacting the continuity of intelligent services and operational efficiency. Therefore, there is an urgent need for a lightweight AI model dynamic loading system for industrial edges to resolve the contradiction between multi-scenario adaptation, efficient scheduling, and reliable operation. Summary of the Invention

[0005] The purpose of this invention is to provide a lightweight AI model dynamic loading system for industrial edge computing, addressing the systemic technical challenges in current industrial edge computing environments, such as rigid AI model deployment, high inference latency, low energy efficiency, and delayed model updates, caused by strong device heterogeneity, limited resources, and variable operating conditions. In existing technologies, industrial sites commonly deploy AI models using static or fully pre-built methods, which struggle to adapt to the diverse needs of different production lines and processes for intelligent functions like detection, prediction, and control. Simultaneously, edge nodes generally face hardware constraints such as small memory capacity, limited storage bandwidth, and uneven computing power distribution, rendering large models unusable while small models fail to meet accuracy requirements. Furthermore, when processes change or failure modes evolve, traditional solutions rely on retraining and redeploying new models from a central cloud, resulting in long deployment cycles, strong network dependence, and weak real-time response capabilities, severely restricting the adaptability and intelligence level of intelligent manufacturing systems.

[0006] The technical solution of this invention is to construct a distributed collaborative system consisting of an edge model scheduling engine, a model shard manager, a context-aware adaptation module, an incremental loading executor, and a localized fine-tuning unit. This system enables on-demand dynamic loading, fine-grained resource matching, and online lightweight optimization of AI models in industrial edge environments. The system is deployed on edge computing servers and embedded devices on the terminal side within the factory's local area network. It interconnects with sensors, controllers, and host computer systems via industrial Ethernet or time-sensitive networks, forming an intelligent service closed loop of edge-edge collaboration. The model shard manager pre-decouples the trained complete AI model into multiple logically independent model shards according to functional semantics and computational characteristics, including input encoding shards, feature extraction shards, attention mechanism shards, and classification head or regression head shards. Each shard is labeled with its computational complexity, peak memory usage, energy consumption coefficient, and applicable scenario label. The context-aware adaptation module continuously collects runtime context information such as the current task type, device operating status, available memory, temperature and power consumption limits, and network connection quality, and generates a model configuration strategy based on a preset multi-dimensional decision matrix. The model scheduling engine receives task instructions from the production execution system, parses the AI ​​function categories and performance indicators required by the task, and determines the optimal combination of target model structures by combining the configuration strategy output by the context-aware adaptation module. The incremental loading executor, based on the loading plan generated by the scheduling engine, pulls only the required model fragments from the local cache or edge server, and dynamically injects them into the runtime inference graph through zero-copy mapping and virtual address space splicing technology, achieving millisecond-level hot-swapping of functional modules. After the model is loaded, the local fine-tuning unit uses a small amount of real-time sampled data under the current operating conditions to locally fine-tune the weights of key layers using a differential learning strategy combining gradient masking and parameter freezing, without affecting the main inference process, thereby improving the model's generalization ability in specific scenarios.

[0007] Furthermore, when segmenting the AI ​​model, the model sharding manager employs an automatic partitioning algorithm based on computational graph topology analysis to identify independently executable subgraph structures with clearly defined input / output interfaces. A standardized serialization / deserialization interface layer is then inserted to ensure data flow consistency across shards after physical separation. The sharded model units are packaged in a containerized image format, containing executable code, weight parameters, a dependency library list, and verification fingerprints, supporting portable deployment across different architecture platforms. The model shards are stored using a tiered caching mechanism. High-frequency, basic shards reside in the non-volatile memory of edge devices, while low-frequency, dedicated shards are cached on high-speed solid-state disk arrays on edge servers. Automatic preloading and eviction are achieved through a two-factor strategy combining LRU and access prediction.

[0008] Furthermore, the multi-dimensional decision matrix in the context-aware adaptation module is generated through reinforcement learning during the offline training phase. Its state space covers 12 context dimensions, including device model, load level, ambient temperature and humidity, and battery level. The action space corresponds to different model segmentation and combination strategies, and the reward function comprehensively considers three core indicators: inference accuracy, response latency, and energy consumption. During online operation, the module inputs the real-time collected context vectors into the converged policy network and directly outputs the recommended model configuration scheme, with a decision latency of less than 50 milliseconds. When a sudden change occurs in the context, such as when the device switches to energy-saving mode, the module triggers a re-evaluation mechanism, notifying the scheduling engine to start the reconfiguration process.

[0009] Furthermore, the incremental loading executor introduces runtime symbol resolution and dynamic linking mechanisms during the model fragment loading process, maintaining a global symbol table to record the function entry addresses and tensor descriptors of all loaded fragments. After a new fragment is loaded, the executor automatically resolves its external references and binds them to existing entries in the symbol table, completing the cross-fragment data channel connection. For complex tasks involving control flow branches, the executor supports conditional loading, meaning that the corresponding fragment is activated only when the input data meets preset trigger conditions, avoiding invalid computation. The loading process employs a double-buffering mechanism: the foreground buffer maintains the current inference task, while the background buffer decompresses and links the new fragments. Seamless transition is achieved through atomic pointer swapping during switching, ensuring service continuity.

[0010] Furthermore, the localized fine-tuning unit adopts an event-driven micro-learning paradigm, setting three trigger modes: periodic triggering, drift detection triggering, and manual instruction triggering. The drift detection module uses a sliding window to statistically analyze the KL divergence of the input data distribution. When it exceeds a set threshold of 8.3, it is determined that concept drift has occurred, and the fine-tuning process is initiated. The fine-tuning process is limited to three consecutive batches, with each batch using no more than 64 of the latest samples. The optimizer step size is limited to 0.1 times the initial value, and only the parameters of the last two network layers are updated; the gradients of the remaining layers are explicitly set to zero. After fine-tuning, the system automatically evaluates the performance gain; if the accuracy improvement is less than 1.2%, it rolls back to the original weights.

[0011] Furthermore, the system supports multi-version coexistence and a canary release mechanism. Different optimized versions of the same model shard can coexist in the shard repository of the edge node, labeled with version number, target chip architecture, and energy efficiency rating. The scheduling engine can specify the precise version according to the target task, or automatically select the optimal compatible version based on the current hardware platform. New version shards can be updated via silent background download. After the integrity verification is passed, it enters a waiting-to-activate state, and the timing of entering the main process is determined by operation and maintenance instructions or policy rules, achieving seamless upgrades.

[0012] Furthermore, the system integrates a secure and trusted execution environment, enabling TrustZone on ARM architecture devices and SGX enclave on x86 platforms to protect the intellectual property and sensitive parameters of model fragments. All model fragments must be digitally signed and verified before loading, and their origin must belong to an authorized model registry. Intermediate parameters generated during fine-tuning are also processed in the trusted environment to prevent data leakage.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are as follows: This system fundamentally changes the static "one-time deployment, long-term immutability" model of industrial edge AI models, establishing a new paradigm of dynamic loading based on context-aware and segmented architecture. It achieves precise matching between on-demand supply of intelligent functions and resource consumption. By decoupling the complete model into independently schedulable semantic segments, it significantly reduces the memory impact of single loading, enabling complex visual inspection models that previously could not run on devices with 4GB of memory to achieve real-time inference through staged loading. Peak memory usage is reduced to 41% of its original level, and startup latency is compressed to within 230 milliseconds. The context-aware adaptation module introduces a multi-dimensional decision matrix and reinforcement learning strategies, allowing the system to autonomously select the model configuration most suitable for the current operating conditions. In high-temperature derating scenarios, it automatically switches to a low-power segment combination, reducing overall energy consumption by 37% while maintaining 95% efficiency. The above detection accuracy is achieved; the incremental loading actuator uses symbolic resolution and double buffer switching technology to ensure service continuity during function expansion, supporting model capability upgrades without downtime, and is suitable for continuous production processes; the local fine-tuning unit combines drift detection and constrained optimization strategies to achieve online self-evolution of the model while preventing catastrophic forgetting, and can complete adaptive adjustments within 10 minutes when facing new defect samples, with a false detection rate reduced by 58% compared to the fixed model; the system supports multi-version coexistence and canary release, providing enterprises with a safe and controllable model iteration path, reducing the risk of new algorithm deployment by 90%; the overall solution eliminates the dependence on frequent interactions with the central cloud, with 98% of model management operations completed locally at the edge, significantly reducing industrial network bandwidth consumption and improving the autonomy, robustness, and response speed of the intelligent manufacturing system. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall technical solution architecture proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of dynamic loading based on context awareness and fragmentation architecture in this invention. Detailed Implementation

[0015] Please refer to Figure 1 and Figure 2 This invention provides a lightweight AI model dynamic loading system for industrial applications, comprising: The edge model scheduling engine is used to receive task instructions from the production execution system, parse the required AI function categories and performance indicators, and generate scheduling decisions for the target model structure combination by combining runtime context information. The model shard manager is used to decouple a complete AI model into multiple logically independent model shards according to its functional semantics and computational characteristics, and to label, encapsulate, and hierarchically store and manage each shard. The context-aware adaptation module is used to continuously collect runtime context information such as current task type, device operating status, available memory, temperature and power consumption limits, and network connection quality, and output model configuration strategies based on a preset multi-dimensional decision matrix. The incremental loading executor is used to pull the required model fragments from the local cache or edge server only according to the loading plan generated by the scheduling engine, and dynamically inject them into the runtime inference graph through zero-copy mapping and virtual address space splicing technology to achieve millisecond-level hot-swapping of functional modules. The local fine-tuning unit is used to fine-tune the weights of key layers locally after the model is loaded, using a small amount of real-time sampled data under the current working conditions, without affecting the main inference process, and without affecting the differential learning strategy that combines gradient masking and parameter freezing, thereby improving the model's generalization ability in specific scenarios.

[0016] A lightweight AI model dynamic loading system for industrial edge computing aims to address systemic technical challenges arising from the heterogeneity, resource constraints, and variable operating conditions of industrial edge devices. These challenges include rigid AI model deployment, high inference latency, low energy efficiency, and delayed model updates. Existing technologies typically deploy AI models using static hardening or full pre-installation, which struggles to adapt to the diverse needs of different production lines and processes for intelligent functions such as detection, prediction, and control. Furthermore, edge nodes face hardware constraints such as limited memory capacity, storage bandwidth, and uneven computing power distribution, making it impossible to run large models while small models fail to meet accuracy requirements. When processes change or failure modes evolve, traditional solutions rely on retraining and redeploying the entire model from a central cloud, resulting in long deployment cycles, strong network dependence, and weak real-time response capabilities, severely restricting the adaptability and intelligence level of intelligent manufacturing systems.

[0017] This invention constructs a distributed collaborative system composed of the aforementioned five major units, deployed on edge computing servers and embedded devices on the terminal side within a factory local area network. It interconnects with sensors, controllers, and host computer systems via industrial Ethernet or time-sensitive networking, forming a closed-loop intelligent service for edge-end collaboration. The system's workflow begins with task triggering, followed by a closed-loop processing mechanism encompassing context-aware analysis, model scheduling decision-making, slice loading and execution, and online fine-tuning optimization. The entire process enables on-demand dynamic loading of AI models, fine-grained resource matching, and online lightweight optimization in industrial edge environments, significantly improving the flexibility, energy efficiency, and response speed of the edge intelligent system.

[0018] The edge model scheduling engine serves as the core control hub of the system, responsible for coordinating task requirements, context states, and available model resources to generate optimal model loading instructions. This engine is a daemon process residing in the user space of the edge computing node's operating system, possessing multi-threaded concurrent processing capabilities and able to simultaneously respond to task requests from multiple production workstations. Internally, the engine maintains a task queue buffer. All tasks awaiting processing are first serialized into message objects of a unified format and temporarily stored in this buffer. Each message object includes a task identifier, target function type (e.g., defect detection, vibration prediction, energy consumption modeling), expected inference frame rate (frames / second), maximum allowable latency (milliseconds), and priority level. The engine periodically scans the task queue using a polling mechanism and dynamically adjusts the processing order based on task priority and system load. When a new task is detected, the engine immediately initiates the scheduling decision-making process.

[0019] The first step in scheduling decisions is task parsing. The engine uses its built-in function classification dictionary to semantically match the function description fields in the task instructions, mapping them to a predefined set of function categories. For example, "weld surface crack recognition" is categorized as "visual defect detection," and "motor bearing temperature trend extrapolation" is categorized as "time series prediction." Each function category is associated with a set of candidate target model structure templates. These templates are determined by historical deployment experience and offline simulation testing, recording the accuracy, average inference time, and peak memory usage of each structure on a standard test set. Based on the performance metrics specified by the task, the engine selects a subset of candidate templates that meet the minimum frame rate and latency requirements.

[0020] The second step is context fusion. The engine initiates a synchronous query to the context-aware adaptation module to obtain the comprehensive context vector at the current moment. This vector consists of 12 dimensions: device model code, CPU utilization, GPU availability flag, remaining memory capacity (in MB), non-volatile memory read / write bandwidth (in GB / s), current power supply mode (mains / battery), battery percentage, ambient temperature (in °C), cooling fan speed (in RPM), network connection status (wired / wireless / Wi-Fi 6), uplink latency (in ms), and a secure and trusted execution environment ready flag. The engine cross-compares this context vector with candidate model structure templates, eliminating options that are not ready for execution due to insufficient resources or environmental limitations. For example, if the remaining memory capacity is less than 512MB, all model structures with peak memory usage exceeding 800MB are eliminated; if the device is on battery power and the battery level is below 20%, low-power optimized versions are prioritized.

[0021] The third step is strategy generation. Based on the filtered list of feasible model structures, the engine selects the final target model structure combination, taking into account task priority and overall system load. The selection algorithm uses a weighted scoring method, and the scoring function is defined as follows: in, The accuracy of the model on the standard test set. To calculate the average time for a single inference attempt, The estimated energy consumed to complete one full reasoning operation (in joules), with the maximum and minimum values ​​being the extreme values ​​of the corresponding indicators in the candidate set; , , The weighting coefficients are configurable; the default value is set to [value]. =0.5、 , =0.2, which can be remotely updated based on the factory operation strategy. The engine calculates the comprehensive score for each candidate structure and selects the highest-scoring combination as the target model structure combination. This combination is output in the form of a structure description file, which explicitly lists the names of all model fragments to be loaded, the expected loading order, version number constraints, and resource reservation suggestions. This file is submitted to the incremental loading executor to start the subsequent loading process.

[0022] The Model Shard Manager aims to achieve fine-grained decoupling and efficient organization of AI models, supporting an infrastructure for on-demand loading. As a standalone service module, it is integrated into the operating system image of edge devices, supporting native operation on both ARMv8 and x86-64 dual-architecture platforms. Its core function is to automatically partition and standardize the pre-trained complete AI model, ensuring that each shard has the ability to be loaded and executed independently.

[0023] The model segmentation process begins with computation graph topology analysis. The manager loads the original model file (supporting three mainstream formats: ONNX, TensorFlow SavedModel, and PyTorch TorchScript), parses its internal computation graph structure, and constructs a directed acyclic graph (DAG). Nodes represent basic operators (such as convolution, batch normalization, and activation functions), and edges represent tensor data flows. The system traverses this graph, identifying connected subgraph regions with a single input port and a single output port, and evaluates their functional cohesion and interface clarity. Specific criteria include: the connection density between operators within the subgraph is higher than a threshold of 0.85; no more than two tensor pins are exposed externally; and the subgraph's execution time accounts for between 8% and 35% of the total graph execution time. Subgraphs meeting these conditions are considered potential segmentable units.

[0024] Furthermore, the system employs a greedy merging strategy to optimize and reorganize the initial candidate subgraphs. For adjacent and functionally related subgraphs (such as feature extraction blocks composed of multiple convolutional layers), if they still meet the maximum size limit (number of operation cycles ≤ 1.2 × 10^6) after merging, they are merged to reduce communication overhead in subsequent loading processes. The resulting stable subgraphs are the basic units for model partitioning. Typical partitioning results include: input encoding partitions, responsible for preprocessing and format conversion of raw sensor data; backbone feature extraction partitions, performing forward propagation of the backbone network; attention mechanism partitions, implementing channel or spatial attention weighting; and classification head or regression head partitions, outputting the final task results.

[0025] Each model slice must maintain data consistency after physical separation. To this end, the system automatically inserts standardized serialization / deserialization interface layers at the input and output boundaries of each slice. The input interface layer receives the flattened byte stream output from the upstream slice and reconstructs it into a multidimensional array according to preset tensor descriptors (including dimension shape, data type, and layout format). The output interface layer serializes the output tensor of its own slice into a byte stream according to the same rules for downstream consumption. The interface layer uses Google FlatBuffers as the serialization protocol because it offers advantages such as random access without parsing, low memory consumption, and good cross-language compatibility.

[0026] After the fragments are packaged, they are stored in a containerized image format. The image adopts the lightweight OCI (Open Container Initiative) specification and contains four components: the executable code segment, compiled into a native binary file optimized for the target architecture; the weight parameter segment, saved in a sparse compression format, supporting INT8 quantization and unstructured pruning; the dependency library list, declaring the required basic mathematical libraries (such as BLAS, Eigen), runtime environments (such as OpenVINO, TensorRT Lite) and their version ranges; and the verification fingerprint segment, a content hash value generated using the SHA-3-512 algorithm for integrity verification. Each image file is attached with metadata tags, including fragment name, function tag, computational complexity (FLOPs), peak memory usage (MB), energy efficiency (Joules / FLOPs), applicable chip architecture (ARM / x86 / GPU), security level (public / restricted / encrypted), and release date.

[0027] Storage management employs a three-tiered caching system. The first tier is a high-speed cache, located in the onboard non-volatile memory (NVM) of the edge devices, with a capacity of 2GB, used to reside in frequently used basic shards (such as general input encoding and basic feature extraction). The second tier is a near-line cache, deployed on the NVMe SSD array of the edge server in the same workshop, with a capacity of 200GB, storing low-frequency dedicated shards (such as fault diagnosis heads for specific motor models). The third tier is a remote repository, located in the object storage cluster of the factory data center, storing all historical versions and infrequently used shards. The cache scheduling strategy combines LRU (Least Recently Used) and access prediction dual-factor mechanisms. The access prediction model is built on an LSTM network, taking the task log sequence of each workstation over the past 7 days as input and outputting the expected call probability of each shard in the next 24 hours. The manager executes a preload task once daily at midnight, pushing shards with a predicted probability higher than 0.65 from the remote repository to the near-line cache, ensuring that high-potential shards are available nearby. The eviction policy is set as follows: when the cache utilization rate exceeds 90%, the cleanup process is triggered, and shards that have not been accessed for 7 consecutive days and whose predicted call probability in the next 7 days is less than 0.1 are removed first.

[0028] The context-aware adaptation module aims to achieve comprehensive awareness and intelligent response to the runtime environment, providing a basis for model configuration decisions. This module consists of a set of dedicated sensor interface drivers, data aggregation middleware, and a policy inference engine, running in a real-time-guaranteed Linux PREEMPT_RT kernel environment to ensure deterministic latency in context acquisition and processing.

[0029] The module's data acquisition layer accesses field information through multiple physical interfaces. Device model codes are obtained by reading the unique device identifier from the motherboard's EEPROM; CPU and GPU utilization are periodically sampled by the kernel performance counter module (every 100 milliseconds); memory usage is obtained through the / proc / meminfo file system interface; non-volatile memory bandwidth is periodically measured (every 5 minutes) using a dedicated stress test program; power supply mode and battery level are determined by the power management integrated circuit via I / O. 2 The C-bus reports the ambient temperature; the ambient temperature is collected by a digital temperature sensor integrated into the device housing (DS18B20 model); the cooling fan speed is obtained by decoding the PWM feedback signal; the network status and link latency are monitored by the network management daemon through active probe packets (ICMP / TCP). All raw data, after being cleaned, filtered, and normalized, is pooled into a shared memory pool for real-time access by the policy inference engine.

[0030] The core of the strategy inference engine is a multi-dimensional decision matrix generated during the offline training phase. This matrix is ​​constructed before factory deployment, relying on a large-scale simulation platform to simulate the performance of various edge devices under different operating conditions. The state space is defined as a hypercube composed of 12 continuous variables, with each dimension normalized and mapped to the [0,1] interval. The action space corresponds to 256 preset model segmentation combination strategies, each strategy specifying the selection preference for each segment under a specific functional category (e.g., "high-precision feature extraction," "low-latency attention," "energy-saving classification head"). The reward function is designed as a weighted sum of three objectives: in, The measured accuracy (normalized to [0,1]) This represents the actual inference delay (milliseconds). Delaying the target required by the task. This refers to the actual energy consumed (joules). To limit the allowable energy consumption, This indicates ReLU truncation (negative values ​​are set to zero). =0.4、 =0.35、 =0.25 is the weight coefficient. The reinforcement learning agent (using the PPO algorithm) explores the long-term cumulative reward of different state-action pairs in the simulation environment, and finally converges to obtain a policy network whose input is the current context vector and whose output is the recommended action number (i.e., the model configuration scheme).

[0031] During online runtime, the context-aware adaptation module continuously updates the context vector at a frequency of 20 Hz and inputs it into the deployed policy network. The inference process is executed on a dedicated neural acceleration coprocessor, with latency controlled within 45 milliseconds. The output recommended configuration scheme is encapsulated in JSON format, including the preferred sharding combination, alternative degradation schemes, and resource adjustment suggestions. This scheme is submitted to the edge model scheduling engine to participate in the final scheduling decision. When a sudden event occurs in the context (such as a CPU temperature spike above 85°C, a network interruption lasting more than 10 seconds, or a device switching to manual mode), the module immediately triggers a re-evaluation interrupt signal, forcing the scheduling engine to pause the current task pipeline and re-initiate context queries and scheduling decisions, ensuring that the model configuration always matches the runtime environment.

[0032] The incremental loading executor aims to achieve dynamic loading and runtime integration of model fragments, ensuring service continuity and loading efficiency. As part of the inference runtime environment, this executor is deeply integrated into the runtime libraries of mainstream compiled inference frameworks such as TVM and OpenVINO, directly manipulating the underlying memory management and symbolic linking mechanisms.

[0033] After the loading process begins, the executor first parses the target model structure description file provided by the scheduling engine to extract the list of model shards to be loaded. For each shard in the list, the executor checks whether a valid image copy of that shard exists in the local cache. The criteria for judgment include: file existence, SHA-3-512 checksum matching, version number compliance with constraints, and valid security signature. If any condition is not met, a remote retrieval request is initiated to the model shard manager, specifying the shard name and version number, to download the latest compliant image from the near-line cache or remote repository. The download process uses a chunked transmission protocol, performing an incremental check every 64KB of data received. If an error is found, the corresponding block is immediately retransmitted to ensure data integrity.

[0034] Once the fragmented image is ready, the executor initiates the loading preparation phase. During this phase, the system allocates an independent virtual address space region for each new fragment, with the starting address aligned to a 4KB page boundary. The mmap system call is used to map the weight parameter segment in the image file to read-only shared memory, and the executable code segment to executable read-only memory, achieving zero-copy loading and avoiding the additional memory copying overhead of malloc-memcpy in traditional methods. After mapping, the executor calls the dlopen-like dynamic linking interface to load the executable code segment and obtain its exported symbol table, which includes the input processing function entry point, the main inference function pointer, the output serialization function address, and references to internal state variables.

[0035] The key steps are runtime symbol resolution and dynamic linking. The executor maintains a global symbol table, recording the function entry addresses and tensor descriptors of all loaded slices. After a new slice is loaded, the executor automatically resolves its external reference symbols (such as "feat_in", "att_out") and searches for a match in the global symbol table. If the corresponding upstream slice's output symbol is found, a data channel connection is established, and the DMA controller is configured to enable direct transfer of tensor memory; if not found, an unresolved reference exception is thrown, the loading process is terminated, and an error log is reported. For control flow branch tasks (such as conditional defect re-inspection), the executor supports a predicate-driven conditional loading mechanism. The system presets a set of trigger condition expressions (such as "input brightness < 50 lux", "vibration amplitude > 2g"), and only when the real-time input data satisfies the expression is the loading and linking of the corresponding auxiliary slice activated, avoiding unnecessary waste of computational resources.

[0036] The loading process employs a double-buffering mechanism to ensure service continuity. The system maintains two sets of parallel inference context buffers: a front-end buffer holds the currently executing inference task, and a back-end buffer is used for decompressing, mapping, and linking new fragments. Once all background preparations are complete, the executor performs an atomic switch during the interval between the next inference cycle. The switch operation is achieved by swapping the root node pointers of the two buffers, which are referenced by all downstream consumers. The swap instruction uses CPU atomic swap instructions (such as x86's XCHG or ARM's LDAXR / STLXR) to ensure the indivisibility of the operation and prevent inference failure due to intermediate states. The entire switch process takes no more than 1.2 milliseconds, and the inference frame rate fluctuation observed by the user is less than 0.5%, achieving a truly seamless transition.

[0037] After loading is complete, the executor registers lifecycle hook functions for the new shard, including initialization callbacks (for weight rescaling during loading), pre-inference processing (input data validation), post-inference processing (output confidence correction), and destruction callbacks (releasing dedicated hardware resources). All hook functions are executed serially in the main inference thread to ensure timing correctness. The system also establishes a runtime health monitoring agent to continuously track memory leak signs, abnormal exit counts, and inference result consistency for each shard. Upon detecting anomalies, problematic shards are immediately isolated, and an attempt is made to roll back to the previous stable configuration.

[0038] The localized fine-tuning unit aims to achieve adaptive optimization of the model under specific operating conditions, improving its long-term accuracy and robustness. This unit runs as a background daemon, employing an event-driven architecture, activating the micro-learning process only under specific conditions to avoid interfering with the main inference task.

[0039] The fine-tuning triggering mechanism is set to three independent modes: periodic triggering, drift detection triggering, and manual command triggering. The periodic triggering mode sets a fixed time interval (default is once every 6 hours), suitable for periodic calibration needs in a steady-state production environment. Drift detection triggering is the core automated mechanism, implemented as follows: The unit maintains a sliding window buffer with a length of 512 samples, continuously collecting raw input data from the main inference path. For each new sample, the Kullback-Leibler divergence between it and the baseline distribution is calculated. The baseline distribution is constructed from 1024 representative samples collected during the initial deployment phase, modeled using kernel density estimation. The KL divergence calculation formula is: in, The empirical distribution of the data within the current sliding window, i.e., the actual observed data in each discrete interval. It is a frequency or probability estimate. It changes dynamically and is updated as new samples arrive. The reference distribution, constructed from 1024 representative samples collected during the initial system deployment, represents the data distribution under "normal" or "steady-state" conditions. It remains unchanged throughout the entire operation. To sum over all discrete intervals, The empirical distribution of the data within the current sliding window. Using the baseline distribution, the summation covers all discretized intervals. The system is updated every 32 new samples received. And recalculate When three consecutive measurements exceed the set threshold of 8.3, concept drift is identified, and a fine-tuning process is initiated. Manual commands are triggered by operations personnel through the security management interface for emergency calibration or verification of new algorithms.

[0040] The fine-tuning process itself is strictly constrained to prevent catastrophic forgetting and resource overload. The process is limited to three consecutive batches, each using no more than 64 newly acquired labeled samples (labels are verified by a quality inspector or generated through semi-supervised learning). The optimizer uses the AdamW algorithm, but the learning rate is forcibly limited to 0.1 times the initial value from the pre-training phase (e.g., if originally 0.001, then set to 0.0001 here), while the momentum parameter remains unchanged. The gradient update strategy employs selective backpropagation: when calculating the gradient of the loss function, only the gradient flow is retained for the last two network layers; the gradients of all other layers are explicitly zeroed during backpropagation. Specifically, a gradient hook function is registered in the automatic differentiation engine to intercept intermediate gradient tensors and perform masking operations. The weight update scope is limited to the fully connected layers of the classification or regression head and the last-level feature normalization layer, ensuring that the general feature extraction capability of the underlying layers is not affected.

[0041] After fine-tuning, the system automatically performs a performance evaluation. An independent validation set (containing 256 up-to-date samples) is used to compare the accuracy changes of the model before and after fine-tuning. If the accuracy improvement is less than 1.2 percentage points, the optimization is deemed ineffective, and the system initiates a rollback mechanism: restoring the original weight file from persistent storage and re-injecting it into the corresponding shards via an incremental loading executor, replacing the current fine-tuned version. The rollback operation also employs double-buffering switching technology to ensure uninterrupted service. Successfully fine-tuned model versions will be marked as "Live Optimized Version," and their metadata will include fields such as "Fine-tuning Timestamp," "Sample Source Batch," and "Gain Amplitude," and will be uploaded to the edge server for record-keeping and future version management reference.

[0042] The system supports multi-version coexistence and canary release mechanisms, aiming to realize safe and controllable model iteration and risk isolation. Different optimized versions of the same model shard can coexist in the shard repository of edge nodes, each marked with a unique version number (following the Semantic Versioning 2.0.0 specification), a target chip architecture (such as arm64-v8a, x86_64-skylake), and an energy efficiency rating (divided into five levels from A++ to D). The shard repository adopts a content-addressable storage structure, where each shard is uniquely identified by its SHA-3-512 fingerprint, allowing multiple aliases to point to the same content entity to save storage space.

[0043] When generating the loading plan, the scheduling engine can precisely specify the required shard version according to the target task (such as "resnet_block_v3.2.1"), or set fuzzy matching rules (such as ">=v3.0,<v4.0"). When multiple compatible versions exist, the engine automatically selects and loads the version with the optimal energy efficiency rating based on the current hardware platform. The update process of new version shards is completely silent: the model shard manager receives the incremental update package from the model registry in the background, which contains the new shard image and metadata description. The system first verifies the digital signature of the update package (based on the RSA-4096 algorithm), and writes it to the staging area after confirming that the source is legitimate. After the integrity check is passed, the new version enters the "to be activated" state and waits for the switching instruction.

[0044] The version cutover strategy is determined by the operation and maintenance policy rules, and supports three modes: immediate switching, which automatically replaces the old version after the new version is ready; canary switching, which is gradually promoted according to the modulo remainder rule of the station number (for example, stations 0-9 are switched on the first day, and stations 10-19 on the next day); conditional switching, which is executed only when specific conditions are met (such as night shift hours, device idle rate >80%). The switching operation is completed by the scheduling engine coordinating the incremental loading executor, and smooth transition is realized through the double buffering mechanism. The old version shard enters the to-be-recycled state after there are no references, and is automatically cleared by the garbage collector 7 days later to release storage space.

[0045] The system integrates a secure and trusted execution environment to protect model intellectual property and sensitive parameters from malicious attacks. On ARM architecture devices, TrustZone technology is enabled to create isolated execution environments between a secure world and a normal world. All operations involving model fragment loading, fine-tuning calculations, and parameter storage are performed within the secure world. The TrustZone Monitor Mode controls context switching between the two worlds, ensuring that the normal world cannot directly access the secure memory region. On x86 platforms, Intel SGX enclave technology is enabled to encapsulate critical code segments and data within a protected enclave, utilizing a Memory Encryption Engine (MEE) to achieve confidentiality and integrity protection at the physical memory level.

[0046] All model shards must be digitally signed before loading. The signing key is issued by an enterprise-level certificate authority, and the private key is stored in the Hardware Security Module (HSM), prohibited from being exported. The verification process is performed within a trusted execution environment, using a Public Key Infrastructure (PKI) system to verify the validity of the shard image's ECDSA-P384 signature. Source checks ensure that shards belong to an authorized list of model registry URIs, preventing unauthorized injection by third parties. Intermediate parameters, gradient tensors, and temporary weight snapshots generated during fine-tuning are all generated and processed within the trusted environment, prohibiting any form of external memory dumping or debugging interface access. Only the final, confirmed, and valid optimized version is allowed to be persisted to non-volatile storage and tagged with a trusted timestamp.

[0047] In summary, this invention achieves dynamic, refined, and intelligent management of industrial edge AI models by constructing a collaborative system consisting of an edge model scheduling engine, a model fragment manager, a context-aware adaptation module, an incremental loading executor, and a localized fine-tuning unit. The system fundamentally changes the traditional static deployment model, establishing a new loading paradigm based on context-aware and fragmented architecture. Experimental data shows that on an embedded industrial computer with a 4GB memory limit, the ResNet-152 visual detection model, which was previously unable to run, successfully achieved real-time inference through a phased loading method. Peak memory usage decreased from 1980MB to 812MB, a reduction of 41% of the original, and startup latency was compressed to 228 milliseconds. Under high-temperature derating operation scenarios, the system automatically switches to a low-power fragment combination, reducing overall power consumption by 37% while maintaining a detection accuracy of 95.2%. Faced with new defect samples, the localized fine-tuning unit completes adaptive adjustments within 9.7 minutes, reducing the false detection rate by 58% compared to the fixed model. 98% of model management operations are completed locally at the edge, significantly reducing network dependence on the central cloud and significantly improving the autonomy, robustness and responsiveness of the intelligent manufacturing system.

Claims

1. A lightweight AI model dynamic loading system for industrial applications, characterized in that, include: The model shard manager is used to decouple a complete AI model into multiple logically independent model shards according to its functional semantics and computational characteristics, and to label, encapsulate, and hierarchically store and manage each shard. The context-aware adaptation module is used to continuously collect runtime context information such as current task type, device operating status, available memory, temperature and power consumption limits, and network connection quality, and output model configuration strategies based on a preset multi-dimensional decision matrix. The edge model scheduling engine is used to receive task instructions from the production execution system, parse the required AI function categories and performance indicators, and determine the optimal combination of target model structures in conjunction with the model configuration strategy. An incremental loading executor is used to pull the required model fragments from the local cache or edge server only according to the target model structure combination, and dynamically inject them into the runtime inference graph through zero-copy mapping and virtual address space splicing technology. The local fine-tuning unit is used to fine-tune the weights of key layers locally using a small amount of real-time sampled data under the current working conditions after the model is loaded, without affecting the main inference process, by adopting a differential learning strategy that combines gradient masking and parameter freezing. The model fragment manager includes: The computation graph topology analysis subunit is used to identify independently executable subgraph structures with clearly defined input and output interfaces in the computation graph of a complete AI model. Interface layer insertion subunits are used to automatically insert standardized serialization / deserialization interface layers at the input and output boundaries of the identified subgraph structure; The containerization encapsulation subunit is used to encapsulate each model fragment in a containerized image format. The containerized image includes executable code, weight parameters, a list of dependent libraries, and a verification fingerprint. The model fragment manager also includes: The hierarchical cache management subunit is used to keep the frequently used basic shards residing in the non-volatile memory of the edge device, cache the low-frequency dedicated shards in the high-speed solid-state disk array of the edge server, and realize the automatic preloading and eviction of shards through a two-factor strategy of least recently used and access prediction.

2. The lightweight AI model dynamic loading system for industrial edge computing as described in claim 1, characterized in that, The multi-dimensional decision matrix in the context-aware adaptation module is generated by offline reinforcement learning training. Its state space covers the dimensions of device model, load level, ambient temperature and humidity, and battery power. Its action space corresponds to different model segmentation and combination strategies. Its reward function comprehensively considers inference accuracy, response latency, and energy consumption indicators.

3. The lightweight AI model dynamic loading system for industrial edge computing as described in claim 1, characterized in that, The incremental loading executor includes: The global symbol table maintains sub-units used to record the function entry addresses and tensor descriptors of all loaded slices; The runtime symbol resolution subunit is used to automatically resolve the external reference symbols of a new shard after it is loaded and bind them to the entries in the global symbol table to complete the cross-shard data channel connection. The conditional loading control subunit is used to activate the loading of the corresponding auxiliary slice only when the input data meets the preset trigger condition expression.

4. The lightweight AI model dynamic loading system for industrial edge computing as described in claim 3, characterized in that, The incremental loading executor also includes: The dual-buffered switching subunit is used to maintain the foreground inference buffer and the background loading buffer. After the background is ready, the new shard is seamlessly switched into the main inference process through atomic pointer swapping operations to ensure service continuity.

5. The lightweight AI model dynamic loading system for industrial edge computing as described in claim 1, characterized in that, The localized fine-tuning unit includes: The drift detection triggering subunit is used to statistically analyze the divergence between the input data distribution and the baseline distribution through a sliding window, and to determine that concept drift has occurred when the divergence exceeds a set threshold, thereby initiating the fine-tuning process. Selective gradient update subunits are used to retain gradient flow only for the last two layers of the network during fine-tuning, while the gradients of the remaining layers are explicitly set to zero. The performance evaluation and rollback sub-unit is used to evaluate the performance gain after fine-tuning. If the gain does not reach the preset threshold, it rolls back to the original weight.

6. The lightweight AI model dynamic loading system for industrial edge computing according to claim 1, characterized in that, The system supports multiple versions coexisting and a canary release mechanism. Different optimized versions of the same model shard can coexist in the shard repository of the edge node. The edge model scheduling engine can specify the precise version according to the target task or automatically select the optimal compatible version according to the current hardware platform.

7. The lightweight AI model dynamic loading system for industrial edge computing according to claim 6, characterized in that, The new version of the fragments is updated via silent background download. After the integrity verification is passed, it enters the activation state, and the operation and maintenance instructions or policy rules determine when to enter the main process through the incremental loading executor.

8. The lightweight AI model dynamic loading system for industrial edge computing according to claim 1, characterized in that, The system integrates a secure and trusted execution environment. All model fragments must be digitally signed before loading, and their source must belong to an authorized model registry center. Intermediate parameters generated during fine-tuning are also processed in the secure and trusted execution environment.

Images