Model deployment method and system

Abstract

The invention discloses a model deployment method and system. Relates to the field of artificial intelligence, and the method comprises the steps: obtaining the operation information, scene information and hardware information of a newly added agent node under the condition that the newly added agent node is detected, and carrying out the lightweight processing of an initial model in a cloud management platform according to the scene information and the hardware information, and obtaining a candidate model; performing fragmented container packaging operation on the candidate model to obtain M initial model fragmented containers, and determining a model deployment mode of a newly added proxy node according to the hardware information and the operation information; selecting a target model fragment container from the M initial model fragment containers according to a model deployment mode to obtain N target model fragment containers; and encrypting and deploying the N target model fragment containers to the newly added proxy node. Through the method and the device, the problem that normal operation of the terminal equipment is affected due to excessive consumption of resources of the terminal equipment caused by a model deployment mode of full-quantity model deployment in related technologies is solved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and more specifically, to a model deployment method and system. Background Technology

[0002] With the increasing application and promotion of neural network models, the scenarios in enterprises that require model deployment are also increasing. Currently, when deploying models, the common approach is to train a fully functional model on the cloud management platform in the data center and then push the centrally trained model to the terminal devices in each branch office, thereby unifying the model version and simplifying the deployment process.

[0003] However, this model exposes significant problems when facing resource-constrained edge terminals: the model is too large, not only occupying the terminal device's storage space but also consuming a large amount of computing resources, causing the terminal device to be overloaded when running the model, directly affecting the normal operation of the terminal and the performance of other applications. Furthermore, the full deployment of the model means that when updating any part of the model, the model on all terminal devices needs to be redeployed and updated. This means that even if a terminal device does not use the updated functional module, the model still needs to be updated, thus affecting the normal application of the model on the terminal device.

[0004] There is currently no effective solution to the problem that the full-scale model deployment method in related technologies leads to excessive consumption of terminal device resources and affects the normal operation of terminal devices. Summary of the Invention

[0005] This application provides a model deployment method and system to solve the problem in related technologies where the full-scale model deployment method leads to excessive consumption of terminal device resources, affecting the normal operation of the terminal device.

[0006] According to one aspect of this application, a model deployment method is provided, applied to a cloud management platform. The method includes: upon detecting a new agent node, acquiring the new agent node's runtime information, scenario information, and hardware information; performing lightweight processing on an initial model in the cloud management platform based on the scenario information and hardware information to obtain a candidate model; performing fragmented container encapsulation on the candidate model to obtain M initial model fragment containers; determining the model deployment method for the new agent node based on the hardware information and runtime information; selecting target model fragment containers from the M initial model fragment containers according to the model deployment method to obtain N target model fragment containers, where M and N are both positive integers, and M is greater than or equal to N; and encrypting and deploying the N target model fragment containers to the new agent node, wherein the new agent node combines the N target model fragment containers to obtain a target model, and processes the pending data sent from the terminal device to the new agent node using the target model.

[0007] Optionally, the initial model in the cloud management platform is lightweighted based on scenario information and hardware information to obtain candidate models, including: determining the resource requirements and constraints of deploying the initial model on the newly added agent node based on scenario information and hardware information; determining the preset operation method and operation coefficients for lightweighting the initial model based on resource requirements and constraints; and lightweighting the initial model based on the preset operation method and operation coefficients to obtain candidate models.

[0008] Optionally, the candidate model is encapsulated into M initial model fragment containers by performing a fragmented containerization operation. This includes: splitting the candidate model according to its functional type to obtain M initial model fragments, wherein the initial model fragments include at least one of the following: basic type fragments, scene type fragments, and hardware optimization type fragments; generating configuration interfaces for each initial model fragment, and updating the code of each initial model fragment according to the configuration interfaces through code refactoring to obtain M target model fragments; obtaining the environment configuration information and dependencies of each target model fragment, and encapsulating each target model fragment and its corresponding environment configuration information and dependencies into an initial model fragment container to obtain M initial model fragment containers.

[0009] Optionally, determining the model deployment method for the newly added agent node based on hardware and operational information includes: determining the architecture type of the newly added agent node based on hardware information and the operational status of the newly added agent node based on operational information; determining the remaining operational resource information of the newly added agent node based on architecture type and operational status; obtaining the operational resource requirements of candidate models and determining whether to allow the newly added agent node to deploy candidate models based on the remaining operational resource information and operational resource requirements; if the newly added agent node is allowed to deploy candidate models, determining the model deployment method as container assembly deployment mode; if the newly added agent node is not allowed to deploy candidate models, determining the model deployment method as lightweight deployment mode.

[0010] Optionally, selecting the target model fragment container from the M initial model fragment containers according to the model deployment method includes: when the model deployment method is container assembly deployment mode, determining the M initial model fragment containers as the target model fragment container; when the model deployment method is lightweight deployment mode, obtaining the basic type fragment container and the scene type fragment container from the M initial model fragment containers, and determining the basic type fragment container and the scene type fragment container as the target model fragment container.

[0011] Optionally, the encrypted deployment of N target model fragment containers to the new agent node includes: constructing a hash tree based on the N target model fragment containers and obtaining the root hash value; obtaining the device fingerprint information of the new agent node and using the device fingerprint information to encrypt the N target model fragment containers to obtain the encrypted model; encrypting the encrypted model and the root hash value using a preset encryption algorithm to obtain encrypted information, and sending the encrypted information to the new agent node.

[0012] Optionally, the method further includes: when the scene information of the newly added agent node is detected to have changed, training the initial model based on the new scene information to obtain an updated model; obtaining a new model fragment container associated with the new scene information from the updated model, and deploying the new model fragment container to the new agent node.

[0013] According to another aspect of this application, a model deployment method is also provided, applied to adding a new agent node. The method includes: receiving an encrypted model sent by a cloud management platform, decrypting the encrypted model to obtain N target model fragment containers, wherein the N target model fragment containers are obtained by the cloud management platform after processing an initial model based on the running information, scene information, and hardware information of the new agent node; combining the N target model fragment containers to obtain a target model; and, upon receiving data to be processed sent by a terminal device, processing the data to be processed using the target model to obtain a data processing result, and feeding the data processing result back to the terminal device.

[0014] Optionally, the method further includes: acquiring the operating data of the terminal device, and determining whether the deployment of the target model is allowed in the terminal device based on the operating data; if the deployment of the target model is allowed in the terminal device, deploying the target model to the terminal device, and processing the data to be processed through the target model in the terminal device.

[0015] According to another aspect of this application, a model deployment system is provided. The system includes: a cloud management platform, multiple initial agent nodes, and terminal devices under each initial agent node. The cloud management platform, connected to the multiple initial agent nodes, is used to, upon detecting a new agent node among the multiple initial agent nodes, acquire the operating information, scene information, and hardware information of the new agent node, process the initial model in the cloud management platform according to the scene information and hardware information to obtain a target model, and deploy the target model to the new agent node. The new agent node, connected to the cloud management platform, is used to receive and deploy the target model, and upon receiving pending data sent by the terminal device under the new agent node, process the pending data through the target model to obtain a data processing result, and feed the data processing result back to the terminal device. The terminal device under the new agent node, connected to the new agent node, is used to send pending data to the new agent node and receive the data processing result sent by the new agent node.

[0016] In this embodiment, upon detecting a new agent node, the following steps are taken: First, the runtime information, scenario information, and hardware information of the new agent node are obtained. Then, the initial model in the cloud management platform is lightweighted based on the scenario and hardware information to obtain candidate models. Next, fragmented container encapsulation is performed on the candidate models to obtain M initial model fragment containers. The model deployment method for the new agent node is determined based on the hardware and runtime information. Then, target model fragment containers are selected from the M initial model fragment containers according to the model deployment method to obtain N target model fragment containers, where M and N are positive integers, and M is greater than or equal to N. Finally, the N target model fragment containers are encrypted and deployed to the new agent node. The new agent node combines the N target model fragment containers to obtain a target model, and uses the target model to process the data to be processed sent from the terminal device to the new agent node. This method fragments the model by encapsulating it into fragmented containers. Based on information from newly added proxy nodes, it determines the model deployment method and the required containers. The model is then deployed to the proxy nodes in a fragmented manner, rather than directly to the terminal devices. When a terminal device needs to use the model, it can interact with the proxy nodes without deploying the model on the terminal device itself. This achieves lightweight model deployment and reduces terminal resource usage. Consequently, it reduces terminal device resource consumption and improves terminal device operating efficiency without affecting model usage. This solves the technical problem of excessive resource consumption and disruption of normal terminal device operation caused by full model deployment in related technologies. Attached Figure Description

[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0018] Figure 1 This is a flowchart of an optional model deployment method provided according to an embodiment of this application;

[0019] Figure 2 This is a flowchart of another optional model deployment method provided according to an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of a model deployment system provided according to an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of an optional model deployment device provided according to an embodiment of this application;

[0022] Figure 5 This is a schematic diagram of another optional model deployment apparatus provided according to an embodiment of this application;

[0023] Figure 6 This is a schematic diagram of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] It should be noted that the model deployment method and system defined in this disclosure can be used in the field of artificial intelligence, or in any field other than artificial intelligence. The application fields of the model deployment method and system defined in this disclosure are not limited.

[0028] The embodiments or examples disclosed herein are not exhaustive, but merely illustrative of some embodiments or examples, and are not intended to limit the scope of protection of this disclosure. Unless otherwise specified, each step in a particular embodiment or example can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a particular embodiment or example can also be implemented as an independent embodiment, and the order of the steps in a particular embodiment or example can be arbitrarily interchanged. Furthermore, optional methods or examples in a particular embodiment or example can be arbitrarily combined; moreover, embodiments or examples can be arbitrarily combined. For example, some or all steps of different embodiments or examples can be arbitrarily combined, and a particular embodiment or example can be arbitrarily combined with optional methods or examples of other embodiments or examples.

[0029] For ease of description, the following explains some of the nouns or terms used in the embodiments of this application:

[0030] CPU: Central Processing Unit.

[0031] GPU: Graphics Processing Unit.

[0032] NPU: Neural Processing Unit.

[0033] ARM architecture: Advanced RISC Machine, a reduced instruction set computer architecture that can be used in mobile devices, embedded systems, and edge computing devices.

[0034] RISC-V: Reduced Instruction Set Computer V, is the fifth generation of a reduced instruction set computer architecture and is an open-source reduced instruction set computer architecture.

[0035] SM4 symmetric encryption algorithm: a commercial cryptographic algorithm used for data encryption and decryption.

[0036] Merkle hash tree algorithm: an algorithm used for data integrity and consistency verification.

[0037] According to embodiments of this application, a model deployment method is provided, applied to a cloud management platform. Figure 1 This is a flowchart of an optional model deployment method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:

[0038] Step S101: When a new agent node is detected, obtain the running information, scenario information and hardware information of the new agent node, and perform lightweight processing on the initial model in the cloud management platform based on the scenario information and hardware information to obtain a candidate model.

[0039] It should be noted that the execution entity in this embodiment is the cloud management platform in the model deployment system. This platform can be a private cloud server deployed in a data center, responsible for model training, dynamic optimization, intelligent scheduling, and global monitoring. The cloud management platform can utilize its built-in intelligent modules, such as a scene-aware model training and optimization module, to perform lightweight processing and optimization of the model based on the hardware and scene information of the edge proxy nodes, while simultaneously managing model deployment, updates, and resource allocation. New proxy nodes can be newly added edge proxy nodes, deployed on devices in the edge computing layer, such as industrial gateways, edge servers, or intelligent terminals with certain computing capabilities. These devices are located at the network edge between the data center and the terminal devices. One edge proxy node can cover and serve multiple terminal devices, responsible for receiving and managing model fragments from the cloud, locally assembling and deploying models and applying them, optimizing resource utilization, and maintaining device security. For example, in a smart factory environment, an industrial gateway, acting as an edge proxy node, can connect dozens or even hundreds of terminal devices such as sensors, robots, and cameras. Operational information refers to the current operating status and resource usage of the agent node device, which may include CPU utilization, memory usage, storage space, network bandwidth, and latency, used to assess the device's real-time performance and resource availability. Scenario information describes the application environment and business requirements of the agent node device, such as industrial quality inspection, intelligent security, and environmental monitoring. Hardware information includes the hardware configuration of the agent node device, such as CPU architecture type, GPU and NPU acceleration unit models, memory and storage capacity, used to identify the device's computing power characteristics. Lightweight processing refers to reducing model size and resource consumption through techniques such as model parameter compression, structural pruning, and knowledge distillation, while maintaining the model's predictive accuracy as much as possible, to adapt to resource-constrained agent node devices.

[0040] For example, when the cloud management platform detects a new edge agent node joining the system, it first triggers an information collection mechanism to obtain the new agent node's operational information, scenario information, and hardware information. Operational information includes, but is not limited to, the agent node's current load status, network conditions, and security posture; scenario information refers to the business requirements and data characteristics of the environment in which the agent node is located; and hardware information covers specific parameters of computing units such as CPU architecture, memory capacity, storage space, and GPU / NPU. This information can be transmitted from the new agent node to the cloud management platform via a secure communication protocol with two-way authentication, improving privacy and security during transmission.

[0041] Furthermore, the cloud-based management platform performs lightweight processing on the initial model based on the above information. By analyzing the hardware limitations and business scenarios of the newly added agent nodes, it adopts adaptive quantization, dynamic structured pruning, and knowledge distillation enhancement techniques to cut off model parts that are irrelevant to the scenario, compress the model size, and at the same time make the accuracy loss less than a preset threshold, thereby obtaining the processed candidate model.

[0042] For example, the cloud management platform may include a scene-aware model training and optimization module. This module can introduce a "scene-hardware" dual-dimensional dynamic branching architecture and integrate a dynamic lightweight toolchain, including adaptive quantization, dynamic structured pruning and knowledge distillation enhancement techniques, to perform specific optimizations for different hardware architectures, such as x86, ARM and RISC-V, so as to achieve efficient operation of large models on heterogeneous hardware.

[0043] In other words, the scene-aware model training and optimization module integrates a dynamic lightweight toolchain, including adaptive quantization, dynamic structured pruning, and knowledge distillation enhancement techniques, which work together to optimize the model. Adaptive quantization intelligently adjusts the precision of model parameters, converting floating-point numbers to integer or low-precision representations, effectively reducing model storage requirements and computational load, and improving operational efficiency on resource-constrained devices. Dynamic structured pruning intelligently trims redundant parts of the model based on its actual computational needs and the resource availability of the proxy node devices, further compressing the model size and reducing inference time. Knowledge distillation enhancement extracts knowledge from a large teacher model into a smaller student model, maintaining the model's prediction accuracy while significantly reducing model complexity, thereby improving the model's deployment efficiency on proxy node devices.

[0044] For example, in intelligent traffic management scenarios, the module optimizes large-scale traffic prediction models using a dynamic lightweight toolchain based on the hardware information of edge agent node devices (ARM architecture) and the business requirements of real-time traffic flow analysis. First, adaptive quantization technology reduces the storage requirements of model parameters. Next, dynamic structured pruning is used to trim non-critical computational paths, reducing redundant computations. Finally, knowledge distillation enhancement technology is used to extract the decision logic of the large model into a lightweight model, thereby achieving fast and accurate traffic flow prediction under the ARM architecture. This also adapts to low-power, low-storage device environments, improving the model's generalization ability and the response speed of agent node devices.

[0045] Step S102: Perform fragmented container encapsulation on the candidate models to obtain M initial model fragment containers, and determine the model deployment method for the newly added agent nodes based on hardware information and runtime information.

[0046] It's important to note that fragmented container encapsulation involves breaking down the model into multiple functionally independent container fragments. This facilitates combined deployment based on the resource status of edge proxy nodes and business needs, while also supporting on-demand loading and incremental updates. Model deployment methods refer to selecting the most suitable deployment mode based on the hardware and operational information of the edge proxy nodes. These include container assembly deployment, native lightweight deployment, and scenario-adaptive switching deployment, ensuring efficient model operation and rational resource utilization.

[0047] For example, after obtaining the lightweight candidate model, the fragmented container encapsulation module of the cloud management platform can adopt fragmented container orchestration technology, including environment adaptive configuration, model fragment construction, and intelligent version management, to split the candidate model into M independent initial model fragment containers. Each model fragment container contains a subset of models for a specific function or scenario and related configurations, facilitating subsequent on-demand assembly and incremental updates of the model. Furthermore, fragmented encapsulation not only considers the functional division of the model but also integrates environment adaptive configuration, enabling the container to run seamlessly on different hardware architectures (such as x86, ARM, and RISC-V). For instance, a complete model can be split into fragmented containers such as a basic model, a scenario model, and a hardware-optimized model to adapt to the hardware configurations and business requirements of different edge agent nodes.

[0048] Furthermore, based on the hardware and operational information of the edge proxy nodes, the cloud-based management platform determines the model deployment method for each edge proxy node by comprehensively considering its resource status, business scenarios, and real-time requirements. This ensures that the model deployment meets the business needs of the edge proxy nodes while fully utilizing their device resources and reducing resource consumption. For example, for edge proxy nodes with sufficient resources, a container assembly deployment mode can be selected, while for edge proxy nodes with limited resources, a native lightweight deployment mode is adopted.

[0049] Step S103: Select target model fragment containers from M initial model fragment containers according to the model deployment method to obtain N target model fragment containers, where M and N are both positive integers, and M is greater than or equal to N.

[0050] For example, after determining the model deployment method, N target model fragment containers can be selected from M initial model fragment containers according to the selected model deployment method. The N fragment containers contain all the model components required by the edge agent node to perform tasks. The selection process is based on the resource status and scenario requirements of the edge agent node, so that the deployed model fragments can not only meet the business functions, but also make full use of device resources, reduce unnecessary model component loading, and further reduce resource consumption.

[0051] Step S104: Deploy N target model fragment containers to the newly added agent node in an encrypted manner. The newly added agent node combines the N target model fragment containers to obtain the target model, and processes the data to be processed sent from the terminal device to the newly added agent node through the target model.

[0052] For example, after obtaining N target model fragment containers, the N target model fragment containers need to be deployed to the newly added agent node through an encrypted channel. The encryption mechanism combines multi-factor authentication of device fingerprints and dynamic keys, uses the SM4 symmetric encryption algorithm to encrypt data, improves the security of model fragments during transmission, and also uses the Merkle hash tree algorithm to verify the integrity and correctness of model fragments, preventing the model from being tampered with or damaged during transmission, thereby improving the security of data transmission.

[0053] Finally, after receiving the encrypted containers of N target model fragments, the newly added agent node decrypts and dynamically combines them. Through local assembly of the model fragments, and based on predefined interface specifications and the hardware characteristics of the edge agent node, it automatically completes the model assembly operation to obtain the target model. Then, the terminal execution engine configured in the newly added agent node can receive and process the data to be processed from the terminal device, perform inference calculations using the target model, and generate analysis results or execution decisions. For example, in intelligent security monitoring, the terminal execution engine receives the real-time video stream transmitted from the camera, uses the target model for face detection and recognition, and feeds back the recognition results to the monitoring system for real-time alerts and identity verification.

[0054] Therefore, the model deployment method adopted in this embodiment not only optimizes the matching between the model and terminal resources, reduces resource consumption and the complexity of model deployment, and improves business continuity and deployment efficiency, but also effectively protects data privacy and model security through three-dimensional security protection and hierarchical privacy protection update mechanism.

[0055] The model deployment method provided in this application fragments the model by performing fragmented container encapsulation. It determines the model deployment method and the required containers based on the information of newly added proxy nodes, and then deploys the model to the proxy nodes in a fragmented manner according to the containers. By deploying the model to the proxy nodes instead of the terminal devices, the terminal devices can use the model through interaction with the proxy nodes when needed, without having to deploy the model on the terminal devices. This achieves lightweight model deployment and reduces terminal resource usage, thereby reducing the resource consumption of the terminal devices and improving their operating efficiency without affecting their ability to use the model. This solves the technical problem in related technologies where full model deployment leads to excessive resource consumption on the terminal devices, affecting their normal operation.

[0056] To improve the accuracy of candidate models, optionally, in the model deployment method provided in this application embodiment, the initial model in the cloud management platform is lightweighted based on scene information and hardware information to obtain candidate models, including: determining the resource requirements and constraints of deploying the initial model on the newly added agent node based on scene information and hardware information; determining the preset operation method and operation coefficients for lightweighting the initial model based on resource requirements and constraints; and lightweighting the initial model based on the preset operation method and operation coefficients to obtain candidate models.

[0057] For example, when lightweighting the model, it is first necessary to determine the resource requirements and constraints of the initial model deployment on the new agent node based on the scenario information and hardware information of the new agent node. The scenario information involves the application field of the model, such as intelligent quality inspection in industrial manufacturing and environmental monitoring in smart homes. This information is used to guide the optimization direction of the model so that the model function matches the scenario requirements. The hardware information includes key parameters such as the computing power, storage capacity, and memory size of the new agent node device, thereby assessing the resource bottleneck of the device and determining the specific goals of model lightweighting.

[0058] For example, in an industrial quality inspection scenario, if a newly added agent node device (such as a smart gateway) has 8GB of memory and 64GB of storage, the goal of lightweighting is to keep the model's memory usage below 4GB and its storage usage below 32GB, ensuring the model can run smoothly on the new agent node device while reserving sufficient resources for other tasks. Simultaneously, considering the high accuracy requirements of industrial quality inspection, lightweighting must minimize resource consumption while keeping accuracy loss within 1%.

[0059] Furthermore, based on defined resource requirements and constraints, preset operation methods and coefficients can be selected from the lightweight toolchain of the cloud management platform. Operation methods include adaptive quantization, dynamic structured pruning, and knowledge distillation enhancement, while operation coefficients define the specific degree of these operations, such as quantization accuracy, pruning ratio, and distillation temperature. For example, based on the requirements of the aforementioned industrial quality inspection scenario, the following lightweight operation method and coefficient combination might be selected: Adaptive Quantization: Quantizes model parameters from 32-bit floating-point to 8-bit integers, reducing memory and storage requirements. Simultaneously, through dynamic quantization range adjustment, the quantization accuracy of key layers meets the requirements of high-precision inference. Dynamic Structured Pruning: Based on computational graph analysis of the model, a pruning ratio of 30% is determined, removing redundant neural network layers and connections to reduce model size and inference time, while retaining critical paths to maintain model accuracy. Knowledge Distillation Enhancement: Employs knowledge distillation technology, selecting a pre-trained teacher model for distillation, with a distillation temperature of 2 for the student model. This allows the student model to maintain high prediction accuracy even at a smaller scale, suitable for low-resource environments of edge agent nodes.

[0060] Finally, based on the selected operation method and operation coefficient, the cloud management platform performs lightweight processing on the initial model to obtain candidate models. Among these, one or more operation methods can be selected from the above operation methods to perform lightweight operations on the model, according to the specific operation method of lightweight processing.

[0061] Through the above steps, the cloud management platform transforms the initial model into a candidate model, reducing the model size and resource consumption, improving the deployment efficiency and performance of the model on edge agent nodes, while maintaining high prediction accuracy, business continuity, and high-quality service.

[0062] To accurately obtain the initial model fragment containers, optionally, in the model deployment method provided in this application embodiment, performing fragmented container encapsulation on the candidate models to obtain M initial model fragment containers includes: splitting the candidate models according to their functional types to obtain M initial model fragments, wherein the initial model fragments include at least one of the following: basic type fragments, scene type fragments, and hardware optimization type fragments; generating configuration interfaces for each initial model fragment, and updating the code of each initial model fragment according to the configuration interfaces through code refactoring operations to obtain M target model fragments; obtaining the environment configuration information and dependencies of each target model fragment, and encapsulating each target model fragment and its corresponding environment configuration information and dependencies into an initial model fragment container to obtain M initial model fragment containers.

[0063] For example, when performing fragmented container encapsulation on candidate models, the candidate models are first split according to their functional types, resulting in M ​​initial model fragments. The goal of this model splitting is to decompose them into three types of fragments: basic type fragments, scenario type fragments, and hardware-optimized type fragments. Basic type fragments contain the core computational logic and key parameters of the model; scenario type fragments need to be customized according to specific application scenario requirements, such as fragments specific to defect detection or dimensional measurement in industrial quality inspection scenarios; and hardware-optimized type fragments are specifically optimized for different hardware architectures, such as adaptive quantization processing on x86 architecture or dynamic structured pruning on ARM architecture. This splitting process maintains the modularity of model functionality, facilitating flexible deployment and management based on the characteristics of different edge proxy nodes and differences in application scenarios.

[0064] Furthermore, configuration interfaces for each initial model fragment need to be generated. Through code refactoring, the code of each initial model fragment is updated according to the configuration interfaces to obtain M target model fragments. The configuration interfaces define the communication protocols, data formats, and invocation methods between model fragments, enabling seamless collaboration on edge proxy nodes. Code refactoring, while maintaining model functionality, reorganizes the code structure to improve readability, maintainability, and portability. Through configuration interfaces and code refactoring, fragments can be dynamically optimized and adjusted based on the actual resource conditions of edge proxy nodes, improving the model's operating efficiency in edge environments. For example, by introducing service communication interfaces, efficient data exchange and invocation between model fragments are achieved. Simultaneously, code refactoring optimizes the model's input and output formats, allowing fragments to run smoothly on different edge proxy nodes and avoiding deployment failures or performance degradation due to data format incompatibility.

[0065] Finally, the environment configuration information and dependencies for each target model fragment are obtained, and each target model fragment and its corresponding environment configuration information and dependencies are encapsulated into an initial model fragment container, resulting in M ​​initial model fragment containers. The environment configuration information includes the basic resource requirements for fragment operation, such as the number of CPU cores, memory size, and disk storage, as well as network parameter settings, such as port mapping and communication protocols. Dependencies refer to the external libraries and tools required for fragment operation. These dependencies must be packaged with the fragment to avoid runtime errors caused by inconsistent library versions on edge proxy nodes. Container encapsulation can use standardized tools to package the model fragment, configuration information, and dependencies into an independent, portable container image, enabling the fragment to be used plug-and-play on any edge proxy node, greatly simplifying the deployment process and improving deployment efficiency.

[0066] This embodiment achieves accurate transformation of candidate models into M initial model fragment containers by splitting the model, refactoring the code, and encapsulating them into containers, thereby improving the accuracy and usability of the initial model fragment containers.

[0067] To accurately determine the model deployment method, optionally, in the model deployment method provided in this application embodiment, determining the model deployment method of the newly added agent node based on hardware information and operating information includes: determining the architecture type of the newly added agent node based on hardware information, and determining the operating status of the newly added agent node based on operating information; determining the remaining operating resource information of the newly added agent node based on the architecture type and operating status; obtaining the operating resource information of the candidate model, and determining whether the newly added agent node is allowed to deploy the candidate model based on the remaining operating resource information and the operating resource information; if the newly added agent node is allowed to deploy the candidate model, determining the model deployment method as container assembly deployment mode; if the newly added agent node is not allowed to deploy the candidate model, determining the model deployment method as lightweight deployment mode.

[0068] For example, when determining the model deployment method, it is first necessary to determine the architecture type of the new agent node based on hardware information, and the running status of the new agent node based on operational information. Hardware information includes, but is not limited to, processor architecture, memory size, storage capacity, and the presence or absence of accelerators such as GPUs / NPUs. Operational information includes the current CPU and memory utilization, storage usage status, network bandwidth and latency, and the applications and services currently running on the edge agent node. For instance, for an edge server, its hardware may be an x86 architecture with ample memory and storage; its operational information might show a current CPU utilization of 60%, memory utilization of 75%, storage usage of 50%, and low network latency, primarily used for tasks such as video analytics and pedestrian recognition.

[0069] Furthermore, after obtaining the architecture type and operational status, it is necessary to determine the remaining operational resource information of the newly added agent nodes. This means determining the computing, storage, and network resources that the edge agent nodes can currently allocate to the new model. This allows for an accurate assessment of the existing load, ensuring that the deployment of the new model does not excessively consume resources and thus affect the stability of existing services. For example, the remaining operational resource information could be: 30% of the edge server's remaining CPU resources, 25% of its remaining memory, and 50% of its remaining storage.

[0070] Furthermore, it is also necessary to obtain the resource requirements for the candidate model, including the basic number of CPU cores, memory size, storage space, and potential accelerator requirements needed for model inference. Then, by comparing the remaining runtime resource information of the newly added agent node with the runtime resource requirements of the candidate model, it can be determined whether the newly added agent node is allowed to deploy the candidate model.

[0071] For example, if the remaining resources of the edge proxy node are sufficient to meet the model's operational requirements, deployment is permitted, and the system determines the model deployment method as container assembly deployment mode. If deployment is not permitted, the system determines the model deployment method as lightweight deployment mode. Container assembly deployment mode is suitable for edge proxy nodes with sufficient resources, using container technology to achieve isolated deployment of the model and environment, improving model integrity and operational stability. Lightweight deployment mode, on the other hand, is designed for resource-constrained scenarios. By discarding container runtime dependencies and directly loading core model fragments, it minimizes resource utilization and meets the low-power, low-latency requirements of edge computing.

[0072] Finally, the system executes the corresponding deployment process based on the determined model deployment method. For the container assembly deployment mode, the system assembles the model fragments into a complete container image and deploys it using container orchestration tools; while for the lightweight deployment mode, the system only deploys the core model fragments to the local file system of the edge proxy node, automatically resolving dependencies through dynamic link library technology to reduce resource consumption.

[0073] This embodiment dynamically determines the optimal model deployment method based on the hardware architecture and operational status of the edge proxy nodes, achieving fine-grained control over model deployment and efficient resource utilization. On one hand, for edge proxy nodes with sufficient resources, the container assembly deployment mode provides a stable and secure model runtime environment, ensuring high-performance inference and data isolation. On the other hand, for resource-constrained scenarios, the lightweight deployment mode significantly reduces model resource consumption, enabling fast and low-power model startup and operation, meeting the specific needs of edge computing environments. Furthermore, this method supports dynamic switching of deployment modes as needed. For example, when the resource status of the edge proxy nodes changes, the deployment strategy can be automatically adjusted, ensuring continuous and efficient model operation while reducing deployment failure rates and the probability of operational anomalies.

[0074] To accurately select the target model fragment container, optionally, in the model deployment method provided in this application embodiment, selecting the target model fragment container from M initial model fragment containers according to the model deployment mode includes: when the model deployment mode is container assembly deployment mode, determining the M initial model fragment containers as target model fragment containers; when the model deployment mode is lightweight deployment mode, obtaining basic type fragment containers and scene type fragment containers from the M initial model fragment containers, and determining the basic type fragment containers and scene type fragment containers as target model fragment containers.

[0075] It's important to note that the container assembly deployment mode is suitable for resource-rich edge proxy nodes. By integrating multiple model fragment containers, it generates a complete model runtime environment, achieving isolated deployment of the model from the hardware environment and ensuring stable model operation and data security. The lightweight deployment mode is used for resource-constrained edge proxy nodes. It discards additional dependencies of the container runtime, loading only basic type fragment containers and scene type fragment containers to achieve minimal resource consumption, maintaining the model's rapid startup and real-time inference capabilities on low-power, low-storage devices.

[0076] For example, when the model deployment method is determined to be container assembly deployment mode, the selection process of the target model fragment container is relatively straightforward. The system will select all containers from the M initial model fragment containers as the target model fragment containers, including basic type fragment containers, scene type fragment containers, and hardware optimization type fragment containers.

[0077] For example, the container assembly deployment mode can be used for edge proxy nodes with sufficient resources (such as x86 architecture edge gateways, edge proxy nodes with ≥16GB of memory and ≥128GB of storage). The core is to dynamically assemble container instances from fragments to achieve isolated deployment of the model and the environment. The specific process is as follows:

[0078] (1) The edge agent node receives the complete model fragment combination (basic fragment + scene fragment + hardware optimization fragment) sent from the cloud. First, it verifies the integrity of the fragments through the Merkle hash tree algorithm, and then completes the compatibility test in combination with the device hardware architecture (such as x86) and operating system to ensure that the fragment combination has no dependency conflict.

[0079] (2) The system automatically generates a minimal container configuration file, specifying resource quotas (such as 2 CPU cores, 4GB memory, 10GB storage), network parameters (such as port mapping, communication protocol) and startup commands, without requiring manual intervention in environment configuration;

[0080] (3) When the container is invoked, the model fragments (basic fragments, scene fragments, hardware optimization fragments) are dynamically associated according to the predefined interface and assembled into a complete container image. The dependencies between fragments are automatically resolved through the association rules preset in the cloud.

[0081] (4) Start the container instance and use Linux cgroups isolation technology to allocate an independent resource pool for the container, isolate other application processes, and avoid resource contention.

[0082] When the model deployment method is determined to be a lightweight deployment mode, the selection of target model fragment containers requires more refined screening. The system retrieves only basic type fragment containers and scene type fragment containers from M initial model fragment containers, discarding unnecessary hardware-optimized type fragment containers to minimize resource consumption. Basic type fragment containers maintain the basic functionality of model inference, while scene type fragment containers are customized and optimized according to the specific scenario of the device.

[0083] For example, lightweight deployment mode can be used for resource-constrained edge proxy nodes (such as terminal devices with ≤8GB memory and ≤64GB storage). The core is to discard container runtime dependencies and directly load core fragments to achieve lightweight deployment. The specific process is as follows:

[0084] (1) Edge proxy nodes receive the core model fragment set sent from the cloud (only basic fragments and key scene fragments are retained, and non-essential function fragments are discarded), skip container runtime compatibility checks, and focus on core inference capability adaptation;

[0085] (2) Automatically extract the core files from the fragments: model files, lightweight dependency libraries and initialization scripts, and deploy them directly to a dedicated directory in the local file system of the edge agent node;

[0086] (3) The dependency relationship is automatically resolved by dynamic link library technology, and only the core dependencies necessary for inference are loaded. There is no need to install the complete runtime environment, which reduces the consumption of redundant resources;

[0087] (4) Execute the initialization script and use memory mapping technology to optimize model storage and retrieval, directly mapping the model to memory;

[0088] (5) Start the inference service as an independent process and interact with the edge agent node through a simplified Socket communication interface. The resource consumption is strictly controlled to CPU≤1 core and memory≤2GB to meet the low resource consumption requirements of embedded devices.

[0089] In other words, the process of selecting target model fragment containers, whether in container assembly deployment mode or lightweight deployment mode, aims to improve the resource efficiency and scenario adaptability of model deployment. With sufficient resources, integrating all fragment containers can achieve the integrity of model functionality and environmental isolation, ensuring high-performance inference and data security. In resource-constrained environments, streamlining to basic type fragment containers and scenario type fragment containers not only reduces model size and runtime overhead but also improves model effectiveness in specific scenarios, meeting the constraints of low power consumption and low storage requirements of the device.

[0090] This embodiment achieves the goal of accurately selecting the target container from the initial model fragment containers by judging two modes: container assembly and lightweight deployment. This enables the model to run efficiently in the edge environment, and the dynamic adjustment of container selection adapts to various resource conditions and application scenario requirements. In the container assembly deployment mode, the system can utilize the abundant resources of the edge proxy nodes to build a complete model runtime environment and provide stable, high-performance inference services. In the lightweight deployment mode, even on devices with extremely limited resources, the system can accurately load basic and scenario model fragments to realize the key functions of the model and meet the real-time inference needs of specific scenarios.

[0091] To improve the security of model deployment, optionally, in the model deployment method provided in this application embodiment, encrypting and deploying N target model fragment containers to a new agent node includes: constructing a hash tree based on the N target model fragment containers and obtaining the root hash value; obtaining the device fingerprint information of the new agent node and using the device fingerprint information to encrypt the N target model fragment containers to obtain an encrypted model; encrypting the encrypted model and the root hash value using a preset encryption algorithm to obtain encrypted information, and sending the encrypted information to the new agent node.

[0092] It's important to note that a hash tree is a tree-like data structure used for data integrity. It generates a root tree through hierarchical hash calculations, allowing verification of data block integrity without downloading the entire data set. A device fingerprint is a unique identifier generated based on the hardware characteristics and software environment of the edge agent node, used for device authentication and security verification. A preset encryption algorithm refers to a pre-defined encryption standard, such as the SM4 symmetric encryption algorithm, used to encrypt the model and related data, ensuring security during transmission.

[0093] For example, to improve the security of the model transmission process, the system constructs a Merkle hash tree based on the contents of N target model fragment containers. Each fragment container is considered a leaf node in the tree, and its data is hashed using a hash function to obtain a hash value. Adjacent hash values ​​are combined and hashed again until the root node of the tree, i.e., the root hash value, is constructed. This root hash value serves as the benchmark for verifying the integrity of the model fragment containers. For example, the system will generate hash values ​​for fragment containers A, B, and C, and then calculate the final root hash value R by combining them pairwise.

[0094] Furthermore, the system also needs to obtain the device fingerprint information of newly added agent nodes. Device fingerprints include the device's hardware serial number, MAC address, CPU model, etc., forming a unique device identifier. The system uses the device fingerprint information to encrypt N target model fragment containers, generating encrypted models. This encryption process is based on device fingerprints, meaning that each agent node obtains different model fragments. Even if a fragment is intercepted, it cannot be decrypted without the correct device fingerprint, increasing the difficulty for attackers to reverse engineer the model. For example, for agent node 1, its device fingerprint is F1. The system uses F1 to encrypt model fragments A, B, and C, generating encrypted models AE, BE, and CE.

[0095] Furthermore, the system performs secondary encryption on the encrypted model and root hash value using a preset encryption algorithm (such as SM4) to obtain encrypted information. This ensures that even if the encrypted model is intercepted during transmission, attackers cannot easily obtain the original content of the model or verify its integrity. The encrypted information is then sent to the newly added agent node via a secure communication protocol. Only after successfully verifying the device fingerprint, decrypting the encrypted information, and confirming the root hash value is correct can the agent node assemble the model fragment container and deploy it. For example, the system performs secondary encryption on the encrypted models AE, BE, CE, and the root hash value R using the SM4 algorithm to generate the final encrypted information E, and then transmits E to the agent node through a secure channel.

[0096] This embodiment improves the data integrity of the model fragment container during transmission by constructing a hash tree and obtaining the root hash value; it uses device fingerprint information for encryption, combining hardware attributes and encryption technology to form an encrypted model bound to the device, thus improving the model's security; by using double encryption and sending encrypted information, it not only protects the model's confidentiality but also verifies the model's integrity and prevents tampering through the root hash value, thereby improving the accuracy and security of model deployment.

[0097] Optionally, in the model deployment method provided in the embodiments of this application, the method further includes: when the scene information of the newly added proxy node is detected to have changed, training the initial model according to the newly added scene information to obtain an updated model; obtaining a new model fragment container associated with the newly added scene information from the updated model, and deploying the new model fragment container to the newly added proxy node.

[0098] For example, if a change in the scene information of a newly added proxy node is detected, the model in the newly added proxy node needs to be updated. For instance, the system monitors the scene information of the edge proxy node in real time, including but not limited to changes in the ambient light level, temperature, humidity, network latency, and data type of the device's environment. When the system detects a significant change in scene information (such as a change in light intensity from a bright daytime environment to a weak nighttime environment), it triggers the subsequent model training and update process.

[0099] Furthermore, after receiving scene information change reports from edge agent nodes, the cloud management platform analyzes the characteristics of the newly added scene information. Taking changes in lighting as an example, if the newly added scene information indicates that the device is frequently in a low-light environment, the cloud platform will select training samples under low-light conditions from historical datasets, combine them with the current model's performance metrics, retrain the model, or fine-tune the existing model through transfer learning to adapt it to image recognition and analysis tasks in low-light environments, thereby obtaining an updated model.

[0100] After the updated model training is complete, the system extracts model fragments closely related to the new scene information (such as low-light environments) from the updated model. For example, a model for low-light environments may require more complex image preprocessing and more powerful feature extraction capabilities. The system breaks down these low-light-specific functions into model fragments, generates new model fragment containers, and obtains the new model fragment containers.

[0101] Finally, the system sends the newly added model fragment container to the corresponding agent node via a secure communication protocol. Upon receiving the container, the agent node verifies its integrity and origin using a multi-factor authentication mechanism combining device fingerprint and dynamic key, and then dynamically assembles and deploys the fragment container to its local environment. If the agent node already has basic type fragment containers or fragment containers for other scenarios deployed, the system will automatically identify and integrate these containers to form a complete model adapted to the new scenario. For example, if a basic model fragment container already exists on the edge agent node, and a new model fragment container optimized for low-light environments is added, the two will be automatically merged according to a predefined interface, enabling rapid deployment and seamless operation of the model in the new scenario.

[0102] For example, if a change in the scenario information of a newly added proxy node is detected, the fragment combination in the newly added proxy node needs to be dynamically switched according to the real-time scenario and resource status. The specific process is as follows:

[0103] (1) Edge agent nodes collect two-dimensional data in real time: one is scene metadata (such as light intensity of industrial cameras, task type of terminal devices, network latency), and the other is device resource status (CPU utilization, remaining memory, storage usage).

[0104] (2) Match the collected scene features with the predefined scene feature library in the cloud (e.g., low light scene similarity 85%, metal detection scene similarity 92%), and at the same time evaluate the suitability of the currently loaded model fragment group (fit threshold set to 70%).

[0105] (3) If the current fragment group has a suitable matching degree of <70%, the fragment switching process is automatically triggered: the background starts the preloading of the optimal fragment combination corresponding to the target scene (such as switching from "strong light fragment" to "weak light fragment"), completes the model initialization and inference engine warm-up, and avoids delay during switching. At this time, new fragments can be obtained from the cloud. If there are no new fragments for the scene in the cloud, the cloud model needs to be trained to obtain new fragments for the scene.

[0106] (4) After preloading is completed, the inference task is smoothly migrated to the new fragment combination. The old fragment combination retains a 30-second cache (for quick rollback in case of switching failure). The entire switching process takes ≤200ms and does not affect continuous business such as production line and security monitoring.

[0107] (5) After the switch is completed, unused redundant fragments (such as the original "strong light fragments") will be automatically unloaded to release storage space and memory resources.

[0108] This embodiment avoids unnecessary resource consumption by updating only the model fragments related to the newly added scene information when switching scenes, while also speeding up the model update and reducing the waiting time of edge proxy nodes.

[0109] According to an embodiment of this application, another model deployment method is provided, which is applied to adding new agent nodes. Figure 2 This is a flowchart of another optional model deployment method provided according to an embodiment of this application, such as... Figure 2 As shown, the method includes the following steps:

[0110] Step S201: Receive the encrypted model sent by the cloud management platform and decrypt the encrypted model to obtain N target model fragment containers. The N target model fragment containers are obtained by the cloud management platform after processing the initial model based on the running information, scene information and hardware information of the newly added agent node.

[0111] Step S202: Combine the N target model fragment containers to obtain the target model.

[0112] Step S203: Upon receiving the data to be processed from the terminal device, the data to be processed is processed through the target model to obtain the data processing result, and the data processing result is fed back to the terminal device.

[0113] For example, the edge agent node first receives an encrypted model sent by the cloud management platform via a dynamic secure communication protocol. The encrypted model is generated in the cloud management platform based on a multi-factor authentication mechanism of "device fingerprint + dynamic key," ensuring that the model can only be decrypted by a specific edge agent node, preventing the risk of model leakage during transmission. After receiving the model, the edge agent node uses a pre-distributed dynamic key to decrypt it, restoring the original state of the model fragments and obtaining N target model fragment containers. Each fragment container contains a portion of the model's functionality or optimized configuration. The selection and generation of these fragments are based on the hardware, operational, and scenario information of the newly added agent node, ensuring a high degree of matching between the model fragments and the edge agent node's environment.

[0114] Furthermore, after decrypting and obtaining the target model fragment containers, the N target model fragment containers need to be combined to generate a target model suitable for the current scenario and hardware environment. This process involves parsing container configuration files, matching fragment interfaces, and setting environment variables to enable seamless collaboration between model components. Once the target model is assembled, the edge proxy node is ready to receive data requests from terminal devices. When it receives data to be processed from the terminal device, the edge proxy node processes the data using the target model to obtain the data processing results. For example, in a smart security scenario, the target model might be an optimized video analysis model capable of quickly identifying and classifying abnormal behaviors in videos. After processing, the data processing results are fed back to the terminal device via a secure communication protocol, achieving a closed-loop data acquisition, processing, and result application.

[0115] Through the above steps, not only is model updating secure, but rapid model adaptation and efficient deployment are also achieved. Specifically, the process of receiving and decrypting the encrypted model combines a "device fingerprint + dynamic key" authentication mechanism, effectively preventing the leakage and tampering of model data during transmission and ensuring model security. By combining and applying target model fragment containers, the model can dynamically adapt to the hardware status, operating status, and specific scenarios of the edge agent nodes, which not only improves model performance and accuracy but also reduces resource consumption, achieving a dual improvement in the flexibility and efficiency of model deployment.

[0116] Optionally, in the model deployment method provided in the embodiments of this application, the method further includes: obtaining the running data of the terminal device, and determining whether the deployment of the target model is allowed in the terminal device based on the running data; if the deployment of the target model is allowed in the terminal device, deploying the target model to the terminal device, and processing the data to be processed through the target model in the terminal device.

[0117] For example, adding a new agent node can also deploy a model to the terminal device. First, the system acquires the terminal device's operational data in real time, which can be achieved through the edge agent node or the terminal device's built-in monitoring system. After the operational data is collected, the system performs in-depth analysis, including calculating the device's current resource utilization (such as CPU and memory usage), storage capacity, network connection quality, etc., thereby accurately grasping the real-time status of the terminal device.

[0118] Furthermore, based on the analysis results of the acquired operational data, the system determines whether the terminal device allows the deployment of the target model. If the terminal device allows the deployment of the target model, the system deploys the target model to the terminal device via a secure communication protocol. The deployment method of the model is selected according to the hardware characteristics of the device; it can be containerized deployment or native lightweight deployment. Once the model is deployed, the terminal device can use the target model to process the data to be processed. Thus, by deploying the model on the terminal device, the efficiency of model application and data processing is improved.

[0119] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0120] In this embodiment, Figure 3 This is a schematic diagram of a model deployment system provided according to an embodiment of this application, such as... Figure 3 As shown, the aforementioned model deployment method is executed using an optional model deployment system as the execution entity. This model deployment system includes: a cloud management platform, multiple initial agent nodes, and terminal devices under each initial agent node, wherein:

[0121] The cloud management platform connects to multiple initial agent nodes. When a new agent node is detected among the multiple initial agent nodes, it obtains the running information, scene information, and hardware information of the new agent node. Based on the scene information and hardware information, it processes the initial model in the cloud management platform to obtain the target model and deploys the target model to the new agent node.

[0122] A new agent node is added and connected to the cloud management platform. It is used to receive and deploy the target model. When it receives the data to be processed sent by the terminal device under the new agent node, it processes the data to be processed through the target model, obtains the data processing result, and feeds the data processing result back to the terminal device.

[0123] The terminal devices under the newly added agent node are connected to the newly added agent node and are used to send data to be processed to the newly added agent node and receive the data processing results sent by the newly added agent node.

[0124] It should be noted that the cloud-based management platform is used for model training, dynamic optimization, intelligent scheduling, and global monitoring. This platform handles large-scale data training tasks through a distributed computing framework and dynamically adjusts model deployment strategies based on real-time resource status and operational data feedback from edge agent nodes. The platform has a built-in intelligent scheduling algorithm that can automatically select the optimal model distribution path and deployment scheme based on the hardware configuration, network conditions, and business needs of the edge agent nodes. The global monitoring function displays the real-time operational status, resource utilization, and abnormal events of each edge agent node through a visual dashboard, supporting remote intervention and troubleshooting by administrators.

[0125] Optionally, the cloud-based management platform may include a tiered privacy-preserving update module, which employs a three-tiered federated learning framework: edge-region-cloud. In this framework, edge agent nodes generate update gradients based on local data, which are then encrypted and aggregated by regional aggregation nodes before being uploaded to the cloud for global optimization. No original data is transmitted throughout the process, and a differential privacy mechanism is introduced to protect data privacy. Specifically, after calculating the model gradient on local data, the edge agent nodes add calibration noise to achieve ε-differential privacy protection, and then transmit the encrypted gradients to the regional aggregation nodes. The regional aggregation nodes use homomorphic encryption technology to aggregate gradients within the encrypted domain, preventing intermediate steps from obtaining the original gradient information. After receiving the aggregated gradients, the cloud updates the global model using a federated averaging algorithm and distributes the optimized model parameters to each edge agent node. This module effectively solves the data silo and privacy leakage problems faced by traditional centralized learning, while improving the accuracy and efficiency of model training.

[0126] Furthermore, the edge proxy node is used for model fragment reception, local assembly and deployment, dynamic resource management, and comprehensive security protection. This proxy adopts a lightweight design, supporting efficient operation on resource-constrained edge proxy devices. The model fragment reception function retrieves model fragments from the cloud via a secure channel and assembles and verifies them locally. The local assembly and deployment function adaptively adjusts the model structure and parameters based on device hardware characteristics and business scenarios to achieve optimal deployment. The dynamic resource management function monitors the usage of hardware resources such as CPU, GPU, and memory in real time, dynamically allocating computing resources according to load conditions to maintain stable model inference performance.

[0127] Optionally, the cloud management platform may also include a heterogeneous node intelligent scheduling module, which enables dynamic node discovery, multi-dimensional computing power assessment, and intelligent deployment planning. By monitoring the computing power, storage capacity, and network status of edge agent nodes in real time, it automatically selects the optimal deployment strategy.

[0128] Optionally, the cloud management platform may also include a hierarchical privacy protection update module. This module adopts a three-level federated learning framework of "edge-region-cloud". Edge agent nodes generate update gradients based on local data training. After being encrypted and aggregated by regional aggregation nodes, the gradients are uploaded to the cloud for global optimization. No original data is transmitted throughout the process. At the same time, a differential privacy mechanism is introduced to protect data privacy.

[0129] Optionally, the edge proxy node may include a model fragment receiving and verification module. This module employs a device fingerprint + dynamic key multi-factor authentication mechanism, combined with the SM4 symmetric encryption algorithm to achieve two-way authentication. The Merkle hash tree algorithm is used to verify the integrity of the model fragment, ensuring no data tampering or loss occurs during transmission. Specifically, the device fingerprint generates a unique identifier by collecting the hardware characteristics of the edge proxy node (such as CPU serial number, MAC address, hard drive ID, etc.), which is combined with a dynamically generated timestamp and a random challenge value to form a multi-factor authentication credential. Before model fragment transmission, both parties exchange authentication information and verify identity legitimacy using the SM4 symmetric encryption algorithm. Upon receiving the model fragment, the system constructs a Merkle hash tree and verifies fragment integrity by comparing the root hash value. Any modification or loss of a fragment will result in a hash value mismatch, triggering a retransmission mechanism. This multi-layered verification mechanism effectively prevents man-in-the-middle attacks and model tampering risks.

[0130] Optionally, the edge proxy node may include a three-dimensional security protection module, which constructs a three-layer security protection system encompassing data, model, and node. This module uses the AES-256 algorithm to encrypt locally stored data, implements dynamic communication encryption based on the QUIC protocol combined with the national cryptographic SM2 algorithm, and constructs behavioral baseline protection through eBPF technology to achieve fine-grained access control based on attributes. At the data level, all sensitive data is encrypted across the entire disk using the AES-256 algorithm, with keys managed through a hardware security module to prevent data leakage. At the model level, model intellectual property is protected through model watermarking and white-box encryption technologies to prevent reverse engineering and model theft. At the node level, eBPF technology monitors system calls and network behavior in real time, establishes a baseline for normal behavior, and intercepts and alerts on abnormal behavior. Attribute-based access control dynamically adjusts access permissions based on multi-dimensional attributes such as user role, device status, and network environment, achieving fine-grained permission management. These three layers of protection work together to form a defense-in-depth system, significantly improving the overall security of the system.

[0131] Optionally, the edge agent node also includes a model fragment receiving and verification module. This module adopts a multi-factor authentication mechanism of "device fingerprint + dynamic key" and combines it with the SM4 symmetric encryption algorithm to achieve two-way authentication. It verifies the integrity of the model fragments through the Merkle hash tree algorithm, so that there is no data tampering or loss during transmission.

[0132] Furthermore, the edge proxy node can be configured with a terminal execution engine for data interaction with terminal devices and for performing data acquisition, model inference, and result feedback operations. This engine adopts a cross-platform design, supporting multiple hardware architectures such as ARM and x86, and can adapt to the performance constraints of different terminal devices. The data acquisition function obtains real-time data through multiple sensor interfaces and performs preprocessing and feature extraction.

[0133] Optionally, the terminal execution engine includes an intelligent data acquisition and preprocessing module, which realizes intelligent format conversion, scene-adaptive preprocessing, and lightweight feature extraction, supports unified processing and feature extraction of multiple data sources, and reduces the burden of subsequent inference computation.

[0134] Optionally, the terminal execution engine includes a heterogeneous inference acceleration module, which provides hardware adaptive acceleration, dynamic batch processing optimization, and intelligent caching mechanisms to optimize the allocation of inference tasks for different computing units such as CPU, GPU, and NPU, maximizing the utilization of heterogeneous computing resources.

[0135] Optionally, the terminal execution engine includes a result intelligent processing and feedback module, which includes result structured parsing, confidence level filtering, local closed-loop storage, and multi-level anomaly reporting functions. It performs intelligent analysis and processing of inference results and supports local decision-making and hierarchical reporting of anomalies.

[0136] Optionally, the terminal execution engine includes a multi-protocol adaptive adaptation module, which supports automatic protocol conversion, standardized interface abstraction, and pluggable protocol extension, enabling seamless integration with various industrial protocols and IoT standards, thereby improving system compatibility and scalability.

[0137] This application also provides a model deployment apparatus. It should be noted that the model deployment apparatus of this application can be used to execute the model deployment method provided in this application. The model deployment apparatus provided in this application will be described below.

[0138] Figure 4 This is a schematic diagram of an optional model deployment apparatus provided according to an embodiment of this application. For example... Figure 4 As shown, the device includes:

[0139] The acquisition unit 41 is used to acquire the running information, scene information and hardware information of the newly added agent node when the newly added agent node is detected, and to perform lightweight processing on the initial model in the cloud management platform based on the scene information and hardware information to obtain a candidate model.

[0140] Unit 42 is used to perform fragmented container encapsulation operations on candidate models to obtain M initial model fragment containers, and to determine the model deployment method of the newly added agent node based on hardware information and runtime information.

[0141] Unit 43 is selected to select target model fragment containers from M initial model fragment containers according to the model deployment method, resulting in N target model fragment containers, where M and N are both positive integers, and M is greater than or equal to N.

[0142] The first deployment unit 44 is used to encrypt and deploy N target model fragment containers to the newly added agent node. The newly added agent node combines the N target model fragment containers to obtain the target model, and processes the data to be processed sent from the terminal device to the newly added agent node through the target model.

[0143] Optionally, in the model deployment apparatus provided in this application embodiment, the acquisition unit 41 includes: a first determining module, used to determine the resource requirements and constraints of deploying the initial model on the newly added agent node based on scene information and hardware information; a second determining module, used to determine the preset operation method and operation coefficients for lightweighting the initial model based on the resource requirements and constraints; and a processing module, used to perform lightweighting processing on the initial model based on the preset operation method and operation coefficients to obtain a candidate model.

[0144] Optionally, in the model deployment apparatus provided in this application embodiment, the determining unit 42 includes: a splitting module, used to split the candidate model according to the functional type to obtain M initial model fragments, wherein the initial model fragments include at least one of the following: basic type fragments, scene type fragments, and hardware optimization type fragments; a generation module, used to generate the configuration interface of each initial model fragment, and update the code of each initial model fragment according to the configuration interface through code refactoring operation to obtain M target model fragments; and an encapsulation module, used to obtain the environment configuration information and dependencies of each target model fragment respectively, and encapsulate each target model fragment and the corresponding environment configuration information and dependencies into an initial model fragment container to obtain M initial model fragment containers.

[0145] Optionally, in the model deployment apparatus provided in this application embodiment, the determining unit 42 includes: a third determining module, used to determine the architecture type of the newly added agent node based on hardware information, and to determine the running status of the newly added agent node based on running information; a fourth determining module, used to determine the remaining running resource information of the newly added agent node based on the architecture type and running status; a judging module, used to obtain the running resource requirement information of the candidate model, and to judge whether the newly added agent node is allowed to deploy the candidate model based on the remaining running resource information and the running resource requirement information; a fifth determining module, used to determine the model deployment method as container assembly deployment mode if the newly added agent node is allowed to deploy the candidate model; and a sixth determining module, used to determine the model deployment method as lightweight deployment mode if the newly added agent node is not allowed to deploy the candidate model.

[0146] Optionally, in the model deployment apparatus provided in this application embodiment, the selection unit 43 includes: a seventh determining module, used to determine M initial model fragment containers as target model fragment containers when the model deployment mode is container assembly deployment mode; and an eighth determining module, used to obtain basic type fragment containers and scene type fragment containers from the M initial model fragment containers when the model deployment mode is lightweight deployment mode, and to determine the basic type fragment containers and scene type fragment containers as target model fragment containers.

[0147] Optionally, in the model deployment apparatus provided in this application embodiment, the first deployment unit 44 includes: a first acquisition module, used to construct a hash tree based on N target model fragment containers and acquire the root hash value; a second acquisition module, used to acquire the device fingerprint information of the newly added agent node and use the device fingerprint information to encrypt the N target model fragment containers to obtain an encrypted model; and an encryption module, used to encrypt the encrypted model and the root hash value using a preset encryption algorithm to obtain encrypted information and send the encrypted information to the newly added agent node.

[0148] Optionally, in the model deployment apparatus provided in the embodiments of this application, the apparatus further includes: a training unit, used to train an initial model based on the new scene information to obtain an updated model when the scene information of the newly added agent node is detected to have changed; and a second deployment unit, used to obtain a new model fragment container associated with the new scene information from the updated model and deploy the new model fragment container to the new agent node.

[0149] Figure 5 This is a schematic diagram of another optional model deployment apparatus provided according to an embodiment of this application. For example... Figure 5 As shown, this device is applied to newly added agent nodes and includes:

[0150] The decryption unit 51 is used to receive the encrypted model sent by the cloud management platform and decrypt the encrypted model to obtain N target model fragment containers. The N target model fragment containers are obtained by the cloud management platform after processing the initial model based on the running information, scene information and hardware information of the newly added agent node.

[0151] Combination unit 52 is used to combine N target model fragment containers to obtain the target model.

[0152] The processing unit 53 is used to process the data to be processed through the target model when it receives the data to be processed sent by the terminal device, obtain the data processing result, and feed the data processing result back to the terminal device.

[0153] Optionally, in the model deployment apparatus provided in the embodiments of this application, the apparatus further includes: a judgment unit, used to obtain the operating data of the terminal device and determine whether the deployment of the target model is allowed in the terminal device based on the operating data; and a third deployment unit, used to deploy the target model to the terminal device if the deployment of the target model is allowed in the terminal device, and process the data to be processed through the target model in the terminal device.

[0154] The aforementioned model deployment device includes a processor and a memory. The aforementioned acquisition unit 41, determination unit 42, selection unit 43, first deployment unit 44, decryption unit 51, combination unit 52, processing unit 53, etc., are all stored in the memory as program units. The processor executes the aforementioned program units stored in the memory to realize the corresponding functions.

[0155] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured. By adjusting kernel parameters, the technical problem of excessive resource consumption in terminal devices caused by the full-scale model deployment method in related technologies, which affects the normal operation of the terminal devices, can be solved.

[0156] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0157] This invention provides a computer-readable storage medium storing a program that, when executed by a processor, implements the model deployment method.

[0158] This invention provides a processor for running a program, wherein the program executes the model deployment method during runtime.

[0159] Figure 6 This is a schematic diagram of an electronic device provided according to an embodiment of this application, such as... Figure 6 As shown, this embodiment of the invention provides an electronic device 60, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the model deployment method described above. The device in this document can be a server, PC, PAD, mobile phone, etc.

[0160] This application also provides a computer program product that, when executed on a data processing device, is adapted to perform the steps of initializing the above-described model deployment method.

[0161] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0163] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0164] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0165] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0166] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0167] Computer-readable media include both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0168] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0169] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A model deployment method, characterized in that, Applications in cloud management platforms include: Upon detecting a new agent node, the system acquires the running information, scenario information, and hardware information of the new agent node, and performs lightweight processing on the initial model in the cloud management platform based on the scenario information and the hardware information to obtain a candidate model. The candidate models are encapsulated in fragmented containers to obtain M initial model fragment containers, and the model deployment method of the newly added agent node is determined based on the hardware information and the running information. Based on the model deployment method, target model fragment containers are selected from the M initial model fragment containers to obtain N target model fragment containers, where M and N are both positive integers, and M is greater than or equal to N; The N target model fragment containers are encrypted and deployed to the newly added proxy node. The newly added proxy node combines the N target model fragment containers to obtain a target model, and processes the data to be processed sent from the terminal device to the newly added proxy node through the target model.

2. The method according to claim 1, characterized in that, Based on the scenario information and hardware information, the initial model in the cloud management platform is lightweighted to obtain candidate models, including: Based on the scenario information and the hardware information, determine the resource requirements and constraints for deploying the initial model on the newly added agent node; Based on the resource requirements and constraints, determine the preset operation method and operation coefficients for lightweighting the initial model; The initial model is lightweighted according to the preset operation method and the operation coefficients to obtain the candidate model.

3. The method according to claim 1, characterized in that, The candidate models are encapsulated using a fragmented container operation to obtain M initial model fragment containers, including: The candidate model is split according to the function type to obtain M initial model fragments, wherein the initial model fragments include at least one of the following: basic type fragments, scene type fragments, and hardware optimization type fragments; The configuration interface for each initial model fragment is generated, and the code of each initial model fragment is updated according to the configuration interface through code refactoring to obtain M target model fragments; The environment configuration information and dependencies of each target model fragment are obtained respectively, and each target model fragment and its corresponding environment configuration information and dependencies are encapsulated into an initial model fragment container to obtain the M initial model fragment containers.

4. The method according to claim 1, characterized in that, The model deployment method for the newly added agent node is determined based on the hardware information and the operational information, including: The architecture type of the newly added agent node is determined based on the hardware information, and the running status of the newly added agent node is determined based on the running information; The remaining runtime resource information of the newly added agent node is determined based on the architecture type and the running status; Obtain the runtime resource requirements information of the candidate model, and determine whether to allow the newly added agent node to deploy the candidate model based on the remaining runtime resource information and the runtime resource requirements information; If the newly added agent node is allowed to deploy the candidate model, the model deployment method is determined to be container assembly deployment mode; If the newly added agent node is not allowed to deploy the candidate model, the model deployment method will be determined as lightweight deployment mode.

5. The method according to claim 4, characterized in that, Selecting a target model fragment container from the M initial model fragment containers according to the model deployment method includes: When the model deployment method is the container assembly deployment mode, the M initial model fragment containers are determined as the target model fragment containers; When the model deployment method is the lightweight deployment mode, a basic type fragment container and a scene type fragment container are obtained from the M initial model fragment containers, and the basic type fragment container and the scene type fragment container are determined as the target model fragment container.

6. The method according to claim 1, characterized in that, Deploying the N target model fragment containers to the newly added agent node in an encrypted manner includes: Construct a hash tree based on the N target model fragment containers and obtain the root hash value; Obtain the device fingerprint information of the newly added agent node, and use the device fingerprint information to encrypt the N target model fragment containers to obtain the encrypted model; The encryption model and the root hash value are encrypted using a preset encryption algorithm to obtain encrypted information, which is then sent to the newly added proxy node.

7. The method according to claim 1, characterized in that, The method further includes: If the scene information of the newly added agent node changes, the initial model is trained based on the new scene information to obtain an updated model. Obtain a new model fragment container associated with the new scene information from the updated model, and deploy the new model fragment container to the new agent node.

8. A model deployment method, characterized in that, Applied to newly added proxy nodes, including: The system receives an encrypted model sent by the cloud management platform and decrypts the encrypted model to obtain N target model fragment containers. The N target model fragment containers are obtained by the cloud management platform after processing the initial model based on the running information, scene information and hardware information of the newly added agent node. The N target model fragment containers are combined to obtain the target model; Upon receiving data to be processed from a terminal device, the target model is used to process the data to obtain a data processing result, which is then fed back to the terminal device.

9. The method according to claim 8, characterized in that, The method further includes: Obtain the operating data of the terminal device, and determine whether the target model is allowed to be deployed on the terminal device based on the operating data; If the target model is allowed to be deployed in the terminal device, the target model is deployed to the terminal device, and the data to be processed is processed through the target model in the terminal device.

10. A model deployment system, characterized in that, This includes a cloud management platform, multiple initial agent nodes, and terminal devices under each initial agent node, among which: A cloud management platform is connected to multiple initial agent nodes. When a new agent node is detected among the multiple initial agent nodes, the platform obtains the running information, scene information, and hardware information of the new agent node, processes the initial model in the cloud management platform according to the scene information and the hardware information to obtain a target model, and deploys the target model to the new agent node. The newly added agent node is connected to the cloud management platform and is used to receive and deploy the target model. When it receives the data to be processed sent by the terminal device under the newly added agent node, it processes the data to be processed through the target model to obtain the data processing result and feeds the data processing result back to the terminal device. The terminal device under the newly added agent node is connected to the newly added agent node and is used to send data to be processed to the newly added agent node and receive the data processing results sent by the newly added agent node.

Images