Cloud native container storage access policy optimization method and system based on federated learning
By introducing kernel sandbox modules and federated learning in cloud-native containers, the storage access policy is optimized, and the challenges of customizing and optimizing container storage access policy are solved, performance and security are improved, and efficient storage management is provided for cloud-native environments.
Patent Information
- Application Number
- CN202411244168.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-06
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-06
AI Technical Summary
Existing cloud-native container storage access policies have challenges in terms of trustworthiness, security, flexibility, and efficiency, especially as different containers share the same operating system kernel and drivers, making storage access policies difficult to customize and optimize.
Introduce the kernel sandbox module, compile Wasm into native binary code and run it in a kernel-state secure sandbox. Through federated learning and AI model training, optimize storage access policies, realize customization and dynamic management of storage access policies, and use Memory protection keys and RDMA interfaces improve performance and security.
It realizes dynamic management and execution of container storage policies, improves overall performance and security, solves the technical challenges of Wasm applying to kernel-state modules, and provides an efficient storage management solution for cloud-native environments.
Smart Images

Figure CN118761098B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of cloud-native container storage access technology, and specifically relates to a cloud-native container storage access strategy optimization method and system based on federated learning. Background Art
[0002] With the rapid development of cloud computing technology, enterprises have been migrating computing tasks to the cloud in order to reduce costs. Currently, enterprises can choose bare metal, virtual machines or containers for migration. Among them, containers are widely favored due to their lightweight, easy deployment and good elasticity. Therefore, most cloud computing tasks are now deployed in the form of containers. However, existing containers only provide a user account-level virtual environment on the operating system. Different containers still share the same operating system kernel when running on the same platform, which means that they also share the same set of drivers and disk provisioning policies. Existing technologies face challenges in terms of credibility, security, flexibility and efficiency in customizing storage access policies. Summary of the invention
[0003] The core of the present invention is to introduce the concept of a secure kernel module, compile Wasm into native binary and run it in a secure sandbox in kernel state, so as to support the customization and dynamic optimization of storage access policies. The technical solution is as follows:
[0004] A method for optimizing storage access policies for cloud native containers based on federated learning sets up a kernel sandbox and places its storage in kernel state, including the following steps:
[0005] S1. The Wasm compiler in the container compiles the high-level language code into wasm code and hands the wasm code to the trusted wasm compiler module;
[0006] S2. The trusted wasm compiler module compiles the wasm code into binary code and passes it to the sandbox for running. The kernel sandbox supports the customization and dynamic optimization of storage access policies. The kernel sandbox passes the running binary code data to the AI model, and uses federated learning to collaborate with various containers to train the AI model, so that the AI model optimizes the storage access policy of the kernel sandbox.
[0007] S3. The container compiles the code generated after AI model optimization into Wasm code, repeats steps S1-S2, and uses the trusted Wasm compiler to compile the Wasm code into native binary code for kernel sandbox execution, replacing the previous storage access policy.
[0008] Preferably, in step S2, the steps of federated learning and AI model training are as follows:
[0009] S21. Each container uses the local model of federated learning to train the AI model. After each node completes local training, it sends the local model parameters to the aggregator instead of sharing the original data.
[0010] S22. After receiving the model parameters from multiple nodes, the central server aggregates these parameters and performs weighted aggregation according to the weight and quality of each container model update;
[0011] S23. After the central server completes the aggregation of model parameters, the global model will be updated; the central server sends the global model parameters to each participating distributed node, so that each node can continue the next round of local training based on the updated global model.
[0012] Preferably, the container gains direct access to the raw hard disk via the kernel sandbox:
[0013] The kernel sandbox provides a secure interface for hardware devices, so that storage operations no longer rely on the abstraction layer of the volume manager or file system. The kernel sandbox directly converts the container's storage requests into low-level operations on the hard disk. Through the isolation and permission management of the kernel sandbox, it ensures that even if the container performs customized storage operations on the hardware, it will not threaten the security of the entire system.
[0014] Preferably, the kernel sandbox exposes the RDMA interface directly to the container through layer vertical penetration technology:
[0015] When a container makes a storage request, the kernel sandbox parses the request and maps it into an RDMA operation;
[0016] An RDMA protocol stack is implemented inside the kernel sandbox, which separates data transmission operations from the network layer, allowing requests to be directly passed to remote storage devices;
[0017] The container directly accesses the storage device of the remote node through the kernel sandbox and the RDMA protocol stack. Through the pre-established RDMA connection, data transmission across nodes does not need to go through the intermediate network layer or storage middleware, achieving low-latency, high-bandwidth storage access across nodes.
[0018] Preferably, a memory protection key MPK is set to reduce memory check overhead in kernel state, and by configuring the MPK and memory page keys, permissions for frequently accessed memory pages are set to allowed, reducing the need for permission checks.
[0019] Preferably, the steps for handling abnormal nodes in federated learning are as follows:
[0020] Step 1. Assume a set of access mode data sets , where each Represents the memory access pattern feature vector of an application; for each isolated tree, a feature is randomly selected and random split points , the dataset is divided into and split point It is divided into two parts and the process is repeated until each data point is isolated or the depth of the tree reaches the preset maximum value;
[0021] Step 2. For each application sample , calculate its path length in the isolation forest, anomaly score It is given by the following formula:
[0022] ;
[0023] in It is a data node Average path length in isolated trees; It is a sample Through the feature selection method in the isolation tree construction stage, the node where the sample is isolated is reached from the root node;
[0024] ;
[0025] Where e is the sample The number of edges from the root node to the leaf node of the tree, that is, the number of splits; T.size represents the sample The number of samples at the same leaf node, C(T.size) is a correction value, which means the average path length of a binary tree constructed by T.size samples;
[0026] n is the size of the dataset, is the normalization coefficient, which is approximately:
[0027] ;
[0028] in is the nth harmonic number:
[0029] ;
[0030] Since different datasets have different sizes n, the expected path length will be different. In order to unify the standards, the path length is standardized;
[0031] The harmonic number is approximately:
[0032] ;
[0033] in is the Euler–Mascheroni constant;
[0034] Step 3. If If it is close to 1, it means that the sample may be abnormal, that is, a pathological application; Close to 0, it means the sample is normal;
[0035] Setting Thresholds , according to the anomaly score Separate sick applications from healthy ones, if , then the application Mark applications as pathological to isolate them from the global model.
[0036] Preferably, the optimization steps of the storage access policy are as follows:
[0037] Step 1. Train a small model using a local dataset (also called local model):
[0038] ;
[0039] in is the learning rate, is the loss function, and the network parameters are defined as , Represents a certain access mode feature vector of the container;
[0040] After completing local training, the small model Network parameters and gradient Transmit to the central server;
[0041] Step 2. Collect the gradients and parameters of the small models from multiple storage servers and combine the collected small models Information, training a more complex large model ;
[0042] Use small models As a teacher model, the big model As a student model, small model The resulting prediction is , The prediction is ;
[0043] Step 3. Define the knowledge distillation loss function , which is optimized by minimizing the difference between the output distributions of the large and small models:
[0044] ;
[0045] In the formula, The Kullback-Leibler divergence is calculated by minimizing the distillation loss to adjust the parameters of the large model network. :
[0046] ;
[0047] is the learning rate;
[0048] Step 4. Prune the large model on the central server to generate a small model suitable for the storage server, and send the pruned model to each storage server;
[0049] The model parameters after pruning are , define the pruning function :
[0050] ;
[0051] Step 5. Use local dataset Fine-tune the small model that was sent:
[0052] ;
[0053] is the learning rate.
[0054] Preferably, an adaptive module is introduced , is the parameter of the adaptive module, which is related to the small model Working together; Dynamically adjust the model's output to accommodate different storage access loads and patterns.
[0055] Fine-tune the adaptive module based on real-time feedback data accessed from storage, using the loss function To reflect the quality of adaptive adjustment;
[0056] Accumulate a collection of significantly changed data points on a local storage server , use this data for incremental training without retraining the entire model:
[0057] ;
[0058] The updated network parameters of the small model, The network parameters of the small model before updating, learning rate, Adaptive module learning gradient, are the parameters of the small model, that is, the parameters of the local model; is the input data on the local storage server, i.e., the feature vector of the storage access policy after preprocessing; is the output corresponding to the input data, that is, the desired optimization strategy;
[0059] Parameter fusion: Combine the updated parameters of incremental learning with the parameters of the adaptive module to generate a new combined model:
[0060] ;
[0061] in is a coefficient that controls the weight, which is used to adjust the contribution of the main model and the adaptive module in the final combined model;
[0062] Restructure storage access policies using the new composition model:
[0063] .
[0064] Preferably, the update frequency of the local model is controlled to ensure that the local model is updated only when necessary, and a rollback mechanism is introduced to ensure rapid recovery when the update fails; before each model update, the existing model parameters are backed up. If the performance of the updated model is not as expected, or the system becomes unstable, the backed-up model parameters are used for rollback;
[0065] A double buffering mechanism is implemented in the container. During an update, the updated local model is loaded into the backup buffer, and the model before the update is kept working in the main buffer. When the updated local model is successfully verified, the main buffer and the backup buffer are switched.
[0066] A system for optimizing cloud-native container storage access policies based on federated learning, including a trusted Wasm compiler module, a kernel sandbox module, a federated learning and AI large-scale model generation module, and a permission management and quota module;
[0067] Trusted Wasm compiler module: The compiler in the container is responsible for compiling high-level language code into intermediate code in Wasm format, and passing the Wasm code generated by the container to the trusted Wasm compiler module in the root privilege process in user mode; the trusted Wasm compiler first uses LLVM to parse the Wasm bytecode into an intermediate representation using LLVM IR, optimizes using LLVM's optimization channel, generates efficient native binary code through the LLVM backend, and runs the binary code in the kernel sandbox;
[0068] Kernel sandbox module: The kernel sandbox runs in a secure execution environment in kernel state and is used to securely execute native binary code generated by the Wasm compiler. The kernel sandbox is located above the driver layer and can be connected to components such as volume managers and file systems. It supports direct exposure of bare hard disks or RDMA interfaces to containers, solving the limitations of storage abstraction in existing technologies. The Wasm security sandbox supports dynamic optimization of storage access policies;
[0069] Federated learning and AI large-scale model generation module: Federated learning is used to collaborate with various containers to train AI models, improving the generalization ability and privacy protection of the model without sharing data. Through local training, parameter updates, and global aggregation, federated learning can dynamically optimize storage access strategies to maximize performance.
[0070] Permission management and quota module: Assign different storage access rights to different kernel sandbox modules: Through the refined management of the permission model, dynamic adjustment mechanism, multi-dimensional quota model and standardized interface design, the system can provide personalized storage access rights and resource quota management for different application scenarios, ensuring fair allocation and safe use of resources.
[0071] Compared with the prior art, the present invention has the following beneficial effects:
[0072] This invention uses kernel sandbox technology to achieve dynamic management and execution of container storage policies, and implements isolation and optimization across containers and storage devices through Wasm modules. Combined with federated learning, the system can perform real-time storage access policy optimization through AI models on different nodes to improve overall performance and security. This system ensures the security of the kernel-state execution environment through the kernel sandbox, and combines compilation optimization, LLVM translation, memory protection and other technologies to solve the technical challenges of applying Wasm to kernel-state modules, thereby providing an efficient storage management solution for cloud-native environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0073] Figure 1 This is a schematic diagram of the structure of this application.
[0074] Figure 2 This is a schematic diagram of the application process. DETAILED DESCRIPTION
[0075] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.
[0076] The present invention proposes a method and system for optimizing storage access policies for cloud-native containers based on federated learning. The solution solves the above problems by introducing a kernel-mode security sandbox and a trusted Wasm compiler in the storage management path. Wasm (WebAssembly) is the next generation technology of eBPF, which supports Turing-complete applications and has efficient execution capabilities. In the present invention, the container provides a Wasm code implementation of the storage management policy, and the trusted Wasm compiler compiles the Wasm code into native binary and executes it in a secure sandbox, thereby achieving full customization of the container's efficient, secure and flexible storage management layer.
[0077] The invention mainly includes a trusted Wasm compiler module, a (kernel sandbox) kernel-mode security sandbox module, a federated learning and AI large-scale model generation module, and a permission management and quota module;
[0078] Trusted Wasm compiler module: The compiler in the container is responsible for compiling high-level language code into intermediate code in Wasm format, and passing the Wasm code generated by the container to the trusted Wasm compiler module in the root privilege process in user mode; the trusted Wasm compiler first uses LLVM to parse the Wasm bytecode into an intermediate representation using LLVM IR, optimizes using LLVM's optimization channel, generates efficient native binary code through the LLVM backend, and runs the binary code in the kernel sandbox;
[0079] Kernel sandbox module: The kernel sandbox runs in a secure execution environment in kernel state and is used to securely execute native binary code generated by the Wasm compiler. The kernel sandbox is located above the driver layer and supports direct exposure of bare hard disks or RDMA interfaces to containers. The kernel sandbox supports dynamic optimization of storage access policies.
[0080] Federated learning and AI large-scale model generation module: Use federated learning to collaborate with various containers to train AI models, improving the generalization ability and privacy protection of the model without sharing data; through local training, parameter updates and global aggregation, federated learning can dynamically optimize storage access strategies to maximize performance;
[0081] Permission management and quota module: Assign different storage access rights to different kernel sandbox modules: Through the refined management of the permission model, dynamic adjustment mechanism, multi-dimensional quota model and standardized interface design, the system can provide personalized storage access rights and resource quota management for different application scenarios, ensuring fair allocation and safe use of resources.
[0082] 1. Trusted Wasm compiler module:
[0083] Application developers in the container write high-level language codes, such as C, C++, Rust, etc., according to application requirements. This code contains storage access policy modules or other Wasm program logic. The compiler in the container (ordinary user-mode process) is responsible for compiling these high-level language codes into intermediate codes in Wasm format, and passing the Wasm code generated by the container to the trusted Wasm compiler module in the root privilege process in user mode.
[0084] The trusted Wasm compiler first uses LLVM to parse the Wasm bytecode into an intermediate representation using LLVM IR, and uses LLVM's optimization channel for optimization, including dead code elimination, loop optimization, instruction combination, etc. It generates efficient native binary code through the LLVM backend and runs the binary code in the kernel sandbox.
[0085] (1) Optimize the overhead caused by memory safety checks:
[0086] In order to ensure the safety of running in the kernel sandbox, it is necessary to introduce memory security check. In order to improve the performance of running in the kernel state, the present invention adopts MPK, address modulo division, and truncation methods to reduce the overhead of memory check.
[0087] Memory Protection Keys (MPK) is a modern processor feature that provides an efficient memory protection mechanism. It can be used to reduce memory checking overhead in kernel mode, especially when we need to handle a large number of memory regions to protect.
[0088] When the system starts or when permissions need to be set, the value of the PKRU register is set through a system call or by directly manipulating the register.
[0089] Assign a key to each memory region and update its permissions when needed.
[0090] Set the corresponding key for each memory page in the kernel and configure the permissions.
[0091] The address is aligned to the beginning of the memory page by address modulo division, which aligns the address to a 4KB boundary. This makes memory access always occur in aligned memory pages, simplifying the bounds checking of memory accesses. The address is ensured to be within a valid memory area by calculating the modulo value of the memory address.
[0092] Address truncation is a method of truncating addresses to fixed boundaries, usually memory page boundaries. For example, the low-order part of the address is truncated, leaving only the high-order part, thereby restricting access to aligned memory regions and reducing boundary checks by ensuring that all addresses fall within a predefined range.
[0093] Use MPK to reduce the frequency of context switching. Traditional memory protection mechanisms may require frequent context switching to check permissions, while MPK allows permissions to be checked directly at the hardware level, reducing the overhead of context switching.
[0094] Optimize memory access by configuring MPK and memory page keys to set permissions for frequently accessed memory pages to allow, reducing the need for permission checks.
[0095] (2) Generate Freestanding Module:
[0096] The kernel sandbox module runs in kernel mode, and the Wasm code in user mode will rely on a large number of standard libraries. However, these libraries cannot be used directly in kernel mode because the kernel and user mode operating environments are different. To address this technical difficulty, the Wasm module must be compiled into Freestanding mode, that is, an independently running module that does not rely on standard libraries.
[0097] Implement a script that automatically analyzes dependencies to help identify libraries or functions that need to be packaged into Wasm modules in order to generate Freestanding modules. Use nm and objdump tools to automatically analyze dependencies and generate a package list:
[0098] Use nm to list undefined symbols in the Wasm module and objdump to see the required dynamic libraries.
[0099] Based on the dynamic library name, use ldconfig to find the path of the dynamic library in the system.
[0100] Copy the found libraries to the target directory, ready to be packaged into the Freestanding module.
[0101] After analyzing the dependencies, external libraries are packaged together with Wasm code into the Freestanding module. Freestanding binaries are generated using the Clang compiler, and the -ffreestanding option is used to compile C / C++ code to avoid linking standard libraries. We combine these libraries with the Wasm module through static linking. In this way, the generated Wasm file contains all statically linked dependent libraries and is an independent Freestanding module.
[0102] The Freestanding module generated above is converted into native binary code that can be run by the kernel through the LLVM backend. The generated native binary is linked into the kernel module, and the kernel module is loaded into the system for execution.
[0103] 2. Kernel Sandbox Module:
[0104] The kernel sandbox module runs in a secure execution environment in kernel state and is used to securely execute native binary code generated by the Wasm compiler. The advantage of using native binary instead of interpreted execution is that it greatly reduces execution overhead, which is superior to the default interpreted execution mode of epbf. The kernel sandbox is located above the driver layer and can be connected to components such as volume managers and file systems. It supports direct exposure of bare hard disks or RDMA interfaces to containers, solving the limitations of storage abstraction in existing technologies. The kernel sandbox supports dynamic optimization of storage access policies.
[0105] (1) Hierarchical vertical penetration technology:
[0106] In order to achieve efficient operation of hardware storage resources, the present invention adopts hierarchical vertical penetration technology, which allows the kernel sandbox to bypass the storage abstraction layer and directly operate the bare hard disk or directly access the remote storage device through RDMA and other methods.
[0107] The container can obtain direct access to the bare hard disk through the kernel sandbox module. We provide a secure interface for hardware devices in the kernel sandbox, so that storage operations no longer rely on the abstract layer of the volume manager or file system. The kernel sandbox module directly converts the container's storage request into low-level operations on the hard disk, such as reading and writing sectors or adjusting block sizes. The isolation and permission management of the kernel sandbox ensure that even if the container performs customized storage operations on the hardware, it will not threaten the security of the entire system. The kernel sandbox will execute the container's storage policy code, while preventing unauthorized hardware operations to ensure the stability and security of the system.
[0108] RDMA (Remote Direct Memory Access) allows data to be read or written directly from the memory of a machine without CPU intervention. This technology greatly optimizes cross-node data transmission by providing low-latency, high-throughput remote storage access, and is particularly suitable for use in containerized distributed storage systems. In the present invention, RDMA is combined with the kernel sandbox to achieve efficient and secure remote storage access.
[0109] The kernel sandbox provides a secure execution environment for containers, exposing the RDMA interface directly to containers through layer vertical penetration technology. RDMA transmits data between different nodes in a zero-copy manner, and the sandbox's management of the RDMA interface ensures that these operations are performed without compromising the overall security of the system.
[0110] When a container makes a storage request, the kernel sandbox parses the request and maps it into an RDMA operation.
[0111] An RDMA protocol stack is implemented inside the sandbox, which separates data transmission operations from the network layer so that requests can be directly passed to remote storage devices.
[0112] Containers can directly access the storage devices of remote nodes through the kernel sandbox and RDMA protocol stack. Through the pre-established RDMA connection, data transmission across nodes does not need to go through the intermediate network layer or storage middleware, achieving low-latency, high-bandwidth storage access across nodes.
[0113] (2) Customization and dynamic optimization of storage access policies:
[0114] The kernel sandbox module allows for customized storage access policies for containers. The customized storage access policies are compiled into native binary code through the Wasm compiler and run in the kernel sandbox module. It also allows for dynamic replacement and real-time tuning of storage access policies.
[0115] Define the storage access policy steps and determine the policy scope:
[0116] Read and write optimization: including how to efficiently process read and write requests, such as using asynchronous IO or batch reading and writing.
[0117] Cache strategy: includes cache strategies such as LRU, LFU, etc. to improve data access speed.
[0118] IO Scheduling: Design how to schedule IO operations to ensure efficient resource usage, such as FIFO, priority scheduling, etc.
[0119] Design strategy modules. Modular design splits different strategy functions into independent modules.
[0120] Read and write operation module: responsible for optimizing read and write performance.
[0121] Cache management module: manages cache content and policies.
[0122] Resource scheduling module: manages the scheduling and resource allocation of IO requests.
[0123] Choose a programming language suitable for Wasm (such as Rust, C++) to write policy code. The code written in accordance with Wasm requirements must comply with Wasm's memory model and execution model, and avoid using unsupported system calls or instructions.
[0124] Use the trusted Wasm compiler to compile the storage access policy code into native binary code, load the compiled native binary code in the kernel sandbox module and execute it.
[0125] The kernel sandbox implements dynamic replacement of storage access policies. Before loading new code, the kernel sandbox execution environment is configured to ensure that all dependent resources and permissions are properly configured. An independent memory area is allocated for the newly loaded binary code to ensure that it is isolated from the existing code. For each binary code loaded, a unique version number is assigned. The old policy code will not be uninstalled immediately before replacement, but will be retained for a certain period of time as a backup version. While executing the new code, the old version is retained for rollback.
[0126] The replacement operation is divided into multiple stages, including preloading, resource switching, state synchronization, etc., to gradually replace the old policy with the new policy to avoid instability caused by sudden switching. When the new code is ready to execute, the relevant resources (such as file handles, network connections, etc.) are switched from the old code to the new code to ensure seamless takeover of the new policy. The state of the old policy (such as cached data, runtime variables, etc.) can be smoothly migrated to the new policy so that it can continue to execute from the point where the old policy was interrupted.
[0127] The kernel sandbox module passes the storage access data in the running environment, including storage access methods, workloads, performance indicators (such as latency, throughput, I / O mode), etc., to the trained AI model. The AI model analyzes these input data and generates an optimization plan for the input storage access policy. It generates the corresponding high-level language code, generates Wasm code through the container, and then generates native binary code through the trusted Wasm compiler. Finally, the generated native binary code replaces the original storage access policy in the kernel module, thereby realizing real-time tuning of the storage access policy.
[0128] 3. Rights management and quota module:
[0129] The permission management and quota module in the present invention is designed to provide refined control for storage access in a containerized environment. This module combines the security isolation characteristics of the kernel sandbox and can effectively manage and adjust the resource usage permissions of each container. Through the refined management of the permission model, dynamic adjustment mechanism, multi-dimensional quota model and standardized interface design, the system can provide personalized storage access rights and resource quota management for different application scenarios, ensuring fair allocation and safe use of resources.
[0130] (1) Definition and management of permission model
[0131] The permission model is designed to assign different storage access permissions to different kernel sandbox modules, and to flexibly adjust them according to the environment and application scenarios in which the kernel sandbox modules are located. For example, kernel sandbox modules in a development environment may be granted looser storage access permissions, allowing them to access more resources. Kernel sandbox modules in a production environment are strictly restricted and are only allowed to access specific storage resources.
[0132] The hierarchy of the permission model is as follows:
[0133] Global permissions: Permissions defined globally, which are common storage access rules that apply to the entire system.
[0134] Environment-level permissions: Different environments (such as development, testing, and production) have their own permission settings, which are valid for all kernel sandbox modules in the environment.
[0135] Kernel sandbox module-level permissions: Permissions defined individually for each kernel sandbox module to precisely control the range of storage resources that the kernel sandbox module can access.
[0136] Permission Type:
[0137] Read and write permissions: Controls whether the kernel sandbox module can read, write, or delete specific storage resources.
[0138] Resource access scope: defines the range of storage devices or partitions that the kernel sandbox module can access.
[0139] Time limit: Limit certain permissions to only be valid for a specific period of time, or expire after a specific operation is completed.
[0140] Implementation of permission management:
[0141] Kernel sandbox module isolation: Using the isolation feature of the kernel sandbox, the permission settings between different kernel sandbox modules are independent. The sandbox is responsible for verifying permissions during storage access and ensuring that the permissions of each container do not interfere with each other.
[0142] Unified interface: The permission management module provides a unified interface for system administrators or applications to call to configure, query, and modify the permission rules of the kernel sandbox module. The interface design follows the standardized API style to facilitate integration and expansion.
[0143] Real-time permission adjustment: By monitoring the running status of the kernel sandbox module, the system can dynamically adjust the permissions of the container in real time. For example, when the behavior of a container deviates from expectations or exceeds the security policy, the permission management module can immediately reduce the access rights of the container or lock it within a specific resource range.
[0144] (2) Multi-dimensional quota model:
[0145] In the kernel sandbox module environment, different kernel sandbox modules may have different resource requirements. In order to ensure fair use of resources and stable operation of the system, the present invention designs a multi-dimensional quota model, which can allocate quotas such as storage resources and I / O throughput to kernel sandbox modules according to different dimensions.
[0146] Quota dimensions:
[0147] Storage capacity quota: Allocate a fixed storage capacity to each kernel sandbox module, limiting the disk space it can occupy. It can be flexibly adjusted according to the application scenario.
[0148] I / O throughput quota: Limits the maximum I / O operation rate of the kernel sandbox module to prevent some kernel sandboxes from excessively occupying system I / O resources and affecting the performance of other containers.
[0149] Other dimensions: It can also be extended to quota management based on network bandwidth, CPU usage, and other dimensions.
[0150] Quota management mechanism:
[0151] Resource allocation and monitoring: When each kernel sandbox is started, the system will allocate quotas to it according to the preset policies. The quota management module continuously monitors the resource usage of the kernel sandbox to ensure that it runs within the quota range.
[0152] (3) Unified interface and API design:
[0153] In order to ensure that the permission management and quota module can be tightly integrated with the Wasm storage access policy module, the present invention implements a unified interface layer for querying and adjusting permissions and quotas.
[0154] API features:
[0155] Permission query and setting: Allows the Wasm storage policy module to query the permission settings of the current kernel sandbox through the interface and adjust permissions when necessary. This includes but is not limited to permission escalation, permission tightening, or temporarily disabling certain storage access capabilities.
[0156] Quota query and adjustment: The interface also supports real-time quota query and adjustment, which can flexibly modify the storage capacity or I / O throughput quota of the kernel sandbox during operation.
[0157] Sandbox isolation mechanism: During permission and quota adjustments, the kernel sandbox continues to provide isolation protection to ensure that the adjustments do not affect other running containers. The kernel sandbox dynamically updates permissions and quotas to ensure that modifications take effect immediately.
[0158] 4. Reference federated learning to optimize storage access strategy:
[0159] Federated learning is a distributed machine learning technology that allows participants to jointly train machine learning models without sharing the original data. With federated learning, each container trains a model locally and sends model updates to a central server, which aggregates these updates to generate a global model. Federated learning is a distributed machine learning framework that allows multiple participants to collaboratively train a model without actually sharing their datasets. This approach is particularly suitable for protecting privacy and sensitive data because it does not require data to leave the local environment, but instead achieves iterative improvements to the global model by aggregating model updates. In this process, each participant trains a model only on its local data and sends the updated model parameters to a central server for aggregation. The central server aggregates these updates to create a global model, and then sends the updated global model back to each participant. This global model can be optimized based on the diverse needs and data of different containers, and dynamically adjust storage access policies to adapt to different application scenarios. This process is repeated until a predetermined convergence criterion is reached or a certain number of iterations are met.
[0160] Through federated learning, corresponding local optimization models and a global optimization model are generated for different kernel modules, and the storage access policies of kernel modules are continuously optimized through these models. The training process requires a large amount of storage access data in real environments. These data consist of storage access methods, workloads, performance indicators (such as latency, throughput, I / O mode), etc. in each kernel sandbox. These data are pre-processed and used as input for training.
[0161] First, the optimization model is trained locally in each container. After each node completes local training, it sends the local model parameters to the aggregator instead of sharing the original data. After the central server receives the model parameters from multiple nodes, it aggregates these parameters and performs weighted aggregation based on the weight and quality of each container model update.
[0162] After the central server completes the aggregation of model parameters, the global model is updated. The central server sends the global model parameters to each participating distributed node, so that each node can continue the next round of local training based on the updated global model.
[0163] Federated learning is usually an iterative process, with each round consisting of local training and global updates. This process continues for multiple rounds until the global model converges and the performance reaches the expected level. As the number of training rounds increases, the global model gradually converges, eventually forming a global optimization model that can generalize the data of each node.
[0164] The role of federated learning in storage access optimization is to improve the generalization ability and privacy protection of the model by coordinating training with each container without sharing data. Through local training, parameter updates and global aggregation, federated learning can dynamically optimize storage access strategies to maximize performance.
[0165] After the optimization model is generated, the model is used to perform real-time reasoning on the intercepted storage access patterns and generate optimization strategies. These strategies may include I / O scheduling optimization, cache management adjustment, data prefetching strategy, or specific storage access strategy optimization. The generated optimization strategy is applied to the Wasm storage access code, compiled by the Wasm compiler to generate new binary code, and loaded into the kernel sandbox for execution to improve storage access performance.
[0166] In the storage access optimization scenario, the core role of federated learning is:
[0167] (1) Data privacy protection: Since storage access data may contain sensitive information, federated learning allows different nodes (such as multiple containers) to train models locally without the need to transfer data to a central server. This effectively reduces the risk of data leakage.
[0168] (2) Improved generalization capability: Federated learning models can learn storage access patterns in different scenarios by training on multiple nodes. This improves the generalization capability of the model, enabling it to perform well in various storage environments.
[0169] (3) Model optimization: Federated learning can effectively combine the computing resources of multiple devices or nodes to collaboratively optimize storage access policies. After each node trains the model locally, the model parameters will be sent to the central server or aggregator for update. After multiple iterations, a global optimization model is finally formed.
[0170] (4) Continuous self-evolution capability: Federated learning allows the model to be continuously updated and evolved as new storage access data is introduced. By periodically collecting model updates from different nodes, it can adapt to dynamically changing storage access patterns.
[0171] First, the AI model is trained locally in each container. After each node completes local training, the local model parameters are sent to the aggregator instead of sharing the original data.
[0172] After receiving the model parameters from multiple nodes, the central server aggregates these parameters and performs weighted aggregation based on the weight and quality of each container model update.
[0173] After the central server completes the aggregation of model parameters, the global model is updated. The central server sends the global model parameters to each participating distributed node, so that each node can continue the next round of local training based on the updated global model.
[0174] Federated learning is usually an iterative process, with each round consisting of local training and global updates. This process continues for multiple rounds until the global model converges and the performance reaches the expected level. As the number of training rounds increases, the global model gradually converges, eventually forming a global optimization model that can generalize the data of each node.
[0175] The role of federated learning in storage access optimization is to improve the generalization ability and privacy protection of the model by training the AI model in collaboration with each container without sharing data. Through local training, parameter updates and global aggregation, federated learning can dynamically optimize storage access strategies to maximize performance.
[0176] The generated optimization model has an adaptive optimization function. When the container is running, it continuously monitors the performance of storage access and adjusts the optimization strategy according to the changing access mode to ensure that the storage access performance is always in the optimal state. Through the performance monitoring and feedback mechanism, the parameters of large model reasoning are dynamically adjusted so that the optimization strategy can quickly adapt to new workloads or storage access patterns.
[0177] In this invention, the introduction of federated learning faces some technical difficulties:
[0178] Pathological application problem: Pathological applications are applications whose access patterns are significantly different from other applications, but they are not malicious. They may be caused by some special load patterns, architectures, or application requirements. Pathological applications may produce very special memory access patterns. If they are mixed into the training of general models, the model performance will be degraded.
[0179] Efficient model training with limited computing resources on storage servers: When performing federated learning on storage servers, traditional methods require each node to participate in training, which brings a large computing burden. However, the main task of the storage server is to handle data storage, reading and writing operations, and it cannot consume too much computing power for model training. Therefore, the design of federated learning in this case must consider how to effectively participate in federated learning without affecting the core tasks of the storage server (storage and I / O processing).
[0180] Adaptability and update frequency: Storage access policies need to be dynamically adjusted based on actual load and access pattern changes. However, frequent model updates may cause system instability, especially when hot updates are performed in kernel mode. The model needs to be adaptive and able to maintain good performance for a long time, while updating safely and seamlessly when adjustments are needed.
[0181] Next, the solution of the present invention will be introduced for each technical point.
[0182] (1) Pathological applications:
[0183] First, we need to detect abnormal nodes. Here we use the Isolation Forest algorithm to detect anomalies. Isolation Forest is a tree-based unsupervised anomaly detection algorithm suitable for detecting pathological applications because it focuses on identifying a small number of samples with abnormal patterns.
[0184] First, we need to collect memory access pattern data of pathological applications and normal applications. These data are a series of feature vectors such as memory access frequency, CPU occupancy, read and write optimization, cache strategy, IO scheduling, etc. .
[0185] Assume there is a set of access mode data sets , where each A feature vector representing the memory access pattern of an application.
[0186] For each isolated tree, a feature is randomly selected and random split points . The dataset is divided into and split point Divided into two parts. Repeat this process until each data point is isolated or the depth of the tree reaches a preset maximum value.
[0187] For each application sample , we calculate its path length in the isolation forest (i.e. the number of steps it takes to be isolated in the tree). Anomaly score It is given by the following formula:
[0188] ;
[0189] in It is a data node Average path length in isolated trees.
[0190] It is a sample The number of edges (i.e., the number of splits) required from the root node to the node where the sample is isolated (being isolated means that the sample eventually reaches the leaf node of the tree) is determined through the feature selection method in the isolation tree construction stage.
[0191] ;
[0192] Where e is the sample The number of edges from the root node to the leaf node of the tree, that is, the number of splits. T.size represents the sample The number of samples at the same leaf node, C(T.size), can be regarded as a correction value (indicating the average path length of a binary tree constructed with T.size samples). Because some sample points have not been isolated, the tree stops growing, and this item gives a correction to its height.
[0193] n is the size of the dataset, is the normalization coefficient, which can be approximated as:
[0194] ;
[0195] in is the nth harmonic number:
[0196] ;
[0197] In order to determine whether a sample is an anomaly, its path length needs to be compared with the expected path length. Due to the different sizes of data sets n, the expected path length will be different. In order to unify the standard, the path length is standardized. The standardized path length can make the results between different data sets comparable and reconcile the number of Played a key role in this process.
[0198] The harmonic number can be approximated as:
[0199] ;
[0200] in is the Euler-Mascheroni constant, which has a value of approximately 0.577.
[0201] This relationship between harmonic numbers and natural logarithms makes them very useful when working with large amounts of data, especially when dealing with problems like the expected depth of a random tree, path length, etc., which are related to the height of the tree.
[0202] if Close to 1, it means that the sample may be abnormal (i.e., pathological application); if Close to 0, it means the sample is normal.
[0203] Set a threshold , according to the anomaly score Separate pathological applications from normal applications. Typically, we can determine this threshold empirically or through cross-validation.
[0204] if , then the application Mark as sick application.
[0205] Through the isolation forest algorithm, pathological applications with special access strategies are detected. In order to prevent them from affecting the training of the global model, they need to be isolated from the global model and trained with independent optimization models for their specific memory access patterns based on the input memory access data. This can effectively avoid the contamination of the global model by pathological applications and provide an optimization solution that is more suitable for pathological applications.
[0206] (2) Efficient model training with limited computing resources on storage servers:
[0207] Train only a small model on the container and transfer the knowledge to the central server for more complex training. Train a small-scale model on the storage server. Upload the trained small model or its gradient information to the central server.
[0208] More complex model training and updating are performed on the cloud server to reverse distill the knowledge of the small model into the large model. The small model is used as a teacher model to guide the learning of the large model. Knowledge transfer is achieved by minimizing the difference between the predicted distribution of the large model and the predicted distribution of the small model. The merged large model is pruned on the cloud server to generate a small model suitable for each storage server. The pruned model is sent to each storage server. The storage server uses its own data to perform local training of the small model again, which can make the small model better adapt to local data characteristics.
[0209] Train a small-scale model on the storage server using limited computing resources to optimize local storage access policies. Use a lightweight neural network , defined as a small model, also called a local model; the input is the access characteristics of the storage server , the output is the optimized storage strategy , the network parameters are defined as .
[0210] Collect memory access pattern data. These data are a series of feature vectors such as memory access frequency, CPU usage, read and write optimization, cache strategy, IO scheduling, etc. , where each A feature vector representing a certain access mode of a container.
[0211] Train a small model using a local dataset:
[0212] ;
[0213] in is the learning rate, is the loss function.
[0214] After completing local training, the small model Parameters and gradient To the central server.
[0215] Combine small models or gradient information from multiple storage servers on the central server to train more complex large models.
[0216] Collect the gradients and parameters of small models from multiple storage servers, combine the collected small model information, and train a more complex large model .
[0217] Through the knowledge distillation method, the knowledge of the small model is transferred to the large model to optimize the learning of the large model. As a teacher model, the big model As a student model, the prediction generated by the small model is , the large model predicts .
[0218] Define the knowledge distillation loss function , which is optimized by minimizing the difference between the output distributions of the large and small models:
[0219] ;
[0220] in is the Kullback-Leibler divergence, which measures the difference between two probability distributions.
[0221] Tuning large model parameters by minimizing distillation loss :
[0222] ;
[0223] Prune the large model on the central server to reduce its complexity and generate a small model suitable for the storage server. Prune unimportant connections and parameters in the large model based on the absolute value or importance of the model weight. It has low computational and memory overhead and is suitable for deployment on storage servers. The model parameters after pruning are , define the pruning function :
[0224] ;
[0225] The pruned model is sent to each storage server, and the storage server data is used locally for fine-tuning to adapt the model to the local data characteristics. Deploy to the storage server.
[0226] Using local datasets Fine-tune the small model that was sent:
[0227] ;
[0228] This allows the model to better adapt to local storage access patterns.
[0229] (3) Adaptability and update frequency:
[0230] In order to solve the adaptability and update frequency problems faced by federated learning in optimizing storage access strategies, especially to maintain the stability and adaptability of the system when updating the model under the dynamic load of the system, the following solutions can be adopted. We will adopt the following core strategies: adaptive design of the model, incremental learning, update frequency control and rollback mechanism, and safe hot update mechanism.
[0231] Design an adaptive model that can respond to changes in storage access patterns and load, and maintain good performance with infrequent updates.
[0232] Introducing the Adaptive Module ,in represents the input features, is the parameter of the adaptive module. This module is related to the small model Work together. Dynamically adjust the model's output to accommodate different storage access loads and patterns.
[0233] In this way, the model can adapt to different workloads over a longer period of time without the need for frequent updates.
[0234] Fine-tune the adaptive module based on real-time feedback data from storage access. Using the loss function To reflect the quality of adaptive adjustment. The adaptive module is designed to be lightweight so that it can be quickly adjusted and deployed in real time on the storage server.
[0235] Through the incremental learning mechanism, the complexity of each model update is reduced, and the system stability is maintained by controlling the update frequency, and only learning is performed on data or load patterns that have changed significantly.
[0236] Accumulate a collection of significantly changed data points on a local storage server , use this data for incremental training without retraining the entire model:
[0237] ;
[0238] Parameter Fusion:
[0239] Combine the updated parameters of the incremental learning with the parameters of the adaptive module to generate a new combined model:
[0240] ;
[0241] in is a coefficient that controls the weight and is used to adjust the contribution of the main model and the adaptive module in the final combined model.
[0242] Restructure storage access policies using the new composition model:
[0243] ;
[0244] Control the update frequency of the model to ensure that the model is updated only when necessary. At the same time, introduce a rollback mechanism to ensure rapid recovery when the update fails. Design an update frequency control strategy based on the load of the storage server and the change measurement of the access pattern. Introduce a threshold to determine whether to update the model. The model update is triggered only when the measurement of the system load change exceeds the threshold.
[0245] Before each model update, back up the existing model parameters. If the updated model performance is not as expected or the system becomes unstable, use the backed-up model parameters for rollback. Implement a double buffering mechanism to load the new model into the backup buffer during the update and keep the old model working in the main buffer. When the new model is successfully verified, the main buffer and the backup buffer are switched to ensure seamless switching of the system during the update process.
[0246] Figure 1 This is an architecture diagram of the invention deployed in the operating system, where the Wasm compilation module is responsible for compiling the Wasm storage access policy code into native binary code. The kernel sandbox provides a secure execution environment to execute the compiled binary code. The storage access policy module implements storage access policy and manages storage read and write, cache, I / O scheduling, etc.
[0247] Figure 2 The operation flow of each module of the invention is as follows:
[0248] ① Generate Wasm code from multiple programming languages such as C, C++, Rust through compilers;
[0249] ② Compile the Wasm code into native binary code through the trusted Wasm compiler module;
[0250] ③ Run the native binary code compiled by the trusted Wasm compiler in the kernel sandbox;
[0251] ④ Collect storage access requests from containers, perform feature analysis, and pass them to the federated learning and AI large model modules for storage access optimization;
[0252] ⑤ Generate high-level language code from the storage access strategy optimized by the AI model;
[0253] ⑥Through the user interface, administrators can dynamically adjust container permissions at runtime;
[0254] ⑦Configure storage access permissions.
[0255] The specific steps are as follows:
[0256] A method for optimizing storage access strategy of cloud native containers based on federated learning is provided. A wasm sandbox is set up, its storage is placed in kernel state, and a trusted wasm compiler module is set up inside the sandbox. The method includes the following steps:
[0257] S1. The Wasm compiler in the container compiles the high-level language code into wasm code and hands the wasm code to the trusted wasm compiler module;
[0258] The steps for interacting with the kernel sandbox are as follows:
[0259] When a container makes a storage request, the kernel sandbox parses the request and maps it into an RDMA operation;
[0260] An RDMA protocol stack is implemented inside the kernel sandbox, which separates data transmission operations from the network layer so that requests can be directly passed to remote storage devices;
[0261] Containers can directly access the storage devices of remote nodes through the kernel sandbox and RDMA protocol stack. Through the pre-established RDMA connection, data transmission across nodes does not need to go through the intermediate network layer or storage middleware, achieving low-latency, high-bandwidth storage access across nodes.
[0262] Set the memory protection key MPK to reduce memory check overhead in kernel state. By configuring the MPK and memory page key, the permissions of frequently accessed memory pages are set to allow, reducing the need for permission checks.
[0263] S2. The trusted wasm compiler module compiles the wasm code into binary code data; the kernel sandbox delivers the running binary code data to the AI model, and uses federated learning to collaborate with each container to train the AI model, so that the AI model optimizes the storage access strategy of the kernel sandbox;
[0264] The steps for federated learning and AI model training are as follows:
[0265] S21. Each container uses the local model of federated learning to train the AI model. After each node completes local training, it sends the local model parameters to the aggregator instead of sharing the original data.
[0266] S22. After receiving the model parameters from multiple nodes, the central server aggregates these parameters and performs weighted aggregation according to the weight and quality of each container model update;
[0267] S23. After the central server completes the aggregation of model parameters, the global model will be updated; the central server sends the global model parameters to each participating distributed node, so that each node can continue the next round of local training based on the updated global model.
[0268] S3. The container compiles the code generated by the AI model into Wasm code, repeats steps S1-S2, and uses the trusted Wasm compiler to compile the Wasm code into native binary code for the kernel sandbox to run, replacing the previous storage access policy.
[0269] S4. Through the user interface, administrators can dynamically adjust container permissions and configure storage access permissions at runtime.
[0270] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for optimizing cloud native container storage access policy based on federated learning, characterized in that: Setting up a kernel sandbox and placing its storage in kernel mode includes the following steps: S1. The Wasm compiler in the container compiles the high-level language code into wasm code and hands the wasm code to the trusted wasm compiler module; S2. The trusted wasm compiler module compiles the wasm code into binary code and passes it to the sandbox for running. The kernel sandbox supports the customization and dynamic optimization of storage access policies. The kernel sandbox passes the running binary code data to the AI model, and uses federated learning to collaborate with each container to train the AI model, so that the AI model optimizes the storage access policy of the kernel sandbox. The steps are as follows: Step 1. Use the local dataset to train the small model M small ; Step 2. Collect the gradients and parameters of the small models from multiple storage servers and combine the collected small models M small Information, train a large model M large ; Step 3. Define the knowledge distillation loss function L disill , which is optimized by minimizing the difference between the output distributions of the large and small models: Where KLDiv is the Kullback-Leibler divergence, and the small model M is small The generated prediction is y small , large model M large The prediction is y large , n is the size of the data set; Step 4. Prune the large model on the central server to generate a small model suitable for the storage server, and send the pruned model to each storage server; Step 5. Use local dataset X i Fine-tune the small model that was sent: μ2 is the learning rate, θ pruned is the model parameter after pruning, L(·) is the loss function, M pruned This is a small model after pruning; S3. The container compiles the code generated after the AI model optimization into Wasm code, repeats steps S1-S2, and uses the trusted Wasm compiler to compile the Wasm code into native binary code for the kernel sandbox to run, replacing the previous storage access policy.
2. The method for optimizing cloud native container storage access policy based on federated learning according to claim 1 is characterized in that: In step S2, the steps of federated learning and AI model training are as follows: S21. Each container uses the local model of federated learning to train the AI model. After each node completes local training, it sends the local model parameters to the aggregator instead of sharing the original data. S22. After receiving the model parameters from multiple nodes, the central server aggregates these parameters and performs weighted aggregation according to the weight and quality of each container model update; S23. After the central server completes the aggregation of model parameters, the global model will be updated; the central server sends the global model parameters to each participating distributed node, so that each node can continue the next round of local training based on the updated global model.
3. The method for optimizing cloud native container storage access policy based on federated learning according to claim 1 is characterized in that: The container gains direct access to the raw hard disk through the kernel sandbox: The kernel sandbox provides a secure interface for hardware devices, so that storage operations no longer rely on the abstract layer of the volume manager or file system; the kernel sandbox directly converts the container's storage requests into low-level operations on the hard disk, and isolates and manages permissions for the kernel sandbox.
4. The method for optimizing cloud native container storage access policy based on federated learning according to claim 1 is characterized in that: The kernel sandbox exposes the RDMA interface directly to the container through the layered vertical penetration technology: When a container makes a storage request, the kernel sandbox parses the request and maps it into an RDMA operation; An RDMA protocol stack is implemented inside the kernel sandbox, which separates data transmission operations from the network layer, allowing requests to be directly passed to remote storage devices; The container directly accesses the storage device of the remote node through the kernel sandbox and the RDMA protocol stack. Through the pre-established RDMA connection, data transmission across nodes does not need to go through the intermediate network layer or storage middleware, achieving low-latency, high-bandwidth storage access across nodes.
5. The method for optimizing cloud native container storage access policy based on federated learning according to claim 1, characterized in that: Set the memory protection key MPK to reduce memory check overhead in kernel state. By configuring the MPK and memory page key, the permissions of frequently accessed memory pages are set to allow, reducing the need for permission checks.
6. The method for optimizing cloud native container storage access policy based on federated learning according to claim 1 is characterized in that: The steps for handling abnormal nodes in federated learning are as follows: Step 1. Assume a set of access pattern data sets X = {X1, X2, X3...X n }, where each X i Represents a memory access pattern feature vector of an application; For each isolated tree, a feature f is randomly selected j and a random split point p j , the data set is divided according to the feature f j and the split point p j It is divided into two parts and the process is repeated until each data point is isolated or the depth of the tree reaches the preset maximum value; Step 2. For each application sample X i , calculate its path length in the isolation forest, the anomaly score s(X i ) is given by the following formula: Where E(h(X i )) is data node X i The average path length in an isolated tree; h(X i ) is the sample X i Through the feature selection method in the isolation tree construction stage, the node where the sample is isolated is reached from the root node; h(X i )=e+C(T.size); Where e is the sample X i The number of edges from the root node to the leaf node of the tree, that is, the number of splits; T.size represents the number of samples X i The number of samples at the same leaf node, C(T.size) is a correction value, which means the average path length of a binary tree constructed by T.size samples; n is the size of the data set, and C(n) is the normalization coefficient, which is approximately: where H(n) is the nth harmonic number: Since different datasets have different sizes n, the expected path length will be different. In order to unify the standards, the path length is standardized; The harmonic number is approximately: H(n)≈ln(n)+Y; where γ is the Euler-Mascheroni constant; Step 3. If s(X i ) is close to 1, indicating that the sample may be abnormal, that is, a pathological application; if s(X i ) is close to 0, indicating that the sample is normal; Set the threshold T according to the anomaly score s(X i ) separates pathological applications from normal applications. If s(X i )>T, then the application X i Mark applications as pathological to isolate them from the global model.
7. The cloud native container storage access policy optimization method based on federated learning according to claim 1 is characterized in that: The steps to optimize the storage access policy are as follows: In step 1, the small model M small The network parameters are defined as θ s ; Where μ1 is the learning rate, L(·) is the loss function, and the network parameters are defined as θ s , X i Represents a certain access mode feature vector of the container; after completing local training, the small model M small The network parameters θ s and gradient Transmit to the central server; In step 2, use the small model M small As a teacher model, the large model M large As a student model, the small model M small The generated prediction is y small =M small (X i ), large model M large The prediction is y largs =M large (X i ); In step 3, the large model network parameters θ are adjusted by minimizing the distillation loss L : μ3 is the learning rate; In step 4, the model parameters after pruning are θ pruned , define the pruning function P(·): i pruned =P(θ L )。 8. The method for optimizing cloud native container storage access policy based on federated learning according to claim 1, characterized in that: Introducing the adaptive module A(X i ,θ A ), θ A are the parameters of the adaptive module, which is related to the small model M small (X i ,θ A ) work together; A(X i ,θ A ) Dynamically adjust the model output to accommodate different storage access loads and patterns; The adaptive module is fine-tuned based on the real-time feedback data of storage access, using the loss function L adapted To reflect the quality of adaptive adjustment; The set of data points D with significant changes accumulated on the local storage server increment ={(X newi ,y newi )}, and use this data for incremental training without retraining the entire model: The updated network parameters of the small model, The network parameters of the small model before updating, μ4 learning rate, The adaptive module learns the gradient, θ M are the parameters of the small model, that is, the parameters of the local model; X new is the input data on the local storage server, i.e., the feature vector of the storage access policy after preprocessing; y new is the output corresponding to the input data, that is, the desired optimization strategy; parameter fusion: the parameters updated by incremental learning are combined with the parameters of the adaptive module to generate a new combined model: i combined =θ M +(1-α)θ A ; Where α is a coefficient of control weight, which is used to adjust the contribution of the main model and adaptive module in the final combined model; Restructure storage access policies using the new composition model: y final =M small (X i ,the combined )+A(X i ,the A )。 9. The method for optimizing cloud native container storage access strategy based on federated learning according to claim 2 is characterized in that: Control the update frequency of the local model to ensure that the local model is updated only when necessary. At the same time, introduce a rollback mechanism to ensure rapid recovery when the update fails. Back up the existing model parameters before each model update. If the performance of the updated model is not as expected, or the system becomes unstable, use the backed-up model parameters for rollback. A double buffering mechanism is implemented in the container. During an update, the updated local model is loaded into the backup buffer, and the model before the update is kept working in the main buffer. When the updated local model is successfully verified, the main buffer and the backup buffer are switched.
10. A system for optimizing cloud native container storage access policy based on federated learning according to any one of claims 1 to 9, characterized in that: Including trusted Wasm compiler module, kernel sandbox module, federated learning and AI large-scale model generation module, permission management and quota module; Trusted Wasm compiler module: The compiler in the container is responsible for compiling high-level language code into intermediate code in Wasm format, and passing the Wasm code generated by the container to the trusted Wasm compiler module in the root privilege process in user mode; the trusted Wasm compiler first uses LLVM to parse the Wasm bytecode into an intermediate representation using LLVM IR, optimizes using LLVM's optimization channel, generates efficient native binary code through the LLVM backend, and runs the binary code in the kernel sandbox; Kernel sandbox module: The kernel sandbox runs in a secure execution environment in kernel state and is used to securely execute native binary code generated by the Wasm compiler. The kernel sandbox is located above the driver layer and supports direct exposure of bare hard disks or RDMA interfaces to containers. The kernel sandbox supports dynamic optimization of storage access policies. Federated learning and AI large-scale model generation module: Use federated learning to collaborate with various containers to train AI models, improving the generalization ability and privacy protection of the model without sharing data; through local training, parameter updates and global aggregation, federated learning can dynamically optimize storage access strategies to maximize performance; Permission management and quota module: Assign different storage access rights to different kernel sandbox modules: Through the refined management of the permission model, dynamic adjustment mechanism, multi-dimensional quota model and standardized interface design, the system can provide personalized storage access rights and resource quota management for different application scenarios, ensuring fair allocation and safe use of resources.
