Serverless-oriented efficient AI model management and secure loading system and method

Abstract

The invention provides an efficient AI model management and safe loading system and method oriented to a Serverless framework, and aims to solve the high concurrency performance bottleneck and potential safety hazards of AI model loading in a Serverless environment. According to the system, an AI model management module is introduced, efficient semantic cutting is carried out on an uploaded AI model, and fine-grained encryption model blocks are generated; a high-performance P2P network is constructed by utilizing a Serverless cluster, and distributed storage and fast routing of model blocks are realized; a function instance loads a model block in a high-concurrency mode through a P2P network, a multi-stage intelligent cache is constructed locally, and dynamic caching and prefetching are achieved. According to the method, the model loading parallelism degree is improved by adopting a heuristic cutting algorithm, lightweight model updating is realized by utilizing incremental updating transactions, and the loading overhead is minimized. According to the method, the AI model loading performance and efficiency in the Serverless scene are effectively improved, meanwhile, the flexibility and safety of model management are considered, and the method has remarkable technical innovation and practical value.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an efficient AI model management and secure loading system and method for a Serverless computing environment. Background Art

[0002] With the rapid development of artificial intelligence technology, AI models have been widely used in various industries, such as intelligent customer service, personalized recommendations, and autonomous driving. To support these applications, the deployment and services of AI models are also showing a trend of serverless, that is, using the serverless computing platform to achieve elastic deployment and on-demand calling of AI models. With its advantages of high elasticity, free operation and maintenance, and pay-as-you-go, the serverless architecture has important application prospects in the field of AI.

[0003] Currently, there are two main ways to implement AI model deployment in a serverless environment. The first is to package the AI ​​model together with the application code and deploy it as a serverless function, but this solution will cause the function package to expand in size and significantly extend the cold start time. The second is to store the AI ​​model in a centralized object storage service, and the function instance pulls the model from the storage service when it is called. However, in high-concurrency scenarios, centralized storage can easily become a performance bottleneck, and the security of the model is difficult to guarantee.

[0004] However, both solutions have significant shortcomings. Although the packaging and deployment method is simple and direct, AI model files are often large in size. Packaging them together with the code will significantly increase the size of the function package, causing a surge in cold start delays. Although the centralized storage method decouples model storage from function computing, under high concurrent requests, centralized storage services can easily become a performance bottleneck, affecting the speed of model loading. In addition, existing solutions have shortcomings in model security, version management, and consistency maintenance.

[0005] To solve the above problems, some studies have proposed using distributed cache or in-memory database to store AI models, and improving model access performance through data sharding and copy management. Other studies have tried to use P2P networks to distribute AI models, and use parallel transmission between nodes to accelerate model loading. Although these methods have improved the loading efficiency of AI models to a certain extent, there are still limitations such as high model synchronization overhead, unbalanced node load, and imperfect security mechanisms.

[0006] Therefore, there is an urgent need for an efficient, secure, and reliable Serverless AI model management and loading solution to meet the growing AI application needs and performance challenges. Such a solution should be able to fully utilize the elastic computing power of the Serverless environment, achieve fine-grained management and parallel loading of AI models, and provide flexible version control and security protection mechanisms. Developing a Serverless AI model management and loading system that meets the above requirements has become a research hotspot in this field. Summary of the invention

[0007] The purpose of this invention is to solve the problems of inefficiency and insecurity in AI model loading in the existing Serverless environment.

[0008] The current mainstream model loading solutions usually adopt centralized storage or packaged deployment. In high-concurrency scenarios, they are prone to problems such as excessive cold start delays, storage performance bottlenecks, and model data leakage, which seriously affect the service quality and security of AI applications.

[0009] In addition, the present invention also aims to provide a fine-grained, versioned AI model management mechanism that supports incremental updates and rapid rollback of models, thereby improving the flexibility and reliability of model management.

[0010] To achieve the above objectives, the present invention provides an AI model management and secure loading system based on Serverless architecture, which is characterized by: the system introduces an AI model management module, performs semantic segmentation on the uploaded AI model, and generates multiple independently encrypted model blocks; uses Serverless cluster to build a P2P network to realize distributed storage of model blocks; function instances load model blocks in parallel through the P2P network, and build a multi-level cache locally to realize efficient and secure access to the model.

[0011] In some embodiments, the system also introduces blockchain technology to store model version information on the chain to achieve decentralization and high reliability of model management.

[0012] Specifically, the AI ​​model management module adopts a heuristic semantic analysis algorithm to cut the model into multiple fine-grained logical units according to the model's hierarchical structure and parameter distribution. Each unit is independently encrypted and serves as a model block.

[0013] Furthermore, the system builds an unstructured P2P network on the Serverless cluster and uses the Kademlia algorithm to implement distributed storage and routing query of model blocks. In the implementation of the P2P network, in addition to using the Kademlia algorithm to optimize routing and the UDT protocol to improve transmission performance, the present invention also introduces the Reed-Solomon erasure code mechanism to increase the system's fault tolerance by adding an appropriate amount of redundant data blocks, thereby preventing data loss caused by the failure of some nodes.

[0014] Preferably, the P2P network uses the UDT transmission protocol to support high-throughput, low-latency data transmission. Optionally, TLS encrypted communication is used between the function instance and the P2P network to prevent the model data from being stolen during transmission.

[0015] In one embodiment, the function instance uses a multi-level cache mechanism locally to cache frequently accessed model blocks in memory and cache the remaining model blocks in a high-speed SSD to increase the loading speed of the model. The function instance will pre-fetch the most likely model blocks to be accessed based on the access pattern of the model blocks to reduce the delay caused by cache misses.

[0016] In some embodiments, the AI ​​model management module adopts a blockchain-based version management mechanism to store the model's version information and hash value on the chain to prevent the model from being maliciously tampered with.

[0017] In addition, the present invention can also introduce a federated learning mechanism to support secure aggregation and incremental training of model blocks distributed in multiple Serverless clusters.

[0018] In other embodiments, the system provides role-based fine-grained access control, and different function instances access model blocks of different scopes according to their permission levels.

[0019] In a preferred embodiment, the AI ​​model management module first converts and standardizes the model uploaded by the user, and extracts the model's meta-information, such as network structure, hyperparameters, etc. Then, the management module calculates the semantic similarity of the model parameters based on the graph embedding algorithm and constructs the semantic vector space of the model.

[0020] Next, the spectral clustering algorithm is used to divide the semantic vector to obtain multiple semantically similar model blocks. Finally, the management module uses the key provided by the user to independently encrypt each model block and pushes the encrypted model block and meta-information to the P2P network storage. In the P2P network, in order to speed up the synchronization and query of meta-information of model blocks, the Gossip protocol is used between nodes to exchange model block summary information.

[0021] At the same time, Bloom filters are deployed on each node to index the locally cached model blocks and speed up the matching and search process of model blocks.

[0022] In another preferred embodiment, the function instance adopts a dual-level cache architecture, the hot model blocks are resident in the memory and managed by the Least Recently Used (LRU) strategy, and other model blocks are cached in the SSD and managed by the Least Frequently Used (LFU) strategy. In the management of the local cache, in addition to comprehensively considering the access frequency of the model blocks, the present invention also takes into account the temporal locality of the access.

[0023] For caches at different levels, the weight ratio of LRU and LFU is dynamically adjusted, and a time decay factor is introduced to optimize the utilization of cache space. When cache space is insufficient, cache elimination is performed based on the comprehensive score of the model block, taking into account both cache hit rate and access latency. The function instance quickly finds the required semantic model block by calling the Approximate Nearest Neighbor (ANN) algorithm.

[0024] In addition, the present invention also designs a performance analysis module for real-time collection and analysis of key performance indicators of the system, including model loading delay, throughput, cache hit rate, etc.

[0025] Through performance bottleneck analysis and anomaly detection algorithms, performance problems can be located in a timely manner, and resource allocation can be dynamically adjusted according to system load to achieve continuous performance optimization.

[0026] By adopting the above scheme, the present invention has the following beneficial effects: (1) Through model semantic segmentation and P2P distributed storage, high concurrency and low latency of model loading are achieved; (2) Independently encrypt model blocks to protect the confidentiality and integrity of model data; (3) Introducing blockchain to implement model version management and support trusted tracking and incremental updates of models; (4) Adopt fine-grained caching strategy to improve model loading speed and cache hit rate; (5) Support federated learning to achieve privacy protection and secure aggregation of models.

[0027] In summary, the present invention provides an efficient, secure, and flexible Serverless AI model management and loading system, which significantly improves the service quality and scalability of AI applications, reduces the complexity of model management, and has broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 This is the overall architecture diagram of the AI ​​model management and secure loading system based on Serverless architecture of the present invention.

[0030] Figure 2 This is the workflow diagram of the AI ​​model management module.

[0031] Figure 3 It is the topology structure and routing principle diagram of the P2P network.

[0032] Figure 4 This is a schematic diagram of the Reed-Solomon erasure code fault tolerance mechanism.

[0033] Figure 5 This is a diagram of the multi-level cache architecture of a function instance.

[0034] Figure 6 It is a flowchart of the security sandbox and model decryption of the function instance.

[0035] Figure 7 It is a schematic diagram of the model version management and audit trail of the blockchain.

[0036] Figure 8 It is the anomaly detection and alarm flow chart of the performance analysis module. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0039] Example 1 This example provides an AI model management and security loading system based on Serverless architecture. Figure 1 As shown in the figure, the system mainly includes AI model management module, Serverless cluster, P2P network and function instance.

[0040] The AI ​​model management module is the core control unit of the system, responsible for receiving and processing AI models uploaded by users. When a user uploads a model through the API interface, the management module first converts and standardizes the model, converts the model into a unified format supported by the system, and extracts the key meta information of the model, such as network structure, hyperparameter configuration, etc. In order to efficiently organize and retrieve this meta information, the management module adopts a meta information representation method based on the knowledge graph. By mapping various meta information of the model to entities, attributes, and relationships in the knowledge graph, the management module builds a semantically rich and structured meta information library, which greatly improves the query and reasoning efficiency of meta information.

[0041] While standardizing the model, the management module will use semantic analysis technology to automatically generate semantic tags for the model. Semantic tags reflect the application field, functional characteristics, data characteristics, etc. of the model, providing an important basis for subsequent semantic segmentation. Based on natural language processing algorithms such as Word2Vec and BERT, the management module can extract keywords and semantic concepts from unstructured information such as the model name, description, code comments, etc., and map them to a predefined semantic tag system. Compared with manual annotation, this method is not only more efficient, but also can capture broader and more fine-grained semantic information.

[0042] Next, the management module calls the semantic segmentation engine to segment the standardized model. Figure 2 As shown in the figure, the semantic cutting engine uses an innovative graph embedding algorithm to convert the model's computational graph into a representation in a low-dimensional vector space. Traditional model cutting methods, such as algorithms based on graph partitioning, usually only consider the topological structure of the model's computational graph and ignore the semantic associations between nodes. The graph embedding algorithm of the present invention not only retains the structural information of the computational graph, but also integrates the semantic features of the nodes.

[0043] Specifically, the graph embedding algorithm first represents the computational graph of the model as an undirected weighted graph ,in represents a collection of computing nodes, Represents the dependency relationship between nodes, weight matrix It describes the semantic similarity between nodes. Semantic similarity can be calculated by methods such as cosine similarity and Jaccard coefficient, or by using pre-trained semantic embedding models such as Word2Vec or BERT to map the semantic labels of nodes to a low-dimensional semantic vector space, and then calculate the Euclidean distance or dot product similarity between vectors.

[0044] After obtaining the weighted graph representation, the graph embedding algorithm adopts the idea of ​​random walk to learn the low-dimensional embedding representation of nodes by simulating the diffusion process of molecules on the graph. The random walk process can be described by the following formula: in, is the time step The state distribution vector of the node at time is the initial state distribution, usually a uniform distribution or a degree distribution, is a hyperparameter that controls the diffusion speed. As the time step increases, the state distribution of the nodes will continue to propagate and mix, and eventually converge to a stable distribution, reflecting the global structural characteristics of the nodes in the graph.

[0045] In order to further improve the embedding quality, the graph embedding algorithm introduces the attention mechanism and multi-view fusion technology. The attention mechanism dynamically adjusts the weight of the edge to make the semantically related nodes closer in the embedding space. Multi-view fusion builds multiple graph embeddings from different perspectives (such as topological structure, semantic attributes, control flow, etc.), and then fuses them into a unified embedding representation using methods such as weighted averaging or autoencoders, which improves the robustness and expressiveness of the embedding.

[0046] In the embedding space, the semantic cutting engine uses the spectral clustering algorithm to cut the model. Compared with the traditional cutting method based on graph partitioning, spectral clustering can adaptively discover the semantic structure of the model and achieve more refined and semantic cutting. The core idea of ​​spectral clustering is to transform the graph cutting problem into a clustering problem in the embedding space. By calculating the similarity matrix between embedded vectors, constructing the Laplacian matrix of the data, and performing eigenvalue decomposition on it, spectral clustering can obtain an optimal partition of the graph, so that the subgraphs after the partition are compact and relatively independent.

[0047] In order to overcome the problems of spectral clustering algorithm's sensitivity to hyperparameters and high computational complexity, the present invention further improves the spectral clustering algorithm. On the one hand, an adaptive similarity threshold selection method is introduced to automatically determine the optimal number of clusters and similarity threshold by maximizing objective functions such as modularity, thus reducing the difficulty of hyperparameter tuning. On the other hand, a matrix approximation technique based on Nyström's method is adopted to reduce the algorithm's time complexity from 100% to 200% by performing low-rank approximation on the Laplace matrix. Reduce to (in is the number of nodes, is the number of sampling points), which greatly improves the scalability of the algorithm.

[0048] The cut model blocks are encrypted by the security manager. Compared with the traditional symmetric encryption algorithm, the present invention adopts the more secure and flexible attribute-based encryption (ABE) technology. ABE allows the generation of keys and ciphertexts based on the attributes of the model blocks (such as the task, developer, security level, etc.), and only users who meet specific attribute conditions can decrypt. This fine-grained access control mechanism can effectively prevent unauthorized access and leakage of model blocks.

[0049] In order to improve encryption efficiency and key management convenience, the security manager adopts an identity-based ABE solution. Compared with traditional ABE based on public key infrastructure (PKI), IBABE does not need to generate and distribute separate public and private key pairs for each user. Instead, it directly uses the user's identity information (such as email, phone, etc.) as the public key and generates the corresponding private key through a trusted key generation center (KGC). This not only simplifies the key management process, but also improves the security of the key because the private key does not need to be transmitted in the network.

[0050] During encryption, the security manager first sets the model block's attributes Generate an access structure (such as access tree or linear secret sharing matrix), and then use the public key of IBE (i.e. the identity information of the model owner) and the random number Generate the ciphertext of the model block : in, is the IBE public key, The model block plain text.

[0051] When an authorized user needs to access a model block, he first requests a private key from KGC. KGC uses the user's identity information $ID$ and the system master key to obtain the private key. Generate the user's IBE private key : User uses private key locally and ciphertext Decrypt to get the model block plaintext : Only when the user's attribute set satisfies the access structure Decryption can succeed only when the model block is decrypted, otherwise the decryption fails. In this way, the security manager implements fine-grained, efficient, and dynamic model block access control, greatly improving the security and flexibility of the system.

[0052] The encrypted model block is pushed to the P2P network together with the extracted metadata and distributedly stored by the working nodes of the Serverless cluster. The P2P network uses the Kademlia protocol for node communication and routing queries. Compared with traditional structured P2P protocols (such as Chord, Pastry, etc.), Kademlia has simpler routing logic and better convergence performance. Kademlia uses the XOR metric to calculate the logical distance between node IDs and constructs a symmetric structure based on XOR operations. This symmetric structure enables nodes to efficiently find and locate data in any direction without maintaining complex routing tables.

[0053] In order to further optimize the routing efficiency and load balancing of the P2P network, the present invention makes two improvements to the Kademlia protocol. First, a load balancing mechanism based on virtual nodes is introduced. The traditional Kademlia protocol uses prefix matching of node IDs to select routing table items, which causes nodes with similar ID prefixes to easily cluster together, causing load imbalance. Virtual nodes divide the ID space more evenly by assigning multiple virtual IDs to each physical node, so that routing requests can be more evenly distributed to each node.

[0054] Second, the connection topology between nodes is optimized. The traditional Kademlia protocol tends to select the node closest to its own ID as a neighbor, which easily leads to the appearance of dense subgraphs in the network, affecting the global routing efficiency. When constructing the routing table, the present invention not only considers the XOR distance between nodes, but also introduces a random walk mechanism to select nodes with a longer distance as neighbors with a certain probability. This randomized connection strategy, on the one hand, increases the randomness of the topology and avoids the generation of dense subgraphs; on the other hand, by introducing "long hop" connections locally, the average routing path between nodes is shortened, thereby improving the global routing performance.

[0055] In the process of data storage and retrieval, the P2P network adopts a data redundancy mechanism based on Erasure Coding to improve data reliability and availability. Traditional data redundancy methods, such as simple backup or Reed-Solomon coding, usually require large storage overhead or slow recovery when a node fails. The present invention adopts Locally Repairable Codes (LRC) technology, which significantly reduces storage overhead and recovery cost while ensuring the same fault tolerance.

[0056] The core idea of ​​LRC is to introduce local check blocks based on global Reed-Solomon coding. Local check blocks build a local fault-tolerant domain by encoding local data blocks. When a data block fails, it can be directly restored using the local check block without accessing the global check block, which greatly speeds up the recovery. At the same time, since the local check block only encodes local data, its computational cost and storage overhead are much smaller than the global check block.

[0057] Formally, LRC divides the original data into data blocks and generate local check blocks and Global check blocks, storing a total of Each local check block is responsible for data blocks for fault tolerance, so LRC can A node fails, or When a local fault-tolerant domain fails, the original data can still be restored. Compared with traditional Reed-Solomon coding, LRC reduces the storage overhead from Reduce to , the recovery cost is Reduce to .

[0058] In summary, this embodiment builds a high-performance, secure, and reliable Serverless AI model management and loading system through innovative technologies such as semantically-aware graph embedding and cutting, fine-grained permission control with attribute encryption, virtual nodes and random walk-optimized P2P routing, and efficient data fault tolerance of LRC, providing a new paradigm of efficiency, flexibility, and economy for the development and deployment of AI applications. Compared with traditional centralized AI platforms, the present invention fully utilizes the advantages of Serverless computing such as elasticity, lightness, and pay-as-you-go, while overcoming the limitations of the Serverless environment in stateful service management, network communication, and data security, providing a full-stack solution for the Serverlessization of AI models.

[0059] Example 2: Based on Example 1, this example further describes a multi-level intelligent caching mechanism in a function instance. In a Serverless environment, function instances are usually stateless and have a short life cycle, which is not suitable for long-term storage of large amounts of model data locally. Frequent loading of models from P2P networks will introduce large communication overhead and latency. In order to resolve this contradiction, the present invention designs a multi-level, intelligent, and semantically-aware caching mechanism, which constructs an efficient and adaptive local cache of model data in function instances, greatly improving the loading speed of the model and the execution efficiency of the function.

[0060] like Figure 5 As shown in the figure, the local cache of the function instance adopts a layered architecture that combines memory and disk. The first-level cache is located in the memory and is used to store the most frequently accessed model blocks. The second-level cache is located on the high-speed NVMe SSD and is used to store the less frequently accessed model blocks or the model blocks that cannot be placed in the memory. The two-level cache dynamically adapts to the local characteristics of model access and the resource constraints of the function instance through intelligent cache management strategies and adaptive cache recycling mechanisms, maximizing the cache hit rate and resource utilization efficiency.

[0061] In order to make full use of the limited cache space, the function instance uses a variety of cache optimization technologies. First, for the memory cache, the function instance uses the Least Recently Used (LRU) strategy for replacement. LRU is based on the principle of temporal locality of data and always eliminates the data that has not been used for the longest time. In practice, it has been proven to be a near-optimal cache replacement strategy. LRU maintains a linked list sorted by access time, moves the most recently accessed data block to the head of the linked list, and eliminates the data block at the end of the linked list when the cache misses, thus achieving The time complexity of .

[0062] Secondly, for SSD cache, function instances are replaced using the Least Frequently Used (LFU) strategy. Compared with LRU, LFU focuses more on the access frequency of data rather than the time, so it is more suitable for use in scenarios where the access pattern is relatively stable. LFU maintains a minimum heap sorted by access frequency and removes the data blocks with the lowest access frequency from the heap, which also achieves The time complexity of .

[0063] However, simply applying LRU and LFU strategies independently is often difficult to adapt to the dynamic changes of actual workloads. To this end, the present invention proposes an adaptive LRU-LFU hybrid cache strategy. This strategy uses an adaptive weight parameter , dynamically balances the contribution of LRU and LFU, weighing the impact of temporal locality and frequency locality. Specifically, for each data block , define its cache priority as follows: in, For data blocks The last visit time of The frequency of its access. The value of is adaptively adjusted according to the cache hit rate and cache conversion rate in the recent period, reflecting the changing trend of workload: in, and are the cache hit rate and miss rate in the recent period, and are the number of cache blocks hit by time and frequency in the recent period, respectively. This adaptive weighting strategy enables the cache manager to dynamically optimize cache replacement decisions based on the real-time characteristics of the workload to achieve the best balance between time locality and frequency locality.

[0064] While managing the cache, function instances also need to be efficient

[0065] The invention can synchronize data between the local cache and the P2P network. On the one hand, the function instance needs to synchronize the updated model blocks in the P2P network to the local cache in time to ensure data consistency. On the other hand, when the local cache misses, the function instance needs to quickly obtain the required model blocks from the P2P network and cache them locally. In order to optimize this synchronization process, the invention introduces a number of innovative mechanisms.

[0066] First, the function instance uses an incremental synchronization mechanism based on the Merkle tree to minimize the overhead of data transmission. Specifically, each model block in the P2P network corresponds to a Merkle tree node, the leaf node stores the hash of the model block, and the non-leaf node stores the hash of its child node hash. By comparing the local cache and the Merkle tree root hash of the P2P network, the function instance can quickly determine which model blocks have changed and synchronize only these changed blocks without repeatedly transmitting unmodified data. This incremental synchronization mechanism of the Merkle tree greatly reduces network bandwidth consumption and improves data synchronization efficiency.

[0067] Secondly, the function instance adopts a pre-fetch and pre-heat strategy, which reduces the latency caused by cache misses and cold starts by actively pre-loading and pre-executing. Based on the recent call history and current resource utilization, the function instance predicts the model blocks that may be accessed in the future, obtains them from the P2P network in advance, and caches them locally. In addition, for newly loaded model blocks, the function instance will pre-process them in the background (such as format conversion, compression and decompression, etc.), and trigger some initialization logic (such as tensor allocation, operator compilation, weight loading, etc.) in advance to keep the model in a "hot" state that can be used at any time. Through this predictive cache management, the function instance largely masks the communication latency of the P2P network and the cold start overhead of the Serverless environment, and realizes on-demand loading of the model in seconds.

[0068] Finally, in order to further improve the cache hit rate across function instances, the present invention also introduces the idea of ​​collaborative caching. Traditional function instance caches are usually independent and closed, and cannot share each other's cached data. Collaborative caching allows function instances in the same Serverless cluster or region to share locally cached model blocks with each other. When a function instance misses the local cache, it first checks whether other function instances in the same cluster or region have cached the required model blocks. If found, it reads data directly from the other party's cache instead of accessing the P2P network. Only when all collaborative caches in the cluster miss, will the P2P network be used to obtain the model block. Through this distributed, collaborative caching mechanism, function instances connect each other's memory and storage resources into a unified large cache pool, play the role of caching in a larger range, and improve the cache hit rate of the entire cluster.

[0069] In addition to multi-level caching and intelligent synchronization, function instances further utilize the semantic information of the model blocks after segmentation to optimize the cached data organization and retrieval strategy. Traditional cache systems mainly index and query based on the address or ID of the data, and rarely consider the characteristics of the data content itself. In the present invention, since the model blocks are semantically segmented, they naturally carry rich semantic tags, which makes semantically aware cache management possible.

[0070] When a function instance caches a model block, it will synchronize the semantic label of the cache block and build an inverted index based on the label. When a function instance needs to load a model block with specific semantics (such as a convolutional layer, LSTM layer, etc.), it can directly query all related model blocks through the inverted index, avoiding the overhead of a full table scan. This semantic index changes the retrieval of model blocks from address-related to content-related, greatly improving the query and filtering efficiency of cached data.

[0071] In addition, when retrieving model blocks in a P2P network, the function instance no longer simply performs multiple independent random searches, but instead fully utilizes the semantic relevance of model blocks to batch-acquire semantically similar model blocks. Since semantically similar model blocks are usually close to each other in the vector space, and the corresponding node IDs are also closer in XOR distance, routing and acquiring them in batches not only reduces the number of network round trips, but also improves the efficiency and success rate of routing. The function instance uses a semantic similarity search algorithm based on Locality Sensitive Hashing (LSH) to quickly find a batch of candidate blocks that are most relevant to the query semantics in a large number of model blocks, and then further refines the results through the Approximate Nearest Neighbor (ANN) algorithm, achieving highly relevant and diverse semantic retrieval within sublinear time complexity.

[0072] Through semantically aware cache organization and query, function instances enable efficient flow and sharing of model blocks in cache and network. This not only speeds up the model loading process of a single function instance, but also promotes cache collaboration and reuse between different function instances, playing the role of cache in a higher dimension.

[0073] In summary, this embodiment combines multiple optimization mechanisms such as multi-level caching, adaptive replacement, incremental synchronization, collaborative sharing, and semantic perception to build a highly intelligent, efficient, and scalable model caching system. This caching system perfectly matches the stateless and elastic characteristics of Serverless computing, providing data access performance close to local persistent storage while maintaining the lightweight and isolation of Serverless functions, greatly simplifying the complexity of model management and loading in Serverless scenarios.

[0074] In actual applications, this caching mechanism can greatly accelerate the startup and execution of various AI applications, reduce the cold start time from minutes to seconds, and increase the throughput by several to dozens of times. For applications that need to frequently load and switch models, such as online reasoning services, multi-task learning, adaptive decision-making, etc., the advantages of this caching mechanism will be more obvious. It provides a high-performance, low-cost, and easy-to-manage model storage and access platform for these applications, allowing developers to focus on the application logic itself without having to worry about the scheduling and handling of the underlying model. At the same time, this caching mechanism also paves the way for further exploration of the serverless, lightweight, and democratization of AI applications, and is expected to become a key component of the next generation of AI application infrastructure.

[0075] Example 3: This example provides a blockchain-based AI model version management and auditing method. Figure 7As shown, this method introduces blockchain technology to achieve traceable and tamper-proof recording and management of AI model versions.

[0076] In this embodiment, the AI ​​model management module is integrated with the blockchain network to jointly build a distributed model version control system. When a user submits a new model version, the management module first semantically segments and encrypts the model to generate a new version of the model block. Then, the management module packages the new version's meta information (such as version number, submission time, modified content, etc.) and the hash summary of the model block into a version update transaction and submits it to the blockchain network.

[0077] The blockchain network adopts a consortium chain architecture, which consists of multiple trusted nodes, and verifies and confirms transactions through the Byzantine FaultTolerance (BFT) consensus algorithm. The BFT algorithm can tolerate less than one-third of malicious nodes, ensuring the consistency and tamper-proofness of the blockchain. When the version update transaction passes the consensus verification, the transaction is packaged into a new block and linked to the end of the blockchain.

[0078] In this way, every update of the model version is recorded on the blockchain, forming an unchangeable version change history. The chain structure and hash pointer mechanism of the blockchain ensure the integrity and unchangeability of the historical records. Any modification to the past version will cause the hash value of the subsequent block to become invalid, thus being quickly discovered.

[0079] When a function instance needs to load a specific version of an AI model, it first queries the blockchain network for the meta information and hash summary of that version. The function instance then retrieves the model block of the corresponding version from the P2P network and uses the hash summary to verify the integrity and consistency of the model block. Only model blocks that pass verification will be loaded and used, ensuring the credibility and security of the model.

[0080] Blockchain not only provides traceable version management, but also supports fine-grained model access control. Each function instance has a unique blockchain account, identified by the account address. Model owners can define access control policies on the blockchain to specify which accounts can access which versions of the model. When a function instance requests access to a version, the blockchain smart contract automatically checks its account permissions to ensure that only authorized instances can load and use the model.

[0081] Compared with the traditional centralized version management system, the blockchain-based solution has the following advantages: First, the distributed nature and consensus mechanism of the blockchain ensure the credibility and immutability of the version history, preventing single point failures and malicious tampering; second, the blockchain's smart contracts support flexible access control strategies and realize fine-grained permission management; finally, the combination of blockchain and P2P storage realizes the decentralization of model management and storage, improving the availability and robustness of the system.

[0082] In summary, this embodiment builds a secure, reliable, and fine-grained AI model version management and audit system by introducing blockchain technology, providing stronger version traceability and access control capabilities for AI applications in the Serverless environment.

[0083] Embodiment 4: This embodiment provides a comprehensive application scenario, demonstrating the collaborative working process of various technical modules in the above embodiments.

[0084] In this scenario, the user uploads a new version of the speech recognition model through the web interface. After receiving the upload request, the AI ​​model management module first converts and standardizes the model format and extracts the model's structure, parameters and other meta-information. Then, the semantic segmentation engine segments the model and divides it into multiple fine-grained semantic blocks based on graph embedding and spectral clustering algorithms. Next, the security manager independently encrypts each semantic block using the key provided by the user.

[0085] The encrypted model block is pushed to the P2P network and stored by the Serverless worker node. At the same time, the model's metadata and hash summary are packaged into a version update transaction, submitted to the blockchain network, confirmed by the BFT consensus algorithm, and a new block is generated.

[0086] When a Serverless function instance of a speech recognition application receives a speech recognition request from a user, it first queries the blockchain network for the latest version of the speech recognition model. After the permission verification is passed, the function instance obtains the version information and hash summary of the model.

[0087] Next, the function instance retrieves the required semantic blocks in parallel through the P2P network. The multi-level intelligent cache dynamically chooses to load semantic blocks from memory, disk, or P2P network according to the local cache status of semantic blocks. The semantic-aware ANN algorithm accelerates the retrieval process of similar semantic blocks.

[0088] The function instance decrypts the retrieved semantic blocks and reconstructs them into a complete speech recognition model. The decrypted model is loaded into the secure sandbox of the function instance to process the user's speech recognition request. The recognition result is returned to the user through the API, and the entire process is completed within milliseconds.

[0089] During the application operation, the performance analysis module continuously monitors the key indicators of the system, including model loading delay, throughput, cache hit rate, etc. Once a performance bottleneck or anomaly is found, the analysis module will generate an alarm, triggering automatic scaling and fault recovery mechanisms to ensure the continuous availability of the application.

[0090] This scenario demonstrates the collaborative work of the various modules of the present invention, and reflects the advantages of the AI ​​model management and loading system based on the Serverless architecture in practical applications: the model upload and update process is simple and efficient, cutting and encryption ensure the security of the model, P2P storage and intelligent caching provide a high-performance model loading experience, and the addition of blockchain further strengthens version management and permission control. This decentralized architecture fully utilizes the elasticity and scalability of Serverless computing, while providing security, reliability, and efficiency that are difficult to achieve with traditional AI platforms.

[0091] Conclusion The AI ​​model management and secure loading system based on Serverless architecture provided by the present invention effectively solves the problems of inefficiency and insecurity of AI model loading in Serverless environment through innovative technologies such as semantic segmentation, P2P distributed storage, multi-level intelligent caching, blockchain version management, etc., and significantly improves the performance, security, scalability and manageability of AI applications, and has broad application prospects.

[0092] The technical solution of the present invention is not only applicable to the Serverless computing platform, but can also be extended to other computing environments such as edge computing and mobile terminals. In addition, the model management and loading mechanism of the present invention is not limited to a specific AI model type, but can be applied to various models such as deep learning, machine learning, and knowledge graphs.

[0093] For those skilled in the art, many improvements and modifications can be made without violating the principles of the present invention, all of which fall within the scope of protection of the present invention. For example, the present invention can be combined with virtualization, edge computing and other technologies to further expand the application scenarios and effects of intelligent scheduling.

Claims

1. An efficient AI model management and secure loading system based on Serverless architecture, characterized by: The system includes: The AI ​​model management module deployed on the Serverless computing platform serves as a unified entry point for AI models. It is used to efficiently semantically segment uploaded AI models and generate multiple independently encrypted fine-grained model blocks to improve subsequent loading efficiency. A high-performance P2P network built using a Serverless cluster is used to distribute and store encrypted AI model blocks. The P2P network introduces Kademlia hash tables for fast routing, uses Reed-Solomon erasure codes for high availability, and uses the UDT transmission protocol to support high-throughput and low-latency transmission. The function instance loads the AI ​​model block with high concurrency by accessing the P2P network, and builds a multi-level LRU / LFU intelligent cache locally to achieve efficient dynamic caching and pre-fetching of the AI ​​model block.

2. The system according to claim 1, characterized in that The AI ​​model management module uses semantic version specifications to manage AI models, achieves consistent incremental model updates through an efficient transaction mechanism, and minimizes model loading overhead.

3. The system according to claim 1, characterized in that When performing semantic segmentation on the AI ​​model, the AI ​​model management module comprehensively considers the model structure and historical access patterns, adopts a heuristic algorithm to achieve optimal segmentation, divides the model into fine-grained model blocks, and fully utilizes the potential for parallel loading.

4. The system according to claim 1, characterized in that The nodes in the P2P network evaluate the heat of the AI ​​model blocks and use an intelligent scheduling algorithm to achieve load balancing. High-heat model blocks are pushed to more nodes to accelerate concurrent access.

5. The system according to claim 1, characterized in that The P2P network adopts a lightweight security protocol based on homomorphic encryption, which provides end-to-end encryption protection for AI model blocks while reducing encryption and decryption overhead and ensuring model loading performance.

6. The system according to claim 1, characterized in that The smart cache built locally by the function instance adopts a multi-level design that mixes memory and disk, and introduces a machine learning model to adaptively optimize the cache strategy, maximize the cache hit rate, and speed up the model loading speed.

7. The system according to any one of claims 1 to 6, characterized in that: The system also includes a performance analysis module for real-time tracking and analysis of end-to-end key performance indicators, including model loading latency, throughput, cache hit rate, etc., to achieve rapid location of performance bottlenecks and optimization decisions.

8. An efficient AI model management and secure loading method based on the system according to any one of claims 1 to 7, characterized in that: The method comprises the following steps: The uploaded AI model files are received through the AI ​​model management module, and the heuristic cutting algorithm is used to generate fine-grained independent encrypted model blocks to improve the loading parallelism; Use high-performance P2P networks to distribute encrypted model blocks and optimize P2P networks for fast routing, high availability, and low-latency transmission; Function instances load model blocks with high concurrency through the P2P network and build multi-level intelligent cache locally to achieve efficient dynamic caching; Incremental update transactions are used to support lightweight model updates and minimize version switching overhead.

9. The method according to claim 8, characterized in that When distributing AI model blocks in a P2P network, the Gossip protocol is used between nodes to synchronize model block metadata, and the required model blocks are quickly found based on Bloom filters.

10. The method according to claim 8, characterized in that When caching AI model blocks locally, the temporal locality and access frequency of the model blocks are comprehensively considered, and the space allocation and elimination strategy of the multi-level cache are dynamically adjusted to improve cache utilization.

Images