Elastic file preheating and distribution method and system for multi-machine room large model

Abstract

The application discloses an elastic file preheating and distribution method for a multi-machine room large model, which comprises the following steps: collecting basic resource information and model file resource information of each machine room through a global resource sensing scheduler; obtaining preheating task instructions corresponding to each machine room based on the model file resource information and the running condition of the corresponding machine room through a model-resource matching strategy; based on the preheating task instructions, scanning a model cache state, preferentially selecting a same-machine-room node to transmit a model file through a P2P network, and performing storage cleaning according to a least recently used algorithm or a use frequency strategy; intercepting an access request of an inference instance to a model file through a unified file access module, and based on the position information of the model file, redirecting the access request to a P2P cache node or a backend storage device of the machine room through a routing strategy, so that transparent access to the underlying storage difference is realized.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and cloud computing technology, and in particular relates to a method and system for elastic file preheating and distribution for large multi-data center models. Background Technology

[0002] Currently, there are three main modes for managing large model files across multiple data centers: The first is the startup-download synchronization mode, where files are directly pulled from the cloud upon service startup, resulting in extremely long startup delays and wasted bandwidth due to repeated downloads; the second is the static pre-provisioning or centralized storage mode, which pre-distributes files manually or through simple scripts, or relies on shared storage such as NAS, but cannot intelligently match heterogeneous computing power and storage resources, lacks replica elasticity and storage space management, and is prone to resource waste or single points of failure; the third is the general P2P file distribution system, which can accelerate transmission, but is completely decoupled from application-layer scheduling, and cannot intelligently preheat and place files based on business needs and resource profiles. The limitation of the above solutions is that file distribution, storage management, and computing task scheduling are disconnected, resulting in insufficient efficiency, elasticity, and reliability of the system, and making it unable to meet the stringent requirements of large model services for rapid startup, high availability, and intelligent resource collaboration. Summary of the Invention

[0003] To address the aforementioned issues, this invention proposes a method and system for elastic file preheating and distribution of large models across multiple data centers. By constructing a priority P2P distribution network within the data center, model files are transmitted between nodes, avoiding duplicate downloads across all nodes and saving data center outbound bandwidth. This application schedules inference service instances to nodes that have cached the required model files, enabling the service to directly read local files upon startup, reducing the download wait time from tens of minutes to seconds, thus achieving hot start of the model. The global resource-aware scheduler in this application can dynamically generate differentiated preheating strategies based on the heterogeneous resources of the data center. Through a dynamic data center management module and a prediction mechanism based on historical traffic, it responds to changes in data center online / offline status, expansion, and business traffic, adjusting preheating tasks and replica targets in real time. This achieves adaptive resource matching and capacity planning, reducing complexity and manual intervention costs. A multi-replica guarantee and self-healing mechanism continuously scans and maintains the target number of replicas for each model in each data center. When a replica is missing, a supplementary task is automatically triggered (preferably via P2P), ensuring the persistent availability of critical model files, effectively avoiding single points of failure, and improving service robustness and service level agreement compliance. By providing a unified file access layer, P2P cache nodes or backend storage devices are provided to the upper-layer inference engine, offering a fully decoupled and consistent file system view, simplifying the complexity of application development and deployment, and improving the user experience.

[0004] A first aspect of the present invention provides a flexible file preheating and distribution system for large multi-datacenter models, comprising: A global resource-aware scheduler is used to collect basic resource information and model file resource information of each data center. Based on the basic resource information and the operating status of each data center, a preheating model list adapted to the data center is generated and the data center change events are managed through a model-resource matching strategy. The distributed data center model preheating controller is used to receive the preheating model list, control the nodes of the corresponding data center to perform preheating download of model files, monitor the storage resource usage of the corresponding data center nodes in real time and provide cleanup strategies, and maintain a multi-copy guarantee mechanism for model files, so that each model file has at least a preset number of node caches in the data center; The P2P distribution network module is used to establish networks within and between data centers, and to provide model files to nodes by selecting strategies. The unified file access module provides a unified file mounting interface, intercepts inference instance access requests to model files, and redirects the access requests to local P2P cache nodes or backend storage devices.

[0005] Preferably, the cleanup strategy is as follows: if the storage utilization rate of the corresponding data center exceeds a preset threshold, a cleanup strategy based on the least recent use or model usage frequency is executed.

[0006] A second aspect of the present invention provides a method for elastic file preheating and distribution for large multi-datacenter models, applied to the elastic file preheating and distribution system for large multi-datacenter models described in any of the preceding claims, comprising: The global resource awareness scheduler collects basic resource information and model file resource information for each computer room. Based on the resource information in the model file and the operational status of the corresponding computer room, the preheating task instructions for each computer room are obtained through a model-resource matching strategy. Based on the preheating task instructions, the storage cleanup is performed by scanning the model cache status, prioritizing the transmission of model files to nodes in the same data center via the P2P network, and according to the least recently used algorithm or usage frequency strategy. By intercepting the inference instance's access requests to the model file through the unified file access module, and redirecting the access request to the P2P cache node in the data center or the backend storage device through the routing strategy based on the location information of the model file, transparent access to the underlying storage differences is achieved.

[0007] Preferably, the steps of the model-resource matching strategy further include: Obtain the current GPU model of the data center node, determine the corresponding computing power specification based on the GPU model, and match the large model to be preheated to the corresponding model size based on the computing power specification. Obtain the physical deployment location or network environment attributes of the data center, and select an appropriate storage backend strategy from the preset strategies based on the physical deployment location or network environment attributes.

[0008] Preferably, the step of selecting an appropriate storage backend strategy from preset strategies based on the physical deployment location or network environment attributes further includes: If the data center is a cloud data center, network-attached storage is used as the storage backend strategy. If the data center is an offline physical data center, a strategy combining disk storage and P2P distribution is adopted as the storage backend strategy.

[0009] Preferably, the step of obtaining the preheating task instruction corresponding to each data center through a model-resource matching strategy based on the model file resource information and the corresponding data center operation status further includes: Obtain historical traffic data and planned data for inference model services to be launched in the data center; Based on the historical traffic data and the inference model service plan data, the number of predictive model service replicas for each data center is obtained through prediction. The target number of file copies in the preheating task instruction is adjusted based on the number of service copies provided by the prediction model.

[0010] Preferably, the steps of scanning the model cache status, prioritizing the transfer of model files to nodes in the same data center via P2P network, and performing storage cleanup based on the least recently used algorithm or usage frequency strategy further include: Based on the preheating task instruction, a download request is initiated through the data center node to obtain the model file. The data center node executes the source node selection strategy through the P2P distribution network. The source node selection strategy is as follows: firstly, other nodes in the same data center that have cached the model file are selected as download sources, and secondly, cross-data center nodes with the best network are selected. The model file transmission process supports resuming interrupted downloads. After the model file transfer is completed, the integrity of the model file and the number of copies of the model file are checked.

[0011] Preferably, the step of verifying the number of copies of the model file further includes: The system periodically scans the model cache status of each node in the data center to count the current number of copies of each model file. If the current number of copies is less than the preset target number of copies, it obtains the target node in the data center to accommodate the copies of the model file, and creates a download task for the model file on the target node to supplement the number of copies until the preset target number of copies is reached.

[0012] Preferably, the step of redirecting the access request to the P2P caching node in the data center or the backend storage device through a routing policy further includes: Determine whether the local node where the inference instance that initiated the access request is located has cached the target model file; if it has been cached, route the access request to the disk of the node that has cached the target model file for reading; if it has not been cached, route the access request to the P2P node or backend storage device in the data center that has cached the target model file, and cache the read target model file to the local node disk.

[0013] A third aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the elastic file warm-up and distribution method for a large multi-datacenter model as described in any of the preceding claims.

[0014] Because the present invention adopts the above technical solution, it has the following advantages and positive effects compared with the prior art: By constructing a priority P2P distribution network within the data center, model files are transmitted between nodes, avoiding duplicate downloads across all nodes and saving data center outbound bandwidth. This application schedules inference service instances to nodes that have cached the required model files, enabling the service to directly read local files upon startup, reducing the download wait time from tens of minutes to seconds, and achieving hot start of the model.

[0015] The global resource-aware scheduler in this application can dynamically generate differentiated preheating strategies based on the heterogeneous resources of the data center. Through a dynamic data center management module and a prediction mechanism based on historical traffic, it can respond to changes in data center online / offline status, expansion, and business traffic, and adjust preheating tasks and replica targets in real time. This achieves adaptive resource matching and capacity planning, reducing complexity and manual intervention costs.

[0016] The multi-replica guarantee and self-healing mechanism continuously scans and maintains the target number of replicas for each model in each data center. When a replica is missing, a replacement task is automatically triggered (preferably via P2P) to ensure the persistent availability of critical model files, effectively avoid single points of failure, and improve the robustness of the service and the service level agreement.

[0017] By providing a unified file access layer, P2P cache nodes or backend storage devices are provided to the upper-layer inference engine, offering a fully decoupled and consistent file system view, simplifying the complexity of application development and deployment, and improving the user experience. Attached Figure Description

[0018] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is an embodiment of the storage backend strategy selected for adaptation in this invention; Figure 2This is a flowchart of the elastic file preheating and distribution method for large multi-datacenter models in this invention. Detailed Implementation

[0019] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The advantages and features of the present invention will become clearer from the following description and claims. It should be noted that the drawings are all in a very simplified form and use non-precise ratios, and are only used to facilitate and clarify the illustration of the embodiments of the present invention.

[0020] It should be noted that all directional indicators (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0021] First Embodiment A first aspect of the present invention provides a flexible file preheating and distribution system for large multi-datacenter models, comprising: The global resource awareness scheduler is used to collect basic resource information and model file resource information for each data center. Based on the basic resource information and the operating status of each data center, it generates a list of preheating models that are adapted to the data center and manages the data center's change events through a model-resource matching strategy. The distributed data center model preheating controller is used to receive the preheating model list, control the nodes in the corresponding data center to perform preheating download of model files, monitor the storage resource usage of the corresponding data center nodes in real time and provide cleanup strategies, and maintain a multi-copy guarantee mechanism for model files, so that each model file has at least a preset number of node caches in the data center; The P2P distribution network module is used to establish networks within and between data centers, and to provide model files to nodes by selecting strategies. The unified file access module provides a unified file mounting interface, intercepts inference instance access requests to model files, and redirects the access requests to local P2P cache nodes or backend storage devices.

[0022] The global resource-aware scheduler collects basic resource information and model file resource information from all data centers. Combining this with the real-time operational status of each data center, it makes intelligent decisions using a model-resource matching strategy to generate a preheating model list adapted to the needs of each data center and manages data center change events uniformly. Next, a distributed data center model preheating controller executes in each data center, with the following rules: receiving the preheating list and controlling data center nodes to preheat and download model files; monitoring node storage utilization in real time, dynamically managing storage space through cleanup strategies, and ensuring that model files within the data center always maintain a preset number of copies through a multi-replica guarantee mechanism, achieving elastic resource scaling. During transmission, a P2P distribution network module is used to construct the network topology within and across data centers, providing file sources to nodes according to a selection strategy (prioritizing data centers within the same data center) to achieve fast point-to-point transmission. When an inference instance needs to read a model, the unified file access module intercepts the access request and redirects the request according to the file's location information using a routing strategy.

[0023] By pre-distributing model files to target nodes using a model-resource matching strategy, the model is ready locally when the inference instance starts, significantly reducing download waiting time. Through real-time monitoring and cleanup strategies, the system automatically cleans up cold data to free up storage space while ensuring the availability of hot models, allowing limited disk resources to support the transmission and storage of more model files and improving storage utilization. Maintaining a multi-replica guarantee mechanism ensures that multiple nodes in the data center always have model replicas, avoiding single points of failure and effectively distributing I / O and network pressure on the backend storage in conjunction with P2P distribution. Through the interception and redirection of the unified file access module, inference instances in the business layer can enjoy acceleration and caching services without modifying code, achieving transparent access to the underlying heterogeneous storage.

[0024] Preferably, the cleanup strategy is as follows: if the storage utilization rate of the corresponding data center exceeds a preset threshold, a cleanup strategy based on the least recent use or model usage frequency is executed.

[0025] The system continuously monitors the storage resource usage of the data center nodes. When storage utilization exceeds a preset threshold, a cleanup process is automatically triggered. Algorithmic logic is used to assess the value of currently cached model files: the Least Recently Used algorithm identifies cold data that has not been accessed for a long period. A frequency strategy identifies low-frequency models with the lowest historical access count. These identified low-value files are prioritized for deletion, freeing up their occupied storage space until storage utilization returns to a safe level.

[0026] Second Embodiment See Figure 2A second aspect of the present invention provides a method for elastic file preheating and distribution for large multi-datacenter models, applied to the elastic file preheating and distribution system for large multi-datacenter models according to any one of the above claims, comprising: The global resource awareness scheduler collects basic resource information and model file resource information for each computer room. Based on the resource information in the model file and the operational status of the corresponding data center, the preheating task instructions for each data center are obtained through a model-resource matching strategy. Based on the preheating task instructions, the storage cleanup is performed by scanning the model cache status, prioritizing the transmission of model files to nodes in the same data center via P2P network, and according to the least recently used algorithm or usage frequency strategy. By intercepting inference instance access requests to model files through a unified file access module, and redirecting access requests to P2P cache nodes in the data center or backend storage devices based on the location information of the model files through routing strategies, transparent access to underlying storage differences is achieved.

[0027] First, a full resource view of basic resource information and model file resource information is established through a global resource-aware scheduler. Then, combined with the real-time operating status of each data center, a model-resource matching strategy is used for calculation and analysis. This process is equivalent to "tailor-making" warm-up task instructions for each data center, ensuring the accuracy of scheduling decisions. During the execution of the warm-up task, a strategy of parallel scanning and transmission is adopted. On the one hand, leveraging the characteristics of the P2P network, priority is given to finding nodes within the same data center for file transmission, shortening the transmission path; on the other hand, during transmission and storage, continuous monitoring and dynamic execution of storage cleanup are performed using least recently used or frequency strategies. This process enables rapid data flow within the data center and adaptive updates of storage resources. When inference instances at the business layer initiate requests, they are intercepted through a unified file access module. The system does not directly allow the business layer to access the underlying storage, but dynamically executes routing strategies based on the model file location information. This process decouples logical access from physical storage location, automatically redirecting to the optimal data source. By prioritizing transmission to nodes within the same data center through the P2P network, the system maximizes the utilization of the data center's intranet bandwidth, reduces expensive cross-data center backbone network traffic consumption, and significantly reduces file transmission latency. By combining an automatic cleanup mechanism based on the Least Recently Used (LRU) algorithm or usage frequency strategy, limited storage space is always occupied by hot data, while cold data is evicted. This not only prevents storage overflow but also improves the return on investment of storage resources. Transparent access through a unified file access module allows upper-layer inference instances to be unaware of whether the underlying system is local disk, P2P cache, or backend object storage. Business code can enjoy acceleration and scheduling services without modification, reducing system migration costs and operational complexity. Preheating task instructions generated based on resource information and operational status ensure that model files are ready before business needs, avoiding immediate download waits during inference startup and improving the startup speed and overall throughput of the inference service.

[0028] Preferably, the steps of the model-resource matching strategy further include: Obtain the current GPU model of the data center node, determine the corresponding computing power specification based on the GPU model, and match the large model to be preheated to the appropriate model size based on the computing power specification. Obtain the physical deployment location or network environment attributes of the data center, and select the appropriate storage backend strategy from the preset strategies based on the physical deployment location or network environment attributes.

[0029] First, the hardware information of the data center nodes is analyzed, and their computing power specifications are quantified by GPU model. Based on the hardware's capacity, large models to be warmed up are matched to appropriate model sizes. This bottom-up hardware adaptation process ensures alignment between computing tasks and physical resources. The physical deployment location or network environment attributes of the data center are further identified. A suitable storage backend strategy is dynamically selected from a pre-defined strategy library. This is a strategy selection process based on infrastructure characteristics. By precisely matching the model size with the GPU computing power specifications, large models are avoided from being assigned to nodes with insufficient computing power, preventing OutOfMemoryError (OOM) errors, avoiding resource waste, and ensuring the smooth execution of warm-up tasks and maximizing the utilization of computing resources. The optimal storage strategy is automatically selected based on the data center environment, fully utilizing the hardware advantages of different environments, significantly improving model file reading speed, and reducing inference service startup latency. It is seamlessly compatible with hybrid cloud architectures. Whether in a cloud network environment or a restricted on-premises network environment, it can automatically adapt to the best storage solution without manual configuration intervention.

[0030] See Figure 1 Preferably, the step of selecting an appropriate storage backend strategy from preset strategies based on physical deployment location or network environment attributes further includes: If the data center is a cloud-based data center, network-attached storage should be used as the storage backend strategy. If the data center is an offline physical data center, a strategy combining disk storage and P2P distribution is adopted as the storage backend strategy.

[0031] When the data center is identified as a cloud-based data center, Network Attached Storage (NAS) is selected as the backend, leveraging the high bandwidth and storage service capabilities of the cloud platform. Model files are centrally stored in a shared file system, and the elasticity of the cloud network allows each computing node to mount on demand, achieving centralized data management and high-concurrency reads. When the data center is identified as an on-premises physical environment, considering that on-premises physical data centers typically have high-performance local disks and that cross-regional bandwidth is expensive or limited, disk storage combined with P2P distribution is selected. The principle is to prioritize the use of the physical machine's local disk for high-speed read and write operations, and distribute data within the local area network through P2P technology, reducing dependence on centralized storage and external networks.

[0032] Preferably, the step of obtaining the preheating task instructions corresponding to each data center through a model-resource matching strategy based on model file resource information and the corresponding data center operation status further includes: Obtain historical traffic data and planned data for inference model services to be launched in the data center; Based on historical traffic data and inference model service plan data, the number of predictive model service replicas for each data center is obtained through prediction. Adjust the target number of file copies in the preheating task instruction based on the number of service replicas using the prediction model.

[0033] Two types of key data were collected: historical traffic data and planned data for inference model services to be launched. This data was input into a prediction algorithm to calculate the number of predictive model service replicas required by each data center to meet service demand within a specific future time period. The prediction results were directly applied to the preheating task command, dynamically adjusting the target number of file distribution replicas. If a traffic peak was predicted, the preheating task command automatically increased the target number of replicas; if a low traffic peak was predicted, the number of replicas was decreased. By predicting traffic peaks in advance and preheating sufficient replicas, it was ensured that the model files were ready on the nodes before the surge in business requests, shortening the startup response time of the inference service and avoiding performance fluctuations caused by waiting for downloads during capacity expansion.

[0034] Preferably, the steps of scanning the model cache status, prioritizing the transfer of model files to nodes in the same data center via P2P network, and performing storage cleanup based on the least recently used algorithm or usage frequency strategy further include: Based on the preheating task instruction, a download request is initiated through the data center node to obtain the model file. The data center node executes the source node selection strategy through the P2P distribution network. The source node selection strategy is as follows: firstly, other nodes in the same data center that have cached the model file are selected as the download source, and secondly, the cross-data center node with the best network is selected. The model file transmission process supports resuming interrupted downloads. After the model file transfer is complete, the integrity of the model file and the number of model file copies are checked.

[0035] When a node initiates a download request based on the preheating task, the P2P distribution network does not randomly select a source, but rather follows a priority logic. First, it searches within the same data center (LAN) to see if a node has already cached the file, prioritizing the use of internal network bandwidth. If no available source exists within the same data center, it then selects the cross-data center node with the lowest network latency or optimal bandwidth as the download source. During transmission, a resume function is enabled, and the transmission progress is recorded in real time. If the network is interrupted, the download can resume from the point of interruption without starting from the beginning. After the download is complete, the integrity of the model file is first verified (e.g., hash comparison) to ensure that the data has not been tampered with or corrupted. Then, the number of replicas is verified to see if the number of replicas in the current data center has reached the system's preset target value, thus determining whether the preheating task is complete.

[0036] Preferably, the step of verifying the number of copies of the model file further includes: The system periodically scans the model cache status of each node in the data center to count the current number of copies of each model file. If the current number of copies is less than the preset target number of copies, it obtains the target node in the data center to accommodate the copies of the model file, and creates a download task for the model file on the target node to supplement the number of copies until the preset target number of copies is reached.

[0037] By setting a timed period, the system proactively scans the cache status of all nodes within the data center, continuously counting the current number of live replicas for each model file and comparing it to a preset target number of replicas. Once a model's current replica count falls below the target, it triggers the selection of target nodes with storage space and automatically creates download tasks on these nodes to retrieve model files from the source. By continuously monitoring the replenishment progress, a round of repair is completed until the replica count of all models reaches or exceeds the preset target. Maintaining the preset replica count ensures that even if some nodes fail, other nodes within the data center still hold the model, thus guaranteeing uninterrupted inference services. Automatic replica gap repair requires no manual intervention. The system can detect resource changes and automatically replenish replicas, significantly reducing the operational difficulty and workload of maintaining data consistency in complex multi-data center environments. It ensures that there are always enough nodes in the data center holding model files to handle sudden concurrent inference requests, avoiding queuing or cross-data center retrieval delays due to insufficient replicas, and ensuring the stability of the service level agreement.

[0038] Preferably, the step of redirecting access requests to the P2P caching node in the data center or the backend storage device through a routing strategy further includes: Determine whether the local node where the inference instance that initiated the access request is located has cached the target model file; if it has been cached, route the access request to the disk of the node that has cached the target model file for reading; if it has not been cached, route the access request to the P2P node in the data center that has cached the target model file or the backend storage device, and cache the read target model file to the local node disk.

[0039] By prioritizing the determination of whether a file is cached on a local node and reading it directly from the local disk, the high I / O characteristics of the local disk are fully utilized. For cached hot data, this mechanism eliminates network transmission overhead, provides the fastest model loading speed, and reduces the first-word latency of the inference service. When a local cache miss occurs, the routing policy can automatically redirect to a P2P node in the data center or a backend storage device. This multi-level lookup mechanism ensures that inference requests can successfully retrieve data regardless of the file's location, avoiding service errors or interruptions caused by nodes lacking cache, and guaranteeing the stability of business operations. By caching the read target model file to the local node's disk, the system possesses the ability to learn on the fly, automatically pulling remote data into the local storage. This transforms the original cross-network access into future local access. This not only dynamically improves the cache hit rate of subsequent requests but also automatically balances the storage load.

[0040] Third Embodiment A third aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of a method for elastic file warm-up and distribution for a large multi-datacenter model as described above.

[0041] The complex strategies for large model warm-up, scheduling, P2P distribution, and caching are transformed into instruction code that can be recognized and executed by computers, enabling them to move beyond theoretical descriptions and run on practical general-purpose computing hardware. This makes the technical solution portable, facilitating storage, replication, distribution, and rapid deployment in different environments. When the processor loads and executes the instructions of this medium, the computer-readable storage medium possesses all the capabilities of the aforementioned methods. Therefore, in practical operation, this medium can directly achieve core beneficial effects such as accelerating large model distribution, improving storage resource utilization, ensuring inference stability under high concurrency, and shielding against underlying storage differences.

[0042] In the description of this application, it should be noted that the terms "inner" and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship commonly used when the product is in use. They are used only for the convenience of describing this application and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0043] It should also be noted that, unless otherwise explicitly specified and limited, the terms "setup" and "connection" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0044] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific identification content executed by the system and device described above can be referred to the corresponding process in the foregoing method embodiments.

[0045] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. Even if various changes are made to the present invention, if these changes fall within the scope of the claims of the present invention and their equivalents, they shall still fall within the protection scope of the present invention.

Claims

1. A flexible file preheating and distribution system for large multi-datacenter models, characterized in that, include: A global resource-aware scheduler is used to collect basic resource information and model file resource information of each data center. Based on the basic resource information and the operating status of each data center, a preheating model list adapted to the data center is generated and the data center change events are managed through a model-resource matching strategy. The distributed data center model preheating controller is used to receive the preheating model list, control the nodes of the corresponding data center to perform preheating download of model files, monitor the storage resource usage of the corresponding data center nodes in real time and provide cleanup strategies, and maintain a multi-copy guarantee mechanism for model files, so that each model file has at least a preset number of node caches in the data center; The P2P distribution network module is used to establish networks within and between data centers, and to provide model files to nodes by selecting strategies. The unified file access module provides a unified file mounting interface, intercepts inference instance access requests to model files, and redirects the access requests to local P2P cache nodes or backend storage devices.

2. The flexible file preheating and distribution system for large multi-datacenter models according to claim 1, characterized in that, The cleanup strategy is as follows: if the storage utilization rate of the corresponding data center exceeds a preset threshold, a cleanup strategy based on the least recent use or model usage frequency is executed.

3. A method for elastic file preheating and distribution for large multi-datacenter models, applied to the elastic file preheating and distribution system for large multi-datacenter models as described in any one of claims 1-2, characterized in that, include: The global resource awareness scheduler collects basic resource information and model file resource information for each computer room. Based on the resource information in the model file and the operational status of the corresponding computer room, the preheating task instructions for each computer room are obtained through a model-resource matching strategy. Based on the preheating task instructions, the storage cleanup is performed by scanning the model cache status, prioritizing the transmission of model files to nodes in the same data center via the P2P network, and according to the least recently used algorithm or usage frequency strategy. By intercepting the inference instance's access requests to the model file through the unified file access module, and redirecting the access request to the P2P cache node in the data center or the backend storage device through the routing strategy based on the location information of the model file, transparent access to the underlying storage differences is achieved.

4. The method for elastic file preheating and distribution for large multi-datacenter models according to claim 3, characterized in that, The steps of the model-resource matching strategy further include: Obtain the current GPU model of the data center node, determine the corresponding computing power specification based on the GPU model, and match the large model to be preheated to the corresponding model size based on the computing power specification. Obtain the physical deployment location or network environment attributes of the data center, and select an appropriate storage backend strategy from the preset strategies based on the physical deployment location or network environment attributes.

5. The method for elastic file preheating and distribution for large multi-datacenter models according to claim 4, characterized in that, The step of selecting an appropriate storage backend strategy from a preset strategy based on the physical deployment location or network environment attributes further includes: If the data center is a cloud data center, network-attached storage is used as the storage backend strategy. If the data center is an offline physical data center, a strategy combining disk storage and P2P distribution is adopted as the storage backend strategy.

6. The method for elastic file preheating and distribution for large multi-datacenter models according to claim 3, characterized in that, The step of obtaining the preheating task instruction corresponding to each data center based on the model file resource information and the corresponding data center operation status through a model-resource matching strategy further includes: Obtain historical traffic data and planned data for inference model services to be launched in the data center; Based on the historical traffic data and the inference model service plan data, the number of predictive model service replicas for each data center is obtained through prediction. The target number of file copies in the preheating task instruction is adjusted based on the number of service copies provided by the prediction model.

7. The method for elastic file preheating and distribution for large multi-datacenter models according to claim 3, characterized in that, The steps of scanning the model cache status, prioritizing the transfer of model files between nodes in the same data center via P2P network, and performing storage cleanup based on the least recently used algorithm or usage frequency strategy further include: Based on the preheating task instruction, a download request is initiated through the data center node to obtain the model file. The data center node executes the source node selection strategy through the P2P distribution network. The source node selection strategy is as follows: firstly, other nodes in the same data center that have cached the model file are selected as download sources, and secondly, cross-data center nodes with the best network are selected. The model file transmission process supports resuming interrupted downloads. After the model file transfer is completed, the integrity of the model file and the number of copies of the model file are checked.

8. The method for elastic file preheating and distribution for large multi-datacenter models according to claim 7, characterized in that, The step of verifying the number of copies of the model file further includes: The system periodically scans the model cache status of each node in the data center to count the current number of copies of each model file. If the current number of copies is less than the preset target number of copies, it obtains the target node in the data center to accommodate the copies of the model file, and creates a download task for the model file on the target node to supplement the number of copies until the preset target number of copies is reached.

9. The method for elastic file preheating and distribution for large multi-datacenter models according to claim 3, characterized in that, The step of redirecting the access request to the P2P cache node in the data center or the backend storage device through a routing policy further includes: Determine whether the local node where the inference instance that initiated the access request is located has cached the target model file; if it has been cached, route the access request to the disk of the node that has cached the target model file for reading; if it has not been cached, route the access request to the P2P node or backend storage device in the data center that has cached the target model file, and cache the read target model file to the local node disk.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the steps of the elastic file warm-up and distribution method for a large multi-data center model as described in any one of claims 3-9.

Images