Edge cloud collaborative distribution method and device for model resources, electronic equipment and medium

Abstract

The invention discloses a side cloud collaborative distribution method and device for model resources, electronic equipment and a medium, and belongs to the technical field of computers. The method comprises the following steps: receiving a model resource acquisition request initiated by a computing node; querying a memory level cache layer based on the target identification information; under the condition that the target resource fragment does not exist in the memory level cache layer, querying a pre-cooling cache layer based on the target identification information; the pre-cooling cache layer is used for pre-storing resource fragments predicted based on resource acquisition behaviors; and under the condition that the target resource fragment does not exist in the pre-cooling cache layer, sending a model resource acquisition request to the target node. According to the method, the cache node containing the memory level cache layer is established at the adjacent position of the computing node, and the resource fragments predicted based on the resource acquisition behavior are pre-stored in the pre-cooling cache layer, so that a layered dynamic management system from hot spots to cold data is formed, and the distribution efficiency of model resources can be improved.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a method, apparatus, electronic device and medium for edge-cloud collaborative distribution of model resources. Background Technology

[0002] With the rapid development of artificial intelligence technology, especially in scenarios such as large language models, autonomous driving, and smart cities, the scale of model parameters has grown from several GB to several TB. The size of the images built to run these models is also becoming increasingly massive. In distributed training and edge computing scenarios, how to quickly and reliably distribute massive model and image resources from central storage to decentralized data storage and hundreds or thousands of computing nodes has become a key bottleneck affecting overall computing efficiency and business response speed.

[0003] The relevant technologies employ a centralized distribution model, relying on a single central image repository or model storage server. When a compute node starts a task, it pulls the necessary resources from the central node via the public network or a dedicated line. However, due to the bandwidth limitations of the central storage's egress, network congestion can easily occur when a large number of compute nodes concurrently pull resources, causing task startup delays to extend from minutes to hours. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a method, apparatus, electronic device, and medium for edge-cloud collaborative distribution of model resources, so as to improve the distribution efficiency of model resources.

[0005] Firstly, this application provides a method for edge-cloud collaborative distribution of model resources, including: Receive a model resource acquisition request initiated by a computing node, wherein the model resource acquisition request carries the target identifier information of the target resource fragment; The memory-level cache layer is queried based on the target identification information; the memory-level cache layer is the first storage space of the cache node, and the cache node is a server deployed adjacent to the computing node; If the target resource fragment does not exist in the memory-level cache layer, the pre-cooled cache layer is queried based on the target identification information; the pre-cooled cache layer is the second storage space of the cache node, used to pre-store resource fragments based on resource acquisition behavior prediction; If the target resource fragment is not present in the pre-cooled cache layer, a model resource acquisition request is sent to the target node.

[0006] This application embodiment improves the transmission speed of model resources by establishing cache nodes with memory-level cache layers in the vicinity of computing nodes; it improves cache hit rate and reduces the time required to acquire data by pre-storing resource shards based on resource acquisition behavior prediction in the pre-cooled cache layer; and it forms a hierarchical dynamic management system from hot data to cold data, which improves the distribution efficiency of model resources.

[0007] According to one embodiment of this application, the first storage space is the local memory space of the cache node; the second storage space is the local non-volatile storage medium of the cache node.

[0008] In this embodiment, by setting the first memory space as the local memory space of the cache node, data is preferentially obtained from the memory space with the highest data transfer speed, which can reduce data transfer time; by setting the second storage space in the local non-volatile storage medium of the same cache node, the cache hit rate within the same cache node can be improved, cross-node transmission can be reduced, thereby improving the distribution efficiency of model resources.

[0009] According to one embodiment of this application, the method further includes: The original model resources are fragmented to obtain multiple resource fragments; The target hash algorithm is used to perform hash calculation on each resource fragment to obtain the fragment hash identifier corresponding to each resource fragment, and the fragment hash identifier is determined as the identification information of the corresponding resource fragment.

[0010] In this embodiment, by dividing the original model resources into multiple resource fragments and using the target hash algorithm to calculate the fragment hash identifier corresponding to each resource fragment, unique identification information can be generated for each resource fragment, thereby facilitating the retrieval of resource fragments.

[0011] According to one embodiment of this application, the method further includes: The access attribute information of each resource fragment in the memory-level cache layer is periodically collected; the access attribute information includes the access time and cumulative access count of the resource fragment; Calculate the first popularity value of each resource fragment based on the access attribute information; Remove resource fragments whose first popularity value is lower than the first target eviction threshold from the memory-level cache layer.

[0012] In this embodiment, by calculating the first popularity value of each resource fragment based on the access time and cumulative access count of the resource fragment, and deleting resource fragments with the first popularity value lower than the first target eviction threshold, the most popular resource fragment can be kept in the memory-level cache layer with the fastest transmission speed, thereby improving the cache hit rate of the memory-level cache layer and thus improving the distribution efficiency of model resources.

[0013] According to one embodiment of this application, based on the formula:

[0014] Calculate the first heat value for each resource fragment; in, This indicates the first popularity value. This indicates the cumulative number of accesses to a resource shard within the target time window. Weighting factors representing the number of visits. Indicates the current time. This indicates the access time of the resource shard most recent to the current time. This represents the weighting factor for the time decay term.

[0015] In this embodiment, by calculating the first popularity value based on the cumulative number of accesses to the resource shard within the target time window and the most recent access time, the judgment can be made by comprehensively considering the historical total popularity and the recent popularity, so that the obtained first popularity value can more accurately reflect the current popularity of the resource shard.

[0016] According to one embodiment of this application, the method further includes: Collect resource acquisition behavior data of the computing nodes within the target historical period; the resource acquisition behavior data includes the identification information, acquisition time and acquisition frequency of each resource fragment; Predict the future acquisition probability of each resource fragment based on the resource acquisition behavior data; Send a preload request to the data source node; the preload request carries the identification information of the resource fragment whose future acquisition probability is greater than the target pre-stored threshold; Receive the resource fragments returned by the data source node and store the resource fragments returned by the data source node in the pre-cooling cache layer.

[0017] In this embodiment, by predicting the future acquisition probability of each resource fragment based on the resource acquisition behavior data of the computing node within the target historical period, and pre-storing resource fragments with a future acquisition probability greater than the target pre-stored threshold in the pre-cooled cache layer, the cache hit rate of the pre-cooled cache layer can be improved.

[0018] According to one embodiment of this application, predicting the future acquisition probability of each resource shard based on the resource acquisition behavior data includes: Based on the resource acquisition behavior data, an acquisition frequency time series is constructed for each resource shard; the acquisition frequency time series includes the total number of acquisitions of the resource shard within different time windows; The acquisition frequency time series is input into the target prediction model to obtain the acquisition probability sequence output by the target prediction model; the acquisition probability sequence includes the probability that resource shards will be acquired in different time windows in the future.

[0019] In this embodiment, by constructing an acquisition frequency time series for each resource shard and using a target prediction model to make predictions based on the acquisition frequency time series, the probability of a resource shard being acquired in different time windows in the future can be predicted based on historical access patterns, thereby improving the accuracy of future acquisition probability prediction.

[0020] According to one embodiment of this application, the method further includes: The second heat value of each resource fragment in the pre-cooled cache layer is periodically obtained; The second heat value of each resource fragment is updated based on the decay coefficient, and resource fragments with a second heat value lower than the second target eviction threshold are deleted from the pre-cooled cache layer.

[0021] In this embodiment, by updating the second popularity value of each resource fragment based on the decay coefficient, the natural decline of access popularity can be simulated; by deleting resource fragments whose second popularity value is lower than the second target eviction threshold, the situation where outdated resource fragments occupy the pre-cooled cache layer space for a long time can be reduced, thereby improving the cache hit rate of the pre-cooled cache layer.

[0022] According to one embodiment of this application, based on the formula:

[0023] Update the second popularity value of each resource fragment; in, This represents the second heat value after the resource shard i is updated. This represents the second most popular value of resource shard i before the update. Indicates the attenuation coefficient. This is the base value for popularity.

[0024] In this embodiment, by setting a base value for popularity, the situation where the popularity value of resource fragments that have not been accessed for a long time but were once very popular can be reduced too quickly, thereby reducing cache oscillation.

[0025] According to one embodiment of this application, sending the model resource acquisition request to the target node includes: A candidate node set is constructed based on the location of the compute node; the candidate node set includes at least one of the following candidate node categories: other cache nodes within the same compute cluster as the compute node; cache nodes within different compute clusters within the same availability zone as the compute node; self-built high-speed object storage nodes in the availability zone; and central object storage nodes on the public internet. Target candidate node categories are selected from the candidate node set according to priority order; A target node is determined from the target candidate node categories, and the model resource acquisition request is sent to the target node.

[0026] In this embodiment, by constructing a candidate node set based on the location of the computing node that initiates the model resource acquisition request, and then selecting the target candidate node category from the candidate node set according to priority order and determining the target node, the situation of cross-regional data transmission can be reduced, thereby improving the distribution efficiency of model resources.

[0027] According to one embodiment of this application, determining the target node from the target candidate node category includes: Calculate the comprehensive access cost between each candidate node in the target candidate node category and the computing node; the comprehensive access cost includes at least one of network latency, bandwidth consumption cost, and access response time. The target node is determined based on the comprehensive access cost.

[0028] In this embodiment, by calculating the comprehensive access cost based on at least one of network latency, bandwidth consumption cost, and access response time, and determining the target node, a decision can be made that integrates network performance, cost, and reliability, thereby improving the accuracy of determining the target node.

[0029] Secondly, this application provides an edge-cloud collaborative distribution device for model resources, comprising: The receiving module is used to receive a model resource acquisition request initiated by the computing engine of the computing node, wherein the model resource acquisition request carries the target identifier information of the target resource fragment; The first query module is used to query the memory-level cache layer based on the target identification information; the memory-level cache layer is the local memory cache of the computing node; The second query module is used to query the pre-cooled cache layer based on the target identification information when the target resource shard does not exist in the memory-level cache layer; the pre-cooled cache layer is a storage medium deployed locally on the computing node, used to pre-store resource shards based on resource acquisition behavior prediction; The sending module is used to send the model resource acquisition request to the target node when the target resource fragment does not exist in the pre-cooled cache layer.

[0030] The edge-cloud collaborative distribution device for model resources according to this application can improve the transmission speed of model resources by establishing cache nodes with memory-level cache layers in the vicinity of computing nodes; it can improve the cache hit rate and reduce the time required to obtain data by pre-storing resource shards based on resource acquisition behavior prediction in the pre-cooled cache layer; and it forms a hierarchical dynamic management system from hot data to cold data, which can improve the distribution efficiency of model resources.

[0031] Thirdly, this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the edge-cloud collaborative distribution method for model resources as described in the first aspect above.

[0032] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the edge-cloud collaborative distribution method for model resources as described in the first aspect above.

[0033] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the edge-cloud collaborative distribution method for model resources as described in the first aspect above.

[0034] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the edge-cloud collaborative distribution method for model resources as described in the first aspect above.

[0035] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects: According to the edge-cloud collaborative distribution method of model resources in this application, by establishing cache nodes with memory-level cache layers in the vicinity of computing nodes, the transmission speed of model resources can be improved; by pre-storing resource shards based on resource acquisition behavior prediction in the pre-cooled cache layer, the cache hit rate can be improved and the time required to acquire data can be reduced; a hierarchical dynamic management system from hot data to cold data is formed, which can improve the distribution efficiency of model resources.

[0036] In some embodiments, by setting the first memory space as the local memory space of the cache node, data can be retrieved preferentially from the memory space with the highest data transfer speed, thereby reducing data transfer time; by setting the second storage space in the local non-volatile storage medium of the same cache node, the cache hit rate within the same cache node can be improved, cross-node transmission can be reduced, thereby improving the distribution efficiency of model resources.

[0037] In some embodiments, by dividing the original model resources into multiple resource fragments and using a target hash algorithm to calculate the fragment hash identifier corresponding to each resource fragment, unique identification information can be generated for each resource fragment, thereby facilitating the retrieval of resource fragments.

[0038] In some embodiments, by calculating the first popularity value of each resource fragment based on the access time and cumulative access count of the resource fragment, and deleting resource fragments with the first popularity value lower than the first target eviction threshold, the most popular resource fragments can be kept in the memory-level cache layer with the fastest transmission speed, thereby improving the cache hit rate of the memory-level cache layer and thus improving the distribution efficiency of model resources.

[0039] In some embodiments, by calculating a first popularity value based on the cumulative number of accesses to the resource shard within the target time window and the most recent access time, the judgment can be made by comprehensively considering the historical total popularity and the recent popularity, so that the obtained first popularity value can more accurately reflect the current popularity of the resource shard.

[0040] In some embodiments, by predicting the future acquisition probability of each resource fragment based on the resource acquisition behavior data of the computing node in the target historical period, and pre-storing resource fragments with a future acquisition probability greater than the target pre-stored threshold in the pre-cooled cache layer, the cache hit rate of the pre-cooled cache layer can be improved.

[0041] In some embodiments, by constructing an acquisition frequency time series for each resource shard and using a target prediction model to make predictions based on the acquisition frequency time series, the probability of a resource shard being acquired in different time windows in the future can be predicted based on historical access patterns, thereby improving the accuracy of future acquisition probability prediction.

[0042] In some embodiments, by updating the second popularity value of each resource fragment based on the decay coefficient, the natural decline of access popularity can be simulated; by deleting resource fragments whose second popularity value is lower than the second target eviction threshold, the situation where outdated resource fragments occupy the pre-cooled cache layer space for a long time can be reduced, thereby improving the cache hit rate of the pre-cooled cache layer.

[0043] In some embodiments, by setting a base value for popularity, it is possible to reduce the situation where the popularity value of resource fragments that have not been accessed for a long time but were once very popular drops to zero too quickly, thereby reducing cache oscillation.

[0044] In some embodiments, by constructing a candidate node set based on the location of the computing node that initiates the model resource acquisition request, and then selecting the target candidate node category from the candidate node set according to priority order and determining the target node, the situation of cross-regional data transmission can be reduced, thereby improving the distribution efficiency of model resources.

[0045] In some embodiments, by calculating the comprehensive cost of access based on at least one of network latency, bandwidth consumption cost, and access response time, and determining the target node, a decision can be made that integrates network performance, cost, and reliability, thereby improving the accuracy of target node determination.

[0046] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the edge-cloud collaborative distribution method for model resources provided in this application embodiment; Figure 2 This is a schematic diagram of the edge-cloud collaborative distribution device for model resources provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0050] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0051] With the rapid development of artificial intelligence and cloud computing technologies, the training of modern large-scale machine learning models often adopts a distributed architecture to reduce data transmission costs and meet data compliance requirements. For example, in the scenario of training machine learning models for autonomous driving, in addition to setting up a central computing cluster at headquarters, branches may be established in different cities or even different countries. Each branch corresponds to an availability zone, and each availability zone contains multiple computing clusters. Each computing cluster contains multiple nodes located within the same region. For example, multiple servers are set up in the same data center. Each server is equipped with computing devices such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), and NPU (Neural Processing Unit), as well as storage devices such as RAM (Random Access Memory), SSD (Solid State Drive), HDD (Hard Disk Drive), optical discs, and magnetic tapes. Servers with different configurations are responsible for completing different tasks. One server corresponds to one node. The server responsible for computing tasks is the computing node, the server responsible for storing frequently updated cached data is the cache node, the server responsible for storing less frequently updated data is the storage node, and so on.

[0052] During the model training process, the headquarters will build multiple model mirrors and distribute them to each branch. The headquarters will use raw data such as driving videos and radar data collected in the branch areas to train the model mirrors in the branch areas. The headquarters will also periodically summarize the training results of each branch to the headquarters, which will update the model parameters based on the training results and then distribute them to the computing clusters of each branch.

[0053] However, with the rapid development of artificial intelligence technology, the size of model parameters has grown from several GB to several TB, and the image size built to run these models is also becoming increasingly large. In distributed training and edge computing scenarios, how to quickly and reliably distribute massive model and image resources from central storage to decentralized data storage and hundreds or thousands of computing nodes has become a key bottleneck affecting overall computing efficiency and business response speed.

[0054] In related technologies, a centralized distribution model is adopted, relying on a single central mirror repository or model storage server. Each computing node pulls the necessary resources from the central node via the public network or dedicated line when starting a task. However, due to the bandwidth limitations of the central storage's outbound bandwidth, network congestion can easily occur when a large number of computing nodes concurrently pull resources, causing task startup delays to increase from minutes to hours. Furthermore, in cross-city distributed training, relying on dedicated lines for full transmission requires expensive dedicated line bandwidth, resulting in low cost-effectiveness.

[0055] Regarding caching strategies, current technologies employ simple Least Recently Used (LRU) or First-In-First-Out (FIFO) caching strategies on local disks of compute nodes or a few cache servers to cache previously used resources and avoid duplicate downloads. However, these simple caching strategies fail to effectively identify and differentiate between "hot data" and "predictive data," resulting in low cache hit rates and underutilization of high-speed storage media such as memory and SSDs.

[0056] When updating models, the relevant technologies rely on operations personnel to copy the required resources to the target machine in advance using scripts or manual commands before task scheduling. Model updates typically require a full redistribution or complex manual processes for differential and replacement. This process is time-consuming and bandwidth-intensive, error-prone, and cannot be scaled. It fails to meet the dynamic and elastic scheduling requirements of cloud-native environments, and compute cards often remain idle while waiting for resources, resulting in low resource utilization.

[0057] To address at least one of the aforementioned technical problems, embodiments of this application provide a method, apparatus, electronic device, and medium for edge-cloud collaborative distribution of model resources.

[0058] The edge-cloud collaborative distribution method, apparatus, electronic device, and medium for model resources provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0059] Among them, the edge-cloud collaborative distribution method of model resources can be applied to the terminal, and can be executed by the hardware or software in the terminal.

[0060] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0061] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0062] The edge-cloud collaborative distribution method for model resources provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the edge-cloud collaborative distribution method for model resources. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, wearable devices, etc. The edge-cloud collaborative distribution method for model resources provided in this application embodiment will be described below using an electronic device as the execution subject.

[0063] like Figure 1 As shown, the edge-cloud collaborative distribution method for model resources includes steps 110, 120, 130, and 140.

[0064] Step 110: Receive the model resource acquisition request initiated by the computing node. The model resource acquisition request carries the target identifier information of the target resource fragment.

[0065] In this embodiment of the application, when the computing task changes or the model version is updated, the computing node will initiate a model resource acquisition request in order to obtain the required model resources.

[0066] To facilitate storage and transmission, model resources have been pre-divided into multiple smaller resource fragments according to certain rules, and a unique identifier is generated for each resource fragment. Therefore, a model resource acquisition request carries the target identifier information for multiple target resource fragments, such as {ID_1, ID_2, ..., ID_n}, where ID_i is the identifier information for the i-th target resource fragment.

[0067] Step 120: Query the memory-level cache layer based on the target identification information; the memory-level cache layer is the first storage space of the cache node, and the cache node is a server deployed adjacent to the compute node.

[0068] In this embodiment, the cache node is a server deployed adjacent to the computing node and specifically responsible for storing cached data.

[0069] In some embodiments, proximity deployment means that cache nodes are geographically close to compute nodes, such as the physical distance between them being less than a preset value, resulting in lower latency and higher bandwidth in the network connection between them. In a distributed computing environment, compute nodes may be distributed across different geographical locations. By deploying cache nodes in locations close to compute nodes, compute nodes can quickly access the necessary model resources when performing tasks such as model inference. For example, cache node servers and compute node servers can be installed on the same rack and directly connected to the compute node servers via high-speed fiber optic cables.

[0070] In some embodiments, proximity deployment can also refer to proximity at the network topology level. For example, cache nodes and compute nodes may be located within the same edge network subnet, or connected via direct links or low-hop-count network links. There are no network devices that can cause transmission delays, such as core network forwarding nodes or cross-regional network gateways, between the cache nodes and compute nodes. This deployment method can minimize the data transmission path and reduce packet loss, latency, and jitter in the network links.

[0071] The cache node is equipped with multiple storage media, such as DRAM (Dynamic Random Access Memory), SSD, HDD, etc., and is connected to the motherboard of the cache node server through interfaces such as DIMM (Dual In-line Memory Module), M.2, PCIe (Peripheral Component Interconnect Express), and SATA (Serial Advanced Technology Attachment), enabling the cache node to manage the data in each storage medium. It can also connect the cache node server to other NVMe storage through network storage protocols such as NVMe-oF (Non-Volatile Memory Express over Fabrics), enabling the cache node server to access remote NVMe storage over the network.

[0072] The first storage space of a cache node can be determined based on the characteristics of the storage medium and the transmission speed of the interface. For example, DRAM memory modules installed directly on the motherboard memory slots via DIMM slots and SSDs installed on the motherboard via M.2 slots can be designated as the first storage space, serving as the memory-level cache layer of the cache node.

[0073] The memory-level cache layer stores multiple resource fragments with accompanying identification information. The cache node server can maintain a memory index table, recording the identification information and corresponding physical addresses of the resource fragments stored in its memory-level cache layer. Based on the received target identification information, the cache index table is searched to see if the same identification information exists. If it does, a cache hit occurs, and the data stored at the corresponding physical address is sent to the compute node that initiated the model resource retrieval request.

[0074] In some embodiments, for cache hit requests, resource fragments in the memory-level cache layer can be sent directly over the network when providing data to the compute node. For example, RDMA (Remote Direct Memory Access) technology can be used to allow the compute node to directly access the memory of the cache node and obtain data, thereby reducing multiple redundant copies of data in the user-space buffer of the operating system kernel of the cache node and thus reducing transmission latency.

[0075] Step 130: If the target resource shard does not exist in the memory-level cache layer, query the pre-cooled cache layer based on the target identifier information; the pre-cooled cache layer is the second storage space of the cache node, used to pre-store resource shards based on resource acquisition behavior prediction.

[0076] In this embodiment, the second storage space of the cache node is the storage space opposite to the first storage space. All storage media outside the first storage space of the cache node server can be allocated as the second storage space, serving as a pre-cooling cache layer for the cache node.

[0077] The pre-cooling cache layer pre-stores multiple resource shards based on resource acquisition behavior prediction. For example, it can periodically count the frequency of each identifier in the received model resource acquisition requests and store the resource shard corresponding to the identifier with the highest frequency in the pre-cooling cache layer.

[0078] The cache node server can maintain a storage index table, recording the identifiers and corresponding physical addresses of resource fragments stored in the pre-cooled cache layer of its local cache node. Based on the received target identifier, the storage index table is searched to see if the same identifier exists. If it does, a cache hit occurs, and the data stored at the corresponding physical address is sent to the compute node that initiated the model resource retrieval request.

[0079] Step 140: If the target resource shard does not exist in the pre-cooled cache layer, send a model resource acquisition request to the target node.

[0080] If neither the memory-level cache layer nor the pre-cooling cache layer of this cache node contains the target resource shard (i.e., both cache levels are cache misses), then to ensure that the compute node can eventually obtain the required data, it is necessary to attempt to obtain the target resource shard from other nodes. In this embodiment, the target node is a node that may store the target resource shard, and can be other cache nodes, nearby high-speed object storage nodes, or central object storage nodes on the public network. The user can pre-define target node determination rules to identify one or more target nodes. For example, if the preset rule is to uniformly obtain resources from the central object storage node on the public network when a cache miss occurs, then the target node is determined to be the central object storage node on the public network, and a model resource acquisition request is sent to it.

[0081] The edge-cloud collaborative distribution method for model resources provided in this application can improve the transmission speed of model resources by establishing cache nodes with memory-level cache layers in the vicinity of computing nodes; it can improve the cache hit rate and reduce the time required to obtain data by pre-storing resource fragments based on resource acquisition behavior prediction in the pre-cooled cache layer; and it forms a hierarchical dynamic management system from hot data to cold data, which can improve the distribution efficiency of model resources.

[0082] In some embodiments, the first storage space is the local memory space of the cache node; the second storage space is the local non-volatile storage medium of the cache node.

[0083] In this embodiment, the local memory space of the cache node refers to the memory area in the physical memory of the cache node server itself, which is specifically used to store cache data. The corresponding physical storage medium can be a DRAM memory module that is directly installed on the motherboard memory slot through a DIMM slot. It has the characteristics of low response time and support for high-concurrency read and write operations, and is the storage medium with the highest data transmission speed.

[0084] Non-volatile storage media is a type of storage device, including SSDs and HDDs, that can retain data after the device is powered off, but its data transfer speed is lower than that of volatile storage media such as RAM. Local non-volatile storage media in a cache node refers to non-volatile storage media directly installed on the cache node server itself. The cache node's CPU can directly read and write data from it via the server's internal bus without going through a network.

[0085] In this embodiment, by setting the first memory space as the local memory space of the cache node, data is preferentially obtained from the memory space with the highest data transfer speed, which can reduce data transfer time; by setting the second storage space in the local non-volatile storage medium of the same cache node, the cache hit rate within the same cache node can be improved, cross-node transmission can be reduced, thereby improving the distribution efficiency of model resources.

[0086] In some embodiments, the method further includes: The original model resources are fragmented to obtain multiple resource fragments; The target hash algorithm is used to perform hash calculation on each resource fragment to obtain the fragment hash identifier corresponding to each resource fragment, and the fragment hash identifier is determined as the identification information of the corresponding resource fragment.

[0087] Sharding involves dividing a complete resource into multiple smaller parts according to certain rules. The shard size can be preset, and the original model resource can be sequentially divided into multiple resource shards of the same size. For example, if an original model resource is K MB in size, and the preset shard size is k MB, then the data in the [0, k] MB portion will be divided into the first shard, the data in the [k+1, 2k] MB portion into the second shard, and so on, resulting in N = K / k resource shards.

[0088] After obtaining N resource fragments, a target hash algorithm is used to perform hash calculations on each resource fragment. A hash algorithm is a type of encryption algorithm that uses a specific hash function to convert the original input data into a fixed-length output value, with each output value uniquely corresponding to the original input data.

[0089] Hash algorithms such as MD5, SHA-1, and SHA-256 can be used as the target hash algorithm to perform hash calculations on each resource shard, generating a string of several characters as the shard hash identifier. The obtained shard hash identifier is then used as the identification information of the corresponding resource shard.

[0090] The HighwayHash algorithm can also be used as the target hash algorithm. Leveraging the multi-core computing power of the CPU, the HighwayHash algorithm first further divides each resource fragment into several smaller data blocks of the same size. Each data block is assigned to a different CPU core, and the hash value of each data block is calculated in parallel. Then, the hash values ​​of each data block are combined sequentially to obtain the hash identifier of the resource fragment. The HighwayHash algorithm can improve computational speed while maintaining sufficient security, making it suitable for fast fragmentation and identification of large model files.

[0091] Of course, other hash algorithms can also be used as the target hash algorithm, and this application does not limit this.

[0092] In this embodiment, resource shards can be cached in the memory-level caching layer of distributed cache nodes, and the cache records of the resource shards can be written to or updated in the global metadata database. In one example, the main record fields are shown in the table below:

[0093] In this embodiment, by dividing the original model resources into multiple resource fragments and using the target hash algorithm to calculate the fragment hash identifier corresponding to each resource fragment, unique identification information can be generated for each resource fragment, thereby facilitating the retrieval of resource fragments.

[0094] In some embodiments, the edge-cloud collaborative distribution method for model resources further includes: The access attribute information of each resource fragment in the memory-level cache layer is periodically collected; the access attribute information includes the access time and cumulative access count of the resource fragment; Calculate the first popularity value of each resource fragment based on the access attribute information; Remove resource fragments whose first popularity value is lower than the first target eviction threshold from the memory-level cache layer.

[0095] In this embodiment of the application, the storage space of the memory-level cache layer is limited. Therefore, when the memory-level cache layer is full, if a new resource fragment enters the memory-level cache layer, some resource fragments need to be deleted to store the new resource fragment.

[0096] The update cycle of the memory-level cache layer can be preset, and the access attribute information of each resource fragment in the memory-level cache layer can be collected in each update cycle.

[0097] Access attribute information includes the access time and cumulative access count of the resource shard. The access time refers to the last time the resource shard was accessed; the cumulative access count refers to the number of times the resource shard has been accessed within a certain time period, which can be from the moment the resource shard was stored in the memory-level cache layer to the current moment. Of course, access attribute information can also include other information, such as the number of compute nodes that have historically accessed the resource shard.

[0098] Based on the access attribute information, calculate the first popularity value for each resource shard. The calculation rules for the first popularity value can be preset by the user. For example, the access frequency of each resource shard can be calculated based on the cumulative number of accesses, and the calculated access frequency value can be directly used as the first popularity value.

[0099] The calculated first popularity value reflects the popularity of a resource fragment. The higher the value, the more popular the corresponding resource fragment is, and the more likely it is to be accessed by the compute node again. The lower the value, the less likely it is to be accessed by the compute node again. Therefore, users can pre-set a first target eviction threshold to delete resource fragments with a first popularity value lower than the first target eviction threshold from the memory-level cache layer to free up memory space, thereby making it possible to store new resource fragments.

[0100] In this embodiment, by calculating the first popularity value of each resource fragment based on the access time and cumulative access count of the resource fragment, and deleting resource fragments with the first popularity value lower than the first target eviction threshold, the most popular resource fragment can be kept in the memory-level cache layer with the fastest transmission speed, thereby improving the cache hit rate of the memory-level cache layer and thus improving the distribution efficiency of model resources.

[0101] In some embodiments, according to the formula:

[0102] Calculate the first heat value for each resource fragment; in, This indicates the first popularity value. This indicates the cumulative number of accesses to a resource shard within the target time window. Weighting factors representing the number of visits. Indicates the current time. This indicates the access time of the resource shard most recent to the current time. This represents the weighting factor for the time decay term.

[0103] In this embodiment of the application, the target time window is a time period that can be preset by the user. For example, the target time window can be set as [ -10T, ], where T is the size of the memory-level cache layer update cycle, and the target time window is the time from the current time to the previous 10 memory-level cache layer update cycles, corresponding to This refers to the number of times a resource shard has been accessed within the time frame from the current time to the previous 10 memory-level cache update cycles, reflecting the resource shard's popularity during that historical period. The weighting factor for access frequency... It is a numerical value used to control the degree to which the first popularity value is used as a reference for the number of visits.

[0104] This represents the time since the resource allocation was last accessed. Its magnitude is negatively correlated with the resource fragment's popularity; that is, the more recent the last access time is, the more popular the resource fragment has been recently. Since the magnitude of the first popularity value should be positively correlated with the resource fragment's popularity, for ease of calculation, we take... The reciprocal of the time decay term is converted to a positive correlation. It is a numerical value used to control the degree to which the first popularity value is based on the most recent access time.

[0105] In this embodiment, by calculating the first popularity value based on the cumulative number of accesses to the resource shard within the target time window and the most recent access time, the judgment can be made by comprehensively considering the historical total popularity and the recent popularity, so that the obtained first popularity value can more accurately reflect the current popularity of the resource shard.

[0106] In some embodiments, the edge-cloud collaborative distribution method for model resources further includes: Collect resource acquisition behavior data of computing nodes within the target historical period; the resource acquisition behavior data includes the identification information, acquisition time and acquisition frequency of each resource fragment; Predict the future acquisition probability of each resource fragment based on resource acquisition behavior data; Send a preload request to the data source node; the preload request carries the identification information of resource fragments whose future acquisition probability is greater than the target pre-stored threshold; Receive resource fragments returned by the data source node and store the resource fragments returned by the data source node in the pre-cooling cache layer.

[0107] In this embodiment, resource acquisition behavior data describes the behavior of a compute node requesting resources from a cache node, and may include multiple data items. The target historical period is a time period, which can be preset by the user as an aggregation time window. The target historical period is That is, every time The data collection and computing nodes start from the current time. Before Resource acquisition behavior data within a specific time period.

[0108] The cache node can maintain a structured access log table. Each time it receives a model resource acquisition request from a compute node, it records a structured access log entry. The log contains the resource fragment identifier (CID) and the access time (AccessTime) of the received request. Each time... Place AccessTime in Logs within the scope are aggregated using CID as the key. This records the last access time of each resource shard, designated as the acquisition time T. The number of times each resource shard is accessed within the target historical period is counted, and this number is used as the acquisition frequency F. Thus, the resource acquisition behavior data {CID, T, F} of the compute node within the target historical period is obtained. Of course, other data, such as cache hit rate, can also be recorded in the resource acquisition behavior data.

[0109] After obtaining resource acquisition behavior data, the future acquisition probability of each resource fragment is predicted based on this data. For example, the resource acquisition behavior data may contain data corresponding to N resource fragments, such as... ,in Let i be the acquisition time of the i-th resource fragment. This refers to the acquisition frequency of the i-th resource fragment. The weight value can be preset by the user. , According to the formula Calculate the heat value of the i-th resource fragment. and each Normalization ,make , will get This represents the future acquisition probability of the i-th resource fragment.

[0110] Users can pre-set a probability value P as a target pre-stored threshold, and then statistically analyze it. The resource fragment identification information is obtained and encapsulated into a preload request according to the communication protocol, and sent to the data source node.

[0111] The data source node is a server node that stores model resource data. It can receive preload requests, extract the identification information they carry, retrieve the resource fragments corresponding to the identification information from its own storage, and send them back to the cache node that issued the preload request. After receiving the resource fragments returned by the data source node, the cache node stores them in its own pre-cold cache layer.

[0112] In some embodiments, the preloading task can also be submitted to the background task queue of the cache node for execution. This process is completely asynchronous, which reduces the likelihood of preloading tasks blocking real-time user data requests, thereby improving response speed.

[0113] In this embodiment, by predicting the future acquisition probability of each resource fragment based on the resource acquisition behavior data of the computing node within the target historical period, and pre-storing resource fragments with a future acquisition probability greater than the target pre-stored threshold in the pre-cooled cache layer, the cache hit rate of the pre-cooled cache layer can be improved.

[0114] In some embodiments, predicting the future acquisition probability of each resource shard based on resource acquisition behavior data includes: Based on resource acquisition behavior data, a time series of acquisition frequency is constructed for each resource shard; the acquisition frequency time series includes the total number of acquisitions of the resource shard within different time windows; The acquisition frequency time series is input into the target prediction model to obtain the acquisition probability sequence output by the target prediction model; the acquisition probability sequence includes the probability that resource fragments will be acquired in different time windows in the future.

[0115] In this embodiment, the resource acquisition behavior data includes the identification information, acquisition time, and acquisition frequency of each resource fragment acquired by the computing node within each historical period. A time window can be preset by the user. Each resource fragment is statistically analyzed at the current moment. The i-th time window, i.e. The sum of the acquisition frequencies from the resource acquisition behavior data within a time period is used to obtain the total acquisition count for that slice within the i-th time window. .

[0116] For each resource shard, construct a time series of acquisition frequencies of length L. . Specifically, Where L is a positive integer, the value of which can be preset by the user. This results in... This reflects the historical access patterns of the computing nodes to this resource shard.

[0117] get Afterwards, Input the target prediction model F. The target prediction model F is a time-series prediction model that uses forward reasoning to deduce the probability of a resource shard being accessed within a future time window from historical access patterns. Input the target prediction model F, F can analyze The study identifies various features and their changing patterns within the data, and outputs a sequence of acquisition probabilities for the next H time windows. . Specifically, H is a positive integer whose value can be preset by the user; This represents the probability that the resource fragment will be acquired within the k-th time window in the future.

[0118] Historical access data can be used to pre-train a Hidden Markov Model as the target prediction model F. Alternatively, other machine learning models with architectures such as XGBoost (eXtreme Gradient Boosting) and Transformer can be used as the target prediction model F. This application does not limit the specific implementation of this method.

[0119] Obtain the probability sequence Then, the maximum prediction probability can be selected. As the probability of acquiring this resource fragment in the future, subsequent... Resource fragments exceeding the target pre-store threshold are preloaded into the pre-cooled cache layer.

[0120] In this embodiment, by constructing an acquisition frequency time series for each resource shard and using a target prediction model to make predictions based on the acquisition frequency time series, the probability of a resource shard being acquired in different time windows in the future can be predicted based on historical access patterns, thereby improving the accuracy of future acquisition probability prediction.

[0121] In some embodiments, the edge-cloud collaborative distribution method for model resources further includes: Periodically obtain the second heat value of each resource fragment in the pre-cooled cache layer; The second heat value of each resource fragment is updated based on the decay coefficient, and resource fragments with a second heat value lower than the second target eviction threshold are removed from the pre-cooled cache layer.

[0122] In this embodiment of the application, the storage space of the pre-cooling cache layer is limited. Therefore, when the pre-cooling cache layer is full, if a new resource fragment enters the pre-cooling cache layer, some resource fragments need to be deleted to store the new resource fragment.

[0123] The pre-cooling cache layer update cycle can be preset. Within each update cycle, the second popularity value of each resource shard in the pre-cooling cache layer can be obtained. The second popularity value is a numerical value that describes the popularity of the resource shard. The higher the second popularity value, the more popular the resource shard.

[0124] The initial value of the second heat value can be preset by the user. For example, the second heat value can be set to S when a resource fragment is stored in the pre-cooled cache layer.

[0125] After each pre-cooling cache layer update cycle, the second heat value of all resource fragments in the pre-cooling cache layer is updated based on the decay coefficient. The decay coefficient is... It is a value between (0, 1) used to control the speed at which resource fragments are evicted. A larger value indicates that resource fragments are eliminated more quickly. The value can be preset by the user. After N pre-cooling cache layer update cycles, the second heat value of this resource shard is... .

[0126] Users can pre-set a second target eviction threshold to remove resource fragments with a second heat value lower than the second target eviction threshold from the pre-cooled cache layer, thereby freeing up storage space.

[0127] In this embodiment, by updating the second popularity value of each resource fragment based on the decay coefficient, the natural decline of access popularity can be simulated; by deleting resource fragments whose second popularity value is lower than the second target eviction threshold, the situation where outdated resource fragments occupy the pre-cooled cache layer space for a long time can be reduced, thereby improving the cache hit rate of the pre-cooled cache layer.

[0128] In some embodiments, according to the formula:

[0129] Update the second popularity value of each resource fragment; in, This represents the second heat value after the resource shard i is updated. This represents the second most popular value of resource shard i before the update. Indicates the attenuation coefficient. This is the base value for popularity.

[0130] If only based on the formula Updating the second popularity value of each resource shard involves a process based on the properties of the exponential function. If resource shard i was once very popular but hasn't been accessed for a long time, its second popularity value will drop too quickly, leading to premature deletion. However, because resource shard i was once very popular, its future access probability predicted from resource acquisition behavior data is likely to be high, causing it to be preloaded into the pre-cooled cache layer again. In this situation, repeatedly performing the "store-delete" process for resource shard i will impact the performance of the cache nodes.

[0131] Introducing the base value B of heat into the second heat value update formula can reduce the rate at which the second heat value decays.

[0132] In this embodiment, by setting a base value for popularity, the situation where the popularity value of resource fragments that have not been accessed for a long time but were once very popular can be reduced too quickly, thereby reducing cache oscillation.

[0133] In some embodiments, sending a model resource acquisition request to the target node includes: A candidate node set is constructed based on the location of the compute node; the candidate node set includes at least one of the following candidate node categories: other cache nodes within the same compute cluster as the compute node; cache nodes within different compute clusters within the same availability zone as the compute node; self-built high-speed object storage nodes in the availability zone; and central object storage nodes on the public internet. Select the target candidate node category from the candidate node set according to priority order; Identify the target node from the target candidate node categories and send a model resource acquisition request to the target node.

[0134] In this embodiment of the application, the location of a computing node can be a logical address. For example, the location of computing node A can be {NodeID = A, AzID = a, ClusterID = 1}, indicating that computing node A is located in computing cluster 1 in availability zone a.

[0135] The candidate node set is a collection of candidate nodes, including at least one of the following categories: other cache nodes within the same compute cluster as the compute node, in which case the cache node {ClusterID = 1} can be retrieved {C1}; cache nodes within the same availability zone but different compute clusters as the compute node, in which case the cache node {AzID = a,ClusterID != 1} can be retrieved {C2}; self-built high-speed object storage nodes in the availability zone, in which case the high-speed object storage node {AzID = a} can be retrieved {C3}; and central object storage nodes on the public network, in which case the list of central object storage nodes can be retrieved {C4}.

[0136] The locations of cache nodes and compute nodes can be compared to construct a candidate node set. The location of a cache node can also be a logical address, such as {NodeID', AzID', ClusterID'}. If {ClusterID' = 1}, meaning the cache node and compute node are in the same compute cluster, a candidate node set containing {C1}, {C2}, {C3}, and {C4} can be constructed. If {AzID' = a, ClusterID' != 1}, meaning the cache node and compute node are in different compute clusters within the same availability zone, then it can be assumed that each node in {C1} has been queried and none of them contain the target resource shard; therefore, a candidate node set containing {C2}, {C3}, and {C4} can be constructed, and so on.

[0137] After obtaining the candidate node set, the target candidate node category is selected from the candidate node set according to the priority order. The priority order can be preset by the user, for example, C1>C2>C3>C4, selecting the category with the highest priority in the candidate node set as the target candidate node category.

[0138] After determining the target candidate node categories, the target node is selected from them. An unvisited node can be randomly selected from the target candidate node categories as the target node, and a model resource acquisition request can be sent to it.

[0139] The model resource acquisition request received by the cache node can be directly forwarded to the target node. If the target node has a target resource shard, the target node will directly send the target resource shard to the computing node that issued the request.

[0140] In this embodiment, by constructing a candidate node set based on the location of the computing node that initiates the model resource acquisition request, and then selecting the target candidate node category from the candidate node set according to priority order and determining the target node, the situation of cross-regional data transmission can be reduced, thereby improving the distribution efficiency of model resources.

[0141] In some embodiments, determining a target node from a target candidate node category includes: Calculate the overall access cost between each candidate node and the computing node in the target candidate node category; the overall access cost includes at least one of network latency, bandwidth consumption cost, and access response time. The target node is determined based on the overall access cost.

[0142] In this embodiment of the application, network latency This refers to the time required for data to be transferred from the cache node to the candidate node; bandwidth consumption cost. This refers to the cost incurred from using network bandwidth, which can be calculated based on the expected amount of bandwidth used or peak bandwidth; access response time. This refers to the total time required from when a request is sent from the cache node to when the candidate node completes its response. It can be calculated based on factors such as network latency and server processing speed.

[0143] Users can pre-set the weights of each item according to the formula. Calculate the comprehensive access cost between each candidate node and the compute node in the target candidate node category. The sum of the weights of each item .

[0144] Of course, other parameters can also be introduced as the basis for determining the overall access cost, such as transmission energy consumption and the busyness of candidate nodes, but this application embodiment does not limit this.

[0145] After calculating the overall access cost of each candidate node, the target node is determined based on the overall access cost. For example, the candidate nodes can be sorted from low to high based on their overall access cost, and the candidate node with the lowest overall access cost can be determined as the target node.

[0146] In this embodiment, by calculating the comprehensive access cost based on at least one of network latency, bandwidth consumption cost, and access response time, and determining the target node, a decision can be made that integrates network performance, cost, and reliability, thereby improving the accuracy of determining the target node.

[0147] The edge-cloud collaborative distribution method for model resources provided in this application can be executed by an edge-cloud collaborative distribution device for model resources. This application uses the execution of the edge-cloud collaborative distribution method for model resources by an edge-cloud collaborative distribution device as an example to illustrate the edge-cloud collaborative distribution device for model resources provided in this application.

[0148] This application also provides an edge-cloud collaborative distribution device for model resources.

[0149] like Figure 2 As shown, the edge-cloud collaborative distribution device for the model resources includes: The receiving module 210 is used to receive a model resource acquisition request initiated by the computing engine of the computing node. The model resource acquisition request carries the target identifier information of the target resource fragment. The first query module 220 is used to query the memory-level cache layer based on the target identifier information; the memory-level cache layer is the local memory cache of the computing node. The second query module 230 is used to query the pre-cooled cache layer based on the target identification information when the target resource shard does not exist in the memory-level cache layer; the pre-cooled cache layer is a storage medium deployed locally on the computing node, used to pre-store resource shards based on resource acquisition behavior prediction; The sending module 240 is used to send a model resource acquisition request to the target node when there is no target resource fragment in the pre-cooled cache layer.

[0150] The edge-cloud collaborative distribution device for model resources according to this application can improve the transmission speed of model resources by establishing cache nodes with memory-level cache layers in the vicinity of computing nodes; it can improve the cache hit rate and reduce the time required to obtain data by pre-storing resource shards based on resource acquisition behavior prediction in the pre-cooled cache layer; and it forms a hierarchical dynamic management system from hot data to cold data, which can improve the distribution efficiency of model resources.

[0151] In some embodiments, the receiving module 210 is further configured to: perform fragmentation processing on the original model resources to obtain multiple resource fragments; The target hash algorithm is used to perform hash calculation on each resource fragment to obtain the fragment hash identifier corresponding to each resource fragment, and the fragment hash identifier is determined as the identification information of the corresponding resource fragment.

[0152] In some embodiments, the first query module 220 is further configured to: Periodically collect access attribute information for each resource fragment in the memory-level cache layer; the access attribute information includes the access time and cumulative access count of the resource fragment; Calculate the first popularity value of each resource fragment based on the access attribute information; Remove resource fragments whose first popularity value is lower than the first target eviction threshold from the memory-level cache layer.

[0153] In some embodiments, the first query module 220 is further configured to: query according to the formula:

[0154] Calculate the first heat value for each resource fragment; in, This indicates the first popularity value. This indicates the cumulative number of accesses to a resource shard within the target time window. Weighting factors representing the number of visits. Indicates the current time. This indicates the access time of the resource shard most recent to the current time. This represents the weighting factor for the time decay term.

[0155] In some embodiments, the second query module 230 is further configured to: Collect resource acquisition behavior data of computing nodes within the target historical period; the resource acquisition behavior data includes the identification information, acquisition time and acquisition frequency of each resource fragment; Predict the future acquisition probability of each resource fragment based on resource acquisition behavior data; Send a preload request to the data source node; the preload request carries the identification information of resource fragments whose future acquisition probability is greater than the target pre-stored threshold; Receive resource fragments returned by the data source node and store the resource fragments returned by the data source node in the pre-cooling cache layer.

[0156] In some embodiments, the second query module 230 is further configured to: Based on resource acquisition behavior data, a time series of acquisition frequency is constructed for each resource shard; the acquisition frequency time series includes the total number of acquisitions of the resource shard within different time windows; The acquisition frequency time series is input into the target prediction model to obtain the acquisition probability sequence output by the target prediction model; the acquisition probability sequence includes the probability that resource fragments will be acquired in different time windows in the future.

[0157] In some embodiments, the second query module 230 is further configured to: Periodically obtain the second heat value of each resource fragment in the pre-cooled cache layer; The second heat value of each resource fragment is updated based on the decay coefficient, and resource fragments with a second heat value lower than the second target eviction threshold are removed from the pre-cooled cache layer.

[0158] In some embodiments, the second query module 230 is further configured to: query according to the formula:

[0159] Update the second popularity value of each resource fragment; in, This represents the second heat value after the resource shard i is updated. This represents the second most popular value of resource shard i before the update. Indicates the attenuation coefficient. This is the base value for popularity.

[0160] In some embodiments, the sending module 240 is further configured to: A candidate node set is constructed based on the location of the compute node; the candidate node set includes at least one of the following candidate node categories: other cache nodes within the same compute cluster as the compute node; cache nodes within different compute clusters within the same availability zone as the compute node; self-built high-speed object storage nodes in the availability zone; and central object storage nodes on the public internet. Select the target candidate node category from the candidate node set according to priority order; Identify the target node from the target candidate node categories and send a model resource acquisition request to the target node.

[0161] In some embodiments, the sending module 240 is further configured to: Calculate the overall access cost between each candidate node and the computing node in the target candidate node category; the overall access cost includes at least one of network latency, bandwidth consumption cost, and access response time. The target node is determined based on the overall access cost.

[0162] The edge-cloud collaborative distribution device for model resources in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the specific implementation.

[0163] The edge-cloud collaborative distribution device for model resources in this application embodiment can be a device with an operating system. This operating system can be a Microsoft (Windows) operating system, an Android operating system, an iOS operating system, or other possible operating systems; this application embodiment does not specifically limit it.

[0164] In some embodiments, such as Figure 3As shown, this application embodiment also provides an electronic device 300, including a processor 301, a memory 302, and a computer program stored on the memory 302 and executable on the processor 301. When the program is executed by the processor 301, it implements the various processes of the above-mentioned edge-cloud collaborative distribution method embodiment of model resources and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0165] It should be noted that the electronic devices in the embodiments of this application include the aforementioned mobile electronic devices and non-mobile electronic devices.

[0166] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described edge-cloud collaborative distribution method embodiment for model resources and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0167] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0168] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the edge-cloud collaborative distribution method for the above-mentioned model resources.

[0169] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0170] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described edge-cloud collaborative distribution method embodiment for model resources, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0171] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0172] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0173] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0174] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0175] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0176] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for edge-cloud collaborative distribution of model resources, characterized in that, include: Receive a model resource acquisition request initiated by a computing node, wherein the model resource acquisition request carries the target identifier information of the target resource fragment; Query the memory-level cache layer based on the target identification information; The memory-level cache layer is the first storage space of the cache node, and the cache node is a server deployed adjacent to the computing node; If the target resource fragment does not exist in the memory-level cache layer, the pre-cooled cache layer is queried based on the target identification information. The pre-cooling cache layer is the second storage space of the cache node, used to pre-store resource fragments based on resource acquisition behavior prediction; If the target resource fragment is not present in the pre-cooled cache layer, a model resource acquisition request is sent to the target node.

2. The method according to claim 1, characterized in that, The first storage space is the local memory space of the cache node; the second storage space is the local non-volatile storage medium of the cache node.

3. The method according to claim 1, characterized in that, The method further includes: The original model resources are fragmented to obtain multiple resource fragments; The target hash algorithm is used to perform hash calculation on each resource fragment to obtain the fragment hash identifier corresponding to each resource fragment, and the fragment hash identifier is determined as the identification information of the corresponding resource fragment.

4. The method according to claim 1, characterized in that, The method further includes: The access attribute information of each resource fragment in the memory-level cache layer is periodically collected; the access attribute information includes the access time and cumulative access count of the resource fragment; Calculate the first popularity value of each resource fragment based on the access attribute information; Remove resource fragments whose first popularity value is lower than the first target eviction threshold from the memory-level cache layer.

5. The method according to claim 4, characterized in that, According to the formula: Calculate the first heat value for each resource fragment; in, This indicates the first popularity value. This indicates the cumulative number of accesses to a resource shard within the target time window. Weighting factors representing the number of visits. Indicates the current time. This indicates the access time of the resource shard most recent to the current time. This represents the weighting factor for the time decay term.

6. The method according to claim 1, characterized in that, The method further includes: Collect resource acquisition behavior data of the computing nodes within the target historical period; the resource acquisition behavior data includes the identification information, acquisition time and acquisition frequency of each resource fragment; Predict the future acquisition probability of each resource fragment based on the resource acquisition behavior data; Send a preload request to the data source node; the preload request carries the identification information of the resource fragment whose future acquisition probability is greater than the target pre-stored threshold; Receive the resource fragments returned by the data source node and store the resource fragments returned by the data source node in the pre-cooling cache layer.

7. The method according to claim 6, characterized in that, The step of predicting the future acquisition probability of each resource shard based on the resource acquisition behavior data includes: Based on the resource acquisition behavior data, an acquisition frequency time series is constructed for each resource shard; the acquisition frequency time series includes the total number of acquisitions of the resource shard within different time windows; The acquisition frequency time series is input into the target prediction model to obtain the acquisition probability sequence output by the target prediction model; the acquisition probability sequence includes the probability that resource shards will be acquired in different time windows in the future.

8. The method according to claim 1, characterized in that, The method further includes: The second heat value of each resource fragment in the pre-cooled cache layer is periodically obtained; The second heat value of each resource fragment is updated based on the decay coefficient, and resource fragments with a second heat value lower than the second target eviction threshold are deleted from the pre-cooled cache layer.

9. The method according to claim 8, characterized in that, According to the formula: Update the second popularity value of each resource fragment; in, This represents the second heat value after the resource shard i is updated. This represents the second most popular value of resource shard i before the update. Indicates the attenuation coefficient. This is the base value for popularity.

10. The method according to claim 1, characterized in that, Sending the model resource acquisition request to the target node includes: A candidate node set is constructed based on the location of the compute node; the candidate node set includes at least one of the following candidate node categories: other cache nodes within the same compute cluster as the compute node; cache nodes within different compute clusters within the same availability zone as the compute node; self-built high-speed object storage nodes in the availability zone; and central object storage nodes on the public internet. Target candidate node categories are selected from the candidate node set according to priority order; A target node is determined from the target candidate node categories, and the model resource acquisition request is sent to the target node.

11. The method according to claim 10, characterized in that, Determining the target node from the target candidate node categories includes: Calculate the comprehensive access cost between each candidate node in the target candidate node category and the computing node; the comprehensive access cost includes at least one of network latency, bandwidth consumption cost, and access response time. The target node is determined based on the comprehensive access cost.

12. An edge-cloud collaborative distribution device for model resources, characterized in that, include: The receiving module is used to receive a model resource acquisition request initiated by the computing engine of the computing node, wherein the model resource acquisition request carries the target identifier information of the target resource fragment; The first query module is used to query the memory-level cache layer based on the target identification information; The memory-level cache layer is a local memory cache of the computing node; The second query module is used to query the pre-cooled cache layer based on the target identification information when the target resource fragment does not exist in the memory-level cache layer. The pre-cooling cache layer is a storage medium deployed locally on the computing node, used to pre-store resource fragments based on resource acquisition behavior prediction; The sending module is used to send the model resource acquisition request to the target node when the target resource fragment does not exist in the pre-cooled cache layer.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-11.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-11.

Images