Peer evaluation method, peer rater, mirror distribution method and system

Abstract

The application discloses a peer evaluation method, a peer grader, a mirror image distribution method and a system, wherein the evaluation method is used for evaluating the applicability score of each peer based on content popularity and network state; the applicability of the target peer is scored according to the following steps; the mirror image distribution method comprises the following steps: in a peer-to-peer downloading process, a peer containing a target mirror image layer is taken as a candidate peer, and the corresponding candidate peer is selected based on the applicability score to perform peer-to-peer downloading. The evaluation method considers the network performance and the content popularity simultaneously, and the applicability score reflects the selection applicability of the peer. In actual application, a person skilled in the art can select the peer based on the applicability score, so that the mirror image downloading efficiency and resource utilization in the edge computing environment are optimized.

Description

Technical Field

[0001] This invention relates to the field of network technology, and in particular to a peer evaluation method, a peer scorer, a mirror distribution method, and a system. Background Technology

[0002] Container technology is becoming increasingly important due to its lightweight and easy-to-migrate characteristics. Whether it's using the complete immutable dependency environment provided by containers to simplify deployment or using cloud-native microservices, containers are one of the core components. However, image distribution faces significant challenges in edge computing environments.

[0003] Existing image distribution solutions largely rely on centralized repositories, which limits their efficiency in dynamic network environments. For example, popular solutions such as Docker Hub (Docker's official public cloud image repository) and Google Container Registry (Google's open-source container image repository) perform well in stable network environments, but face many challenges in edge computing scenarios with fluctuating network quality.

[0004] These challenges are mainly manifested in two aspects: first, when the network conditions are poor, the time to pull images from the centralized image repository will increase significantly; second, when a large number of containers are deployed at the same time, the concentrated image pull requests will also put a heavy burden on the centralized image repository.

[0005] To address the aforementioned issues, current methods to improve image distribution efficiency include optimizing the download order of image layers and using compression techniques to reduce data transmission volume. However, in edge computing environments where network conditions change frequently, these methods have limited effectiveness.

[0006] To address the aforementioned issues, P2P (peer-to-peer) technology is currently used to distribute the workload, but existing P2P technology lacks consideration for network resources in edge environments. Summary of the Invention

[0007] This invention addresses the shortcomings of existing P2P technologies used in mirror distribution scenarios, which lack peer evaluation methods and cannot intelligently select suitable peers under dynamic network environment changes and limited resources. It provides a peer evaluation method and peer scoring device, as well as a mirror distribution method and system.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0009] A peer evaluation method, which treats each discovered peer as a target peer, and scores the applicability of the target peer according to the following steps:

[0010] Obtain the image owned by the target peer to obtain a target image set, wherein the image contains several image layers;

[0011] The popularity of the mirror layer is determined based on its distribution across all discovered peers.

[0012] Based on the popularity of each mirror layer in the target mirror set, a content popularity score for the target peer is generated.

[0013] Obtain the network score of the target peer;

[0014] The suitability score for the target peer is generated based on the content popularity score and the network score.

[0015] As one possible implementation method:

[0016] The method for obtaining the network score is as follows:

[0017] Obtain the current transmission speed from the target peer to the target client;

[0018] Obtain the global average speed of the target client at the current moment;

[0019] Based on the transmission speed and the global average speed, a network score for the target peer is generated.

[0020] A peer scorer is used to score the applicability of each discovered peer as a target peer.

[0021] It includes a network score calculation unit, a content popularity calculation unit, and a scoring unit;

[0022] The network score calculation unit is used to obtain the network score of the target peer;

[0023] The content popularity calculation unit includes:

[0024] The image acquisition unit is used to acquire the images owned by the target peer and obtain a target image set, wherein the image contains several image layers;

[0025] A mirror layer scoring unit is used to determine the popularity of the mirror layer based on its distribution among all discovered peers.

[0026] The peer scoring unit is used to generate a content popularity score for the target peer based on the popularity of each mirror layer in the target mirror set.

[0027] The scoring unit is used to generate a suitability score for the target peer based on the content popularity score and the network score.

[0028] A mirror distribution method includes the following steps:

[0029] Obtain an image download request containing information about the layer to be downloaded, and use the image layer corresponding to the information about the layer to be downloaded as the target image layer;

[0030] When the local cache does not contain the target image layer, peer discovery is performed based on a preset discovery time. During the peer discovery process, each discovered peer is scored based on the peer evaluation method described above, and the applicability score of each peer is generated and dynamically updated.

[0031] Peers containing the target image layer are selected as candidate peers, and peer-to-peer downloads are performed based on the applicability score.

[0032] As one possible implementation, the target image layer is divided into several data blocks, and a corresponding candidate peer is selected for each data block to make a download request. The step of selecting a corresponding candidate peer for the target data block includes:

[0033] Randomly select at least one candidate peer to obtain the target peer;

[0034] Send a download request to the target peer with the highest applicability score to download the target data block.

[0035] As one possible implementation, peer discovery is performed using a DHT network, with the Tracker and IP multicast.

[0036] As one possible implementation, a Tracker can be automatically elected based on the stability of the nodes.

[0037] As one possible implementation, cache information is obtained based on a preset cleanup cycle, and when it is determined based on the cache information that cache space is scarce, mirror cache cleanup is performed according to the following steps:

[0038] Sort the cached image layers from lowest to highest usage frequency;

[0039] Starting with the least frequently used mirror layer, check in turn whether other nodes in the local area network retain the same mirror layer, until the mirror layer is found to be retained by other nodes, then clean up the mirror layer and end this cleanup cycle.

[0040] When none of the cached image layers are retained by other nodes, clean up the least frequently used image layer and end the current cleanup cycle.

[0041] As one possible implementation, when the volume of the target mirror layer is less than a preset first volume threshold, or when no peer containing the target mirror layer is found within a preset discovery time, the target mirror layer is downloaded from the central mirror repository.

[0042] An image distribution system includes an image download system, the image download system comprising:

[0043] Interface module;

[0044] Cache module;

[0045] Download engine;

[0046] The interface module is used to obtain an image download request containing information about the layer to be downloaded, and to use the image layer corresponding to the information about the layer to be downloaded as the target image layer.

[0047] The download engine:

[0048] Used to retrieve the target image layer from the cache module;

[0049] It is also used to perform peer discovery based on a preset discovery time when the cache module does not contain the target image layer, and to score each discovered peer based on the above-mentioned peer evaluation method during the peer discovery process, and to generate and dynamically update the applicability score of each peer.

[0050] It is also used to select the corresponding candidate peer based on the applicability score for peer-to-peer download, using the peer containing the target image layer as a candidate peer.

[0051] These features and advantages of the present invention will be disclosed in detail in the following specific embodiments and accompanying drawings. The preferred embodiments or means of the present invention will be shown in detail in conjunction with the accompanying drawings, but are not intended to limit the technical solutions of the present invention. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the evaluation process of an evaluation method for an equivalent entity according to the present invention;

[0054] Figure 2 This is a flowchart illustrating the P2P download task in Example 3;

[0055] Figure 3 This is a schematic diagram of the module connections of a mirror distribution system according to the present invention. Detailed Implementation

[0056] The present invention will be further described in detail below with reference to the embodiments. The following embodiments are explanations of the present invention, but the present invention is not limited to the following embodiments.

[0057] Example 1: A peer evaluation method, wherein each discovered peer is taken as a target peer, and the applicability of the target peer is scored according to the following steps:

[0058] S100, Obtain the content popularity score of the target peer;

[0059] Content popularity score is based on the frequency of requests for specific content by peers in the network;

[0060] The specific steps for obtaining the content popularity score include:

[0061] S110. Obtain the image owned by the target peer to obtain a target image set, wherein the image contains several image layers;

[0062] S120. Determine the popularity of the mirror layer based on its distribution among all discovered peers;

[0063] The formula for calculating popularity is:

[0064]

[0065] in:

[0066] ρ l This represents the popularity of the mirror layer l in mirror i;

[0067] This represents the set of peers known to the target client, where the target client refers to the client that issued the image download request. Represents the set of discovered peers;

[0068] Represents the set of mirror images possessed by peer p;

[0069] When the mirror layer l exists in the peer p, then Defined as 0, otherwise as 1.

[0070] S130. Based on the popularity of each mirror layer in the target mirror set, generate the content popularity score of the target peer.

[0071] Content popularity score of peer p The calculation formula is:

[0072]

[0073] in:

[0074] This represents the set of mirror layers that constitute mirror i;

[0075] ρ l This represents the popularity of the corresponding mirror layer l;

[0076] λ is a preset weight parameter.

[0077] S200, Obtain the network score of the target peer;

[0078] The specific steps include:

[0079] S210. Obtain the transmission speed from the target peer to the target client at the current moment;

[0080] Obtain the historical transmission speed from the target peer to the target client, and calculate the transmission speed based on the historical transmission speed;

[0081] In this embodiment, a sliding window is pre-maintained for each peer. The sliding window is used to record the historical transmission speed from the peer to the target client, and the historical transmission speed is extracted from the sliding window corresponding to the target peer.

[0082] The formula for calculating the transmission speed in this embodiment is:

[0083]

[0084] in:

[0085] This represents the transmission speed of peer p at the current time t;

[0086] This represents the sliding window corresponding to the peer p, and the length of the sliding window is L. Those skilled in the art can set the length of the sliding window themselves.

[0087] This represents the historical transmission speed of peer p at time t'.

[0088] S220. Obtain the global average speed of the target client at the current moment;

[0089] Obtain the historical global average speed corresponding to the target client, and calculate the transmission speed based on the historical global average speed;

[0090] In this embodiment, a global sliding window is pre-maintained. This global sliding window is used to record historical global average velocity;

[0091] The formula for calculating the global average velocity in this embodiment is:

[0092]

[0093] in:

[0094] This represents the global average velocity at the current time t.

[0095] This represents a global sliding window, the length of which is L. Those skilled in the art can set the length of the global sliding window themselves.

[0096] This represents the global average velocity at time t'.

[0097] S230. Based on the transmission speed and the global average speed, generate the network score of the target peer.

[0098] The network score of peer p at time t. It can be obtained from the following formula:

[0099]

[0100] S300. Generate a suitability score for the target peer based on the content popularity score and the network score.

[0101] Knowing the content popularity score and the network score, those skilled in the art can set their own adaptive scoring mechanism, such as weighted summation. In this embodiment, a score of [0, 100] is assigned to the peer based on the content popularity score and the network score.

[0102] The evaluation method proposed in this embodiment considers both network performance and content popularity. The applicability score reflects the suitability of the peer selection. In practical applications, those skilled in the art can select peers based on the applicability score to optimize mirror download efficiency and resource utilization in edge computing environments.

[0103] Reference Figure 1 Furthermore, when the target peer and the target client are located in the same local area network, the target peer is assigned the highest score.

[0104] Existing P2P technology cannot distinguish between local and remote neighbors. When both local and remote neighbors have mirror copies, it cannot prioritize downloading from the local neighbor, thus wasting valuable outbound bandwidth.

[0105] This embodiment further incorporates network location for applicability evaluation, directly assigning the highest applicability score to the target peer located on the same local area network as the target client, which can reduce the computational load of the evaluation algorithm while ensuring the optimization effect.

[0106] Example 2: A peer scoring device, used to score the applicability of each discovered peer as a target peer;

[0107] It includes a network score calculation unit, a content popularity calculation unit, and a scoring unit;

[0108] The network score calculation unit is used to obtain the network score of the target peer;

[0109] The content popularity calculation unit includes:

[0110] The image acquisition unit is used to acquire the images owned by the target peer and obtain a target image set, wherein the image contains several image layers;

[0111] A mirror layer scoring unit is used to determine the popularity of the mirror layer based on its distribution among all discovered peers.

[0112] The peer scoring unit is used to generate a content popularity score for the target peer based on the popularity of each mirror layer in the target mirror set.

[0113] The scoring unit is used to generate a suitability score for the target peer based on the content popularity score and the network score.

[0114] Example 3: A mirror distribution method, including a mirror download method;

[0115] The mirror download method includes the following steps:

[0116] S100, Obtain the image download request initiated by the client;

[0117] The image download request;

[0118] The image layer corresponding to the information of the layer to be downloaded is used as the target image layer.

[0119] S200. Check whether the target image layer is contained in the local cache;

[0120] When the local cache contains the target image layer, the target image layer is returned to the client;

[0121] If the local cache does not contain the target image layer, proceed to step S300;

[0122] S300, Perform peer discovery and score the discovered peers;

[0123] In this embodiment, a preset discovery time is established. When the time for the peer to be discovered reaches the preset discovery time, the peer discovery step ends and the process proceeds to step S400.

[0124] This embodiment limits the time for peer discovery to avoid the system wasting too much time confirming whether the image is cached on the network. Those skilled in the art can set the discovery time according to actual needs.

[0125] In this embodiment, peer discovery is performed in parallel using a DHT (Distributed Hash Table) network, a Tracker, and IP multicast. This embodiment of the invention prioritizes a local peer discovery method based on multicast, thereby avoiding the more time-consuming DHT and tracker discovery process. Furthermore, local peer discovery based on multicast keeps network traffic within the local area network, thereby minimizing unnecessary traffic overhead and significantly reducing data transmission volume and time consumption.

[0126] During the peer discovery process, each discovered peer is scored based on the peer evaluation method described in Example 1, and the applicability score of each peer is generated and dynamically updated.

[0127] S400: Generate corresponding download tasks based on peer discovery results;

[0128] If no peer containing the target image layer is found, an HTTP download task is generated, and the target image layer is downloaded from the central image repository using HTTP download.

[0129] When a peer containing the target image layer is found, a P2P download task is generated and step S500 is executed.

[0130] In DHT networks, peer location efficiency is a key consideration in the process, as the multiple network hops involved can lead to significant time delays, especially when there are poor network conditions between peers (such as limited bandwidth or high packet loss rate), which can further prolong the peer discovery time.

[0131] Considering the potential negative impact of such delays on user experience, this embodiment implements a time-limited mechanism for peer aggregation through the design of discovery time. Specifically, if the availability of the image can be confirmed within the set time window, it indicates that the image holder exists in the vicinity of the network, making P2P download the preferred path. Conversely, if the image holder cannot be successfully located within the time limit, it usually means that there is no holder in the local network or the holder is far away from the requesting node, so the system will choose to download the image directly from Docker Hub.

[0132] S400, execute P2P download task;

[0133] Peers containing the target image layer are selected as candidate peers. Based on the applicability score, corresponding candidate peers are selected for peer-to-peer download. That is, the target image layer is divided into several data blocks, and corresponding candidate peers are selected for each data block to make download requests.

[0134] The specific steps include:

[0135] S410. Obtain the data block corresponding to the current download batch, and obtain the target data block;

[0136] The target image layer is pre-divided into several data blocks, and when performing a P2P download task, the data blocks are divided into several download batches.

[0137] In peer-to-peer (P2P) downloads, dividing content into several blocks is a common technique. By dividing the content into blocks, different blocks can be downloaded from multiple peers in the network during the subsequent download process, thereby making full use of available bandwidth.

[0138] In this embodiment, after obtaining the image download request initiated by the client, the image layer size information is obtained according to the information of the layer to be downloaded, and the target image layer is converted to the block range according to the byte range information (i.e., the "Range" header) in the request to obtain the corresponding data block;

[0139] If the chunks are too large, it becomes difficult to request a large number of peers simultaneously, thus making it difficult to fully utilize all available potential bandwidth. However, if the chunks are too small, it will also lead to higher system communication overhead.

[0140] In this embodiment, a second image volume threshold, a first block size, and a second block size are preset, wherein the first block size is greater than the second block size;

[0141] When the volume of the target image layer is greater than or equal to the second volume threshold, the layers are divided into blocks according to the first block size; otherwise, the layers are divided into blocks according to the second block size.

[0142] That is, for images that are large in size, the present invention uses a larger block size, thereby balancing parallelism and overhead.

[0143] The smaller the mirror image, the more complex it is;

[0144] Those skilled in the art can set the first image volume threshold, the first block size, and the second block size according to actual needs; this embodiment does not impose detailed limitations on them.

[0145] S420: Select candidate peers for the target data block and perform peer-to-peer download;

[0146] S421. Randomly select at least one candidate peer to obtain the target peer;

[0147] S422. Send a download request to the target peer with the highest applicability score to download the target data block.

[0148] This embodiment avoids requests being concentrated on a few high-scoring candidate peers by designing a sampling method for candidate peers, thereby effectively ensuring parallel execution of download tasks and further optimizing download efficiency.

[0149] The downloaded data blocks will be stored in the local cache and sent to the client. Once the current batch is successfully completed, the process will repeat to download new batches.

[0150] Reference Figure 2 A P2P download task consists of several download batches, each containing several data blocks to be downloaded. The download process for the current batch includes the following steps:

[0151] (1) Batch start; determine the target data block, as shown in the figure, the target data block is block x.

[0152] (2) Sampling based on fractions;

[0153] The applicability scores of the peers aggregated in the peer aggregator are updated in real time, that is, the applicability scores of all discovered peers are updated dynamically.

[0154] The candidate peers are sampled to obtain the target peer. In this embodiment, three candidate peers are randomly selected as the target peer.

[0155] Obtain the applicability score of the target peer and determine the target peer with the highest applicability score, as shown in the figure, peer 1 has the highest applicability score.

[0156] (3) Peer 1 was selected as the source of block x;

[0157] At this point, a request to download block x is sent to peer 1.

[0158] (4) Obtain block x from peer 1;

[0159] (5) Hash tree verification;

[0160] After downloading and obtaining block x, the integrity of block x is verified. If the block verification fails, it is returned to the waiting queue. If the block verification succeeds, it will be stored in the local cache and sent to the client. This is the standard procedure for existing P2P downloads, so it will not be described in detail in this manual.

[0161] Furthermore:

[0162] When the volume of the target image layer is less than a preset first volume threshold, the target image layer is downloaded from the central image repository.

[0163] In this embodiment, when it is found in step S200 that the local cache does not contain the target image layer, the layer volume determination step is performed.

[0164] The first volume threshold is less than the second volume threshold mentioned above. In this embodiment, the first volume threshold is set to 1 MiB.

[0165] To improve the efficiency of P2P network retrieval, an in-depth analysis of the latest image data on Docker Hub was conducted, focusing on the characteristics of the top 100 most popular images. The analysis results show that the vast majority of image layers are relatively small, with a median layer size of only 1.03 MiB.

[0166] These comprehensive peer discovery processes may seem disproportionate in duration compared to downloading directly from the central repository; however, given the prevalence of small layer sizes, relying solely on downloads from the central mirror repository could overload it.

[0167] To address this issue, this embodiment balances download efficiency and resource utilization by designing the layer volume determination step. When the layer volume is small, it is downloaded directly from the central mirror repository, avoiding the long time required for peer discovery that relies on the complete DHT and trackers, thus ensuring fast access and minimizing the load on the central mirror repository.

[0168] Furthermore, P2P technology uses Trackers to accelerate the process of nodes discovering each other. The implementation of Trackers is centralized, which may lead to single points of failure. To address this issue, this embodiment automatically elects Trackers based on the stability of nodes. Here, nodes are clients, and online time is used as a measure of node stability. The longer the online time, the higher the stability of the node.

[0169] In decentralized P2P networks, Trackers are typically used to coordinate communication between peers. However, traditional centralized Trackers are prone to becoming system bottlenecks and single points of failure. This embodiment embeds the Tracker functionality directly into the client, implementing a fully autonomous Tracker system where the election and management of Trackers are completely automated, requiring no manual intervention. This design not only eliminates the drawbacks of centralized Trackers but also enables the system to recover quickly in the event of node failures, ensuring the continuity of the mirror distribution process and the overall stability of the system. It completely changes the existing separation between Trackers and clients, thoroughly solving the centralization problem of traditional Trackers.

[0170] The specific workflow is as follows:

[0171] ① The target node announces the start of the Tracker election process to all its neighbors.

[0172] After a node starts up, it will query all its neighbors to see if there is a Tracker available, and store the nodes with Trackers for subsequent requests.

[0173] Once the Tracker election process begins, each node with a Tracker participates in the election by using its online time as a metric. The FloodMax algorithm is used to select the node p with the longest current online time. max ;

[0174] The purpose of selecting the node with the longest online time is to ensure the stability of the Tracker as much as possible.

[0175] ② Node p max Once it confirms that it has become a Tracker, it activates its built-in Tracker module and broadcasts the information that it has become a Tracker to the entire network.

[0176] At the same time, every node in the network should forward the announcement information to ensure that the newly created Tracker is known to all nodes in the network.

[0177] If node p max If a node loses its processing capability for any reason, a node will discover and request p. max If the process fails, the target node will initiate a Tracker election process until a new Tracker is elected. Furthermore, P2P technology inevitably introduces local caching overhead. In edge computing environments, caching overhead is significant relative to limited storage space. To address this issue, this embodiment introduces a method based on the container image's historical usage and distribution across other nodes to remove part of the image cache when storage space is limited.

[0178] In this embodiment, cache information is obtained based on a preset cleanup cycle, and when it is determined based on the cache information that the cache space is insufficient, the following steps are performed to clean up the mirror cache:

[0179] Sort the cached image layers from lowest to highest usage frequency;

[0180] Starting with the least frequently used mirror layer, check in turn whether other nodes in the local area network retain the same mirror layer, until the mirror layer is found to be retained by other nodes, then clean up the mirror layer and end this cleanup cycle.

[0181] When none of the cached image layers are retained by other nodes, clean up the least frequently used image layer and end the current cleanup cycle.

[0182] Those skilled in the art can set the cleanup cycle themselves; in this embodiment, a cache cleanup judgment is performed every 30 seconds.

[0183] In this embodiment, the cache information is the proportion of available disk space. If it is greater than the preset available threshold, the current cleanup is skipped; otherwise, the image cache cleanup process is entered. Each time the cache cleanup process is entered, an image layer will be deleted.

[0184] This embodiment determines the image layers to be cleared based on the usage frequency of the image layers and their distribution in the local network. That is, it prioritizes removing image layers that have not been used for a long time and image layers that already exist in the local network, ensuring that the removed images will not have a significant impact on the operation of system services. Even if the image is needed again, it can be retrieved at a low cost.

[0185] This embodiment, through the design of an image cache cleanup mechanism, can intelligently decide which image caches can be cleaned up based on the usage frequency of container images and their distribution on the local network. By periodically assessing the importance and substitutability of images, the system can automatically remove image caches that have not been used for a long time or are easily accessible on the local network, thereby freeing up valuable storage space. The implementation of this mechanism not only optimizes the utilization efficiency of storage resources but also reduces the network bandwidth requirements caused by frequent image downloads. This ensures that the system can maintain high-performance operation while adapting to the needs of edge computing environments with limited storage space, and can efficiently manage the limited storage resources of edge computing devices.

[0186] Example 4: Image distribution system, such as Figure 3 As shown, it includes an image download system, a central image repository, and a Tracker system:

[0187] The image download system includes:

[0188] The interface module is used to obtain an image download request containing information about the layer to be downloaded, and to use the image layer corresponding to the information about the layer to be downloaded as the target image layer.

[0189] Cache module;

[0190] Download engine:

[0191] Used to retrieve the target image layer from the cache module;

[0192] It is also used to perform peer discovery based on a preset discovery time when the cache module does not contain the target image layer, and to score each discovered peer based on the peer evaluation method described in Embodiment 1 during the peer discovery process, and to generate and dynamically update the applicability score of each peer.

[0193] It is also used to select the corresponding candidate peer based on the applicability score for peer-to-peer download, using the peer containing the target image layer as a candidate peer.

[0194] The download engine is used to execute the image download method described in Example 3.

[0195] In this embodiment, the download engine includes:

[0196] Peer aggregator, used for peer discovery and aggregation, in this embodiment it is based on DHT network, with Tracker and IP multicast performing peer discovery in parallel;

[0197] The peer scorer is used to score each peer in the peer aggregator based on content popularity and network status, i.e., the peer scorer described in Example 2.

[0198] The P2P downloader is used to respond to P2P download tasks. It takes the peer containing the target image layer as a candidate peer, selects the corresponding candidate peer based on the applicability score, and performs peer-to-peer download. The downloaded image will be stored in the cache.

[0199] The request dispatcher:

[0200] Used to check whether the target image layer is contained in the local cache;

[0201] It is also used to return the target image layer to the client when the local cache contains the target image layer;

[0202] It is also used to generate a P2P download task or a remote download task when the local cache does not contain the target image layer;

[0203] It is also used to generate a remote download task when the layer volume of the target image layer is less than a preset first volume threshold;

[0204] It is also used to request the dispatcher to request the peer aggregation to start peer discovery when the layer volume of the target mirror layer is greater than or equal to a preset first volume threshold, and to generate a P2P download task or a remote download task based on the peer discovery result. That is, if no peer containing the target mirror layer is found within the preset discovery time, a remote download task is generated.

[0205] It is also used to perform remote download tasks, that is, to download the target image layer from the central image repository.

[0206] Furthermore: The Tracker system is used for peer discovery and also for automatically electing Trackers based on node stability.

[0207] Furthermore:

[0208] The caching module includes a cache memory and a cache manager;

[0209] The cache memory is used to store the image layer;

[0210] The cache manager is used to search for mirror layers from the cache storage, obtain cache information based on a preset cleanup cycle, and perform mirror cache cleanup when it is determined that the cache space is scarce based on the cache information.

[0211] Furthermore:

[0212] The interface module uses the Docker API.

[0213] The Docker API interface fully leverages the standard APIs defined by the Open Container Initiative (OCI). More specifically, this system adapts to a portion of the Docker Registry API, enabling complex container image retrieval and management while maintaining user-friendly operation. This integration achieves seamless image processing, hiding the complex processes behind it. The key to the Docker engine's handling of image pull requests lies in processing multiple HTTP requests defined in the standards: one for image metadata, and others for layers that do not currently exist on the device. Image metadata is typically a JSON structure containing detailed information about the requested image, while layer-to-layer requests return a tarball containing the actual data. This invention specifically optimizes these two API endpoints:

[0214] Image metadata: Considering that metadata is typically small in size, this invention does not employ distributed storage. Instead, metadata is cached in an in-memory storage structure and continuously updated to reflect changes in the remote Docker Registry.

[0215] Mirror layer: The mirror download interface is a lightweight encapsulation layer that delegates the complex download strategies behind it to the download engine.

[0216] The Docker engine's API wrapper provides a concise interface. During initialization, this invention queries the Docker engine version via a UNIX socket and then provides runtime data to other components of the system. Essentially, the Docker API interface acts as a bridge between the Docker engine and the internal mechanisms of this invention. Through this ingenious encapsulation, this invention can precisely control and monitor the operation of the Docker engine, allowing the focus to be on optimizing its internal workings.

[0217] This section is implemented as a transparent proxy component. Requests to and from the local Docker engine and the remote Registry should transparently pass through the wrapper layer. However, the wrapper layer should also record necessary metadata, such as the size of each image layer. Image pull requests should be intercepted and handled by the download engine. The wrapper layer needs to ensure that the format of each API request and response remains unchanged so that it can be seamlessly loaded and used without modifying the Docker engine.

[0218] This system adopts a decentralized distributed architecture, intelligently integrating key technical components such as the Docker API interface, download engine, embedded autonomous tracker, and cache cleanup mechanism to optimize the container image distribution process in edge computing environments. The Docker API interface serves as the portal for system interaction with the outside world, ensuring seamless integration with existing container environments while shielding complex backend processing logic. The download engine dynamically selects the optimal image source based on real-time network conditions and content popularity, ensuring high-efficiency data transmission whether it's a remote central repository or other nodes in a P2P network. The embedded autonomous tracker overcomes the weaknesses of traditional centralized trackers, which are prone to becoming system bottlenecks and single points of failure, improving system robustness and self-recovery capabilities through a decentralized approach. The cache cleanup mechanism, while ensuring data transmission efficiency, manages local storage resources through intelligent algorithms, optimizing storage utilization and reducing network bandwidth requirements. The overall system design philosophy is to maximize the use of limited resources in the edge computing environment, improving user experience and system performance while ensuring container image distribution efficiency and reliability.

[0219] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0220] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0221] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0222] This invention is described with reference to flowchart illustrations and / or block diagrams of the method, terminal device (system), and computer program product according to the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0223] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0224] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0225] It should be noted that:

[0226] The phrase "an embodiment" or "an embodiment" used in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. Therefore, the phrase "an embodiment" or "an embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment.

[0227] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0228] Furthermore, it should be noted that the shapes and names of the parts and components described in the specific embodiments described in this specification may differ. All equivalent or simple variations made to the structure, features, and principles described in this patent concept are included within the protection scope of this patent. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to replace them, as long as they do not depart from the structure of this invention or exceed the scope defined in these claims, they should all fall within the protection scope of this invention.

Claims

1. A method for evaluating peers, characterized in that, Each discovered peer is taken as a target peer, and the applicability of the target peer is scored according to the following steps: Obtain the image owned by the target peer to obtain a target image set, wherein the image contains several image layers; The popularity of the mirror layer is determined based on its distribution across all discovered peers. Based on the popularity of each mirror layer in the target mirror set, a content popularity score for the target peer is generated. Obtain the network score of the target peer; A suitability score for the target peer is generated based on the content popularity score and the network score. The popularity of the mirror layer is determined based on its distribution across all discovered peers. The formula for calculating popularity is: ； in: Indicates mirror image i Mid-mirror layer l The corresponding popularity; This represents the set of peers known to the target client, where the target client refers to the client that issued the image download request. Represents the set of discovered peers; Representing peers The set of mirrors owned; When the mirror layer l Existing in the peer In the middle, then Defined as 0 if true, otherwise 1; Equivalent Content popularity score The calculation formula is: ； in: Indicates the formation of a mirror image i A collection of mirrored layers; Indicates the corresponding mirror layer l The corresponding popularity; λ is a preset weight parameter; (2) A sliding window is pre-maintained for each peer, which is used to record the historical transmission speed from the peer to the target client, and the historical transmission speed is extracted from the sliding window corresponding to the target peer; The formula for calculating transmission speed is: ； in: Representing peers p At the present moment t The corresponding transmission speed; Representing peers p The corresponding sliding window, the length of which is L ; Representing peers p At any moment t’ The corresponding historical transmission speed; The formula for calculating the global average velocity is: ； in: Indicates the current time t The corresponding global average velocity; This represents a global sliding window, the length of which is... L ; Indicates at time t’ The corresponding global average velocity; In time t The counterpart p Network score It can be obtained from the following formula: 。 2. A peer scoring device, characterized in that, Used to score the applicability of each discovered peer as a target peer; It includes a network score calculation unit, a content popularity calculation unit, and a scoring unit; The network score calculation unit is used to obtain the network score of the target peer; The content popularity calculation unit includes: The image acquisition unit is used to acquire the images owned by the target peer and obtain a target image set, wherein the image contains several image layers; A mirror layer scoring unit is used to determine the popularity of the mirror layer based on its distribution among all discovered peers. The peer scoring unit is used to generate a content popularity score for the target peer based on the popularity of each mirror layer in the target mirror set. The scoring unit is used to generate a suitability score for the target peer based on the content popularity score and the network score. The popularity of the mirror layer is determined based on its distribution across all discovered peers. The formula for calculating popularity is: ； in: Indicates mirror image i Mid-mirror layer l The corresponding popularity; This represents the set of peers known to the target client, where the target client refers to the client that issued the image download request. Represents the set of discovered peers; Representing peers The set of mirrors owned; When the mirror layer l Existing in the peer In the middle, then Defined as 0 if true, otherwise 1; Equivalent Content popularity score The calculation formula is: ； in: Indicates the formation of a mirror image i A collection of mirrored layers; Indicates the corresponding mirror layer l The corresponding popularity; λ is a preset weight parameter; (2) A sliding window is pre-maintained for each peer, which is used to record the historical transmission speed from the peer to the target client, and the historical transmission speed is extracted from the sliding window corresponding to the target peer; The formula for calculating transmission speed is: ； in: Representing peers p At the present moment t The corresponding transmission speed; Representing peers p The corresponding sliding window, the length of which is L ; Representing peers p At any moment t’ The corresponding historical transmission speed; The formula for calculating the global average velocity is: ； in: Indicates the current time t The corresponding global average velocity; This represents a global sliding window, the length of which is... L ; Indicates at time t’ The corresponding global average velocity; In time t The counterpart p Network score It can be obtained from the following formula: 。 3. A mirror distribution method, characterized in that, Includes the following steps: Obtain an image download request containing information about the layer to be downloaded, and use the image layer corresponding to the information about the layer to be downloaded as the target image layer; When the local cache does not contain the target image layer, peer discovery is performed based on a preset discovery time. During the peer discovery process, each discovered peer is scored based on the peer evaluation method described in claim 1, and the applicability score of each peer is generated and dynamically updated. Peers containing the target image layer are selected as candidate peers, and peer-to-peer downloads are performed based on the applicability score.

4. The mirror distribution method according to claim 3, characterized in that, The target image layer is divided into several data blocks, and a corresponding candidate peer is selected for each data block to make a download request. The steps of selecting a corresponding candidate peer for the target data block include: Randomly select at least one candidate peer to obtain the target peer; Send a download request to the target peer with the highest applicability score to download the target data block.

5. A mirror distribution method according to claim 3 or 4, characterized in that, Based on the DHT network, Tracker and IP multicast perform peer discovery.

6. The mirror distribution method according to claim 5, characterized in that, Trackers are automatically elected based on node stability.

7. A mirror distribution method according to claim 3 or 4, characterized in that, Cache information is obtained based on a preset cleanup cycle, and when it is determined that cache space is scarce based on the cache information, mirror cache cleanup is performed according to the following steps: Sort the cached image layers from lowest to highest usage frequency; Starting with the least frequently used mirror layer, check in turn whether other nodes in the local area network retain the same mirror layer, until the mirror layer is found to be retained by other nodes, then clean up the mirror layer and end this cleanup cycle. When none of the cached image layers are retained by other nodes, clean up the least frequently used image layer and end the current cleanup cycle.

8. A mirror distribution method according to claim 3 or 4, characterized in that, When the volume of the target mirror layer is less than a preset first volume threshold, or when no peer containing the target mirror layer is found within a preset discovery time, the target mirror layer is downloaded from the central mirror repository.

9. An image distribution system, comprising an image download system, characterized in that: The image download system includes: Interface module; Cache module; Download engine; The interface module is used to obtain an image download request containing information about the layer to be downloaded, and to use the image layer corresponding to the information about the layer to be downloaded as the target image layer. The download engine: Used to retrieve the target image layer from the cache module; It is also used to perform peer discovery based on a preset discovery time when the cache module does not contain the target image layer, and to score each discovered peer based on the peer evaluation method of claim 1 during the peer discovery process, and to generate and dynamically update the applicability score of each peer. It is also used to select the corresponding candidate peer based on the applicability score for peer-to-peer download, using the peer containing the target image layer as a candidate peer.

Images