Image distribution system and methods
By having the EPG central service node and edge service node in the image distribution system work together to pull only the necessary image layer data, the problems of resource waste and increased time in container image distribution are solved, and efficient image transmission is achieved.
Patent Information
- Application Number
- CN202311552771.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-20
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2043-11-20
AI Technical Summary
In the process of distributing container images, existing technologies result in a large amount of redundant data transmission due to the inheritance relationship between images, which leads to a waste of network and storage resources, increases data transmission time, and creates a performance bottleneck.
An image distribution system is adopted, which works in collaboration between the EPG central service node and edge service nodes. Utilizing a distributed architecture of hash tables and image repositories, only the necessary image-level metadata and file data are pulled to achieve parallel transmission.
It effectively reduces server resource waste, shortens data transmission time, improves image distribution efficiency, and solves performance bottlenecks in the image distribution process.
Smart Images

Figure CN117675837B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of cloud computing technology, and more specifically, to an image distribution system and method. Background Technology
[0002] Currently, starting a container requires pulling the entire container image from a remote container repository and writing it to the local computer's disk. During this process, due to image inheritance, all image layers are mounted together, generating a large amount of redundant data. This redundant data not only consumes network and storage resources but can also become a performance bottleneck for the entire system. For example, if an image has 10 image layers, each image layer will be transmitted completely during container image distribution. However, some of these layers will not be used in the final container. Therefore, this image distribution method wastes server resources and increases data transmission time.
[0003] There is currently no effective solution to the above problems. Summary of the Invention
[0004] This application provides an image distribution system and method to at least solve the technical problem that when transmitting container images, the related technologies directly obtain all the image metadata of the image layer where the image metadata is located, resulting in serious waste of server resources and increased data transmission time.
[0005] According to one aspect of the embodiments of this application, an image distribution system is provided. The image distribution system includes at least one cluster, and each cluster includes an Electronic Program Guide (EPG) center service node and at least one EPG edge service node, wherein: a first EPG edge service node in a first cluster is used to receive an image acquisition request sent by a terminal device for acquiring a target image, wherein the target image includes: target image metadata and target image file data; determine the image level to be pulled that stores the target image metadata, and send a first image metadata acquisition request carrying the image level to be pulled to the first EPG center service node in the first cluster, wherein the first cluster is any one of the at least one cluster in the image distribution system, the first EPG edge service node is any one of the EPG edge service nodes in the first cluster, and the target image includes: target image metadata and target image file data; the first EPG center service node is used to respond to the first image metadata acquisition request carrying the image level to be pulled, and determine the image level to be pulled from the first image repository. The first EPG edge service node receives the target image metadata and sends a target response message carrying the target image metadata back to the first EPG edge service node. The first image repository includes at least one of the following: the first image metadata of each EPG edge service node in the first cluster, and the second image metadata of each second EPG edge service node in other clusters. Other clusters include clusters in the image distribution system other than the first cluster. The first EPG edge service node is also used to receive the target response message carrying the target image metadata, pull the target image file data from the target node in parallel according to the target image metadata, and send the pulled target image file data and target image metadata to the terminal device. The target image file data includes at least one of the following: the first image file data of other EPG edge service nodes in the first cluster, and the second image file data pulled by the first EPG center service node from the second EPG edge service nodes in other clusters. Other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node.
[0006] Optionally, the first image repository includes multiple first image levels, and each first image level is used to store at least one first image metadata.
[0007] Optionally, the first EPG central service node includes: a first image loader, wherein the first image loader is further configured to determine whether the image layer to be pulled is a first image layer within the first image repository, wherein, if the image layer to be pulled is a first target image layer within the first image repository, it retrieves first target image metadata from the first target image layer and sends a first response message carrying the first target image metadata to the first EPG edge service node, wherein the first target image layer is any one of multiple first image layers within the first image repository; if the image layer to be pulled is not any of the first image layers within the first image repository, it determines the second cluster to which the image layer to be pulled belongs based on a preset distributed hash table, wherein the distributed hash table is used to store the hash values corresponding to all image metadata within each cluster in the image distribution system and the image layer information, and the second cluster is a cluster within another cluster; it sends a request to the second EPG central service node of the second cluster to retrieve the second image metadata carrying the image layer to be pulled, receives the second target image metadata from the second EPG central service node, and sends a second response message carrying the second target image metadata to the first EPG edge service node.
[0008] Optionally, the first image loader is also used to send an image file data acquisition request to the second EPG central service node of the second cluster, and to receive the second target image file data fed back by the second image loader of the second EPG central service node and store it in the first image repository. The second image loader is used to pull the second target image metadata file in parallel from the corresponding second EPG edge service node according to the second target image metadata.
[0009] Optionally, the first EPG edge service node includes an edge image loader, wherein the edge image loader is used to aggregate the first image metadata of at least one first image on the first EPG edge service node, and upload the first image metadata to the corresponding first image level in the first image repository according to the first hash value corresponding to the first image metadata of each first image.
[0010] Optionally, the first EPG edge service node further includes: an edge container engine, wherein the edge image loader is also used to calculate the target hash value corresponding to the target image metadata corresponding to the target image file data using a hash function, and determine the image level to be pulled corresponding to the target hash value according to a preset distributed hash table; receive the target response message carrying the target image metadata from the first EPG central service node, and send the target image metadata to the edge container engine; the edge container engine is used to pull the target image file data from the target node in parallel according to the target image metadata.
[0011] Optionally, the system also includes an image metadata center node, wherein the first EPG edge service node is further configured to send a request to the image metadata center node to retrieve the third image metadata carrying the image layer to be retrieved; the image metadata master node is configured to respond to the request to retrieve the third image metadata carrying the image layer to be retrieved, determine the target image metadata corresponding to the image layer to be retrieved from the third image repository, and send a target response message carrying the target image metadata back to the first EPG edge service node, wherein the third image repository is used to store all image metadata of each cluster in the image distribution system.
[0012] According to another aspect of the embodiments of this application, a mirror distribution method is also provided, applied to a first electronic program guide (EPG) edge service node within a first cluster, wherein the first cluster is any one of at least one cluster in a mirror distribution system, and the first EPG edge service node is any one EPG edge service node within the first cluster, comprising: receiving a mirror acquisition request sent by a terminal device for acquiring a target mirror, wherein the target mirror includes: target mirror metadata and target mirror file data; determining the image level to be pulled that stores the target mirror metadata, and sending a first mirror metadata acquisition request carrying the image level to be pulled to a first EPG central service node within the first cluster; receiving a target response message carrying the target mirror metadata fed back by the first EPG central service node, wherein the target mirror metadata is obtained by the first EPG central service node from a first mirror repository within the first cluster. The target image metadata corresponding to the image level to be pulled is determined. The first image repository includes at least one of the following: the first image metadata of each EPG edge service node in the first cluster, and the second image metadata of each second EPG edge service node in other clusters. Other clusters include clusters in the image distribution system other than the first cluster. The target image file data is pulled from the target node in parallel according to the target image metadata, and the pulled target image file data and target image metadata are sent to the terminal device. The target image file data includes at least one of the following: the first image file data of other EPG edge service nodes in the first cluster, and the second image file data pulled by the first EPG center service node from the second EPG edge service nodes in other clusters. Other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node.
[0013] Optionally, determining the image level to be pulled, which stores the target image metadata, includes: using a hash function to calculate the target hash value corresponding to the target image metadata corresponding to the target image file data; and determining the image level to be pulled corresponding to the target hash value based on a preset distributed hash table, wherein the distributed hash table is used to store the hash values and image level information corresponding to all image metadata in each cluster of the image distribution system.
[0014] According to another aspect of the embodiments of this application, a mirror distribution method is also provided, applied to a first Electronic Program Guide (EPG) center service node within a first cluster, wherein the first cluster is any one of at least one cluster in the mirror distribution system, comprising: obtaining a first mirror metadata acquisition request carrying a mirror layer to be pulled sent by a first EPG edge service node within the first cluster, wherein the first mirror metadata acquisition request is generated by the first EPG edge service node receiving a mirror acquisition request sent by a terminal device for acquiring a target mirror, the target mirror including: target mirror metadata and target mirror file data, the mirror layer to be pulled being used to store the target mirror metadata, and the first EPG edge service node being any one of the EPG edge service nodes within the first cluster; responding to the first mirror metadata acquisition request carrying the mirror layer to be pulled, determining the target mirror metadata corresponding to the mirror layer to be pulled from a first mirror repository, wherein the first mirror repository includes at least the following 1. The first image metadata of each EPG edge service node in the first cluster and the second image metadata of each second EPG edge service node in other clusters, including clusters in the image distribution system other than the first cluster; 2. Feedback of a target response message carrying the target image metadata to the first EPG edge service node, wherein the first EPG edge service node is used to receive the target response message carrying the target image metadata, pull the target image file data from the target node in parallel according to the target image metadata, and send the pulled target image file data and target image metadata to the terminal device, and the target image file data includes at least one of the following: the first image file data of other EPG edge service nodes in the first cluster, and the second image file data pulled by the first EPG center service node from the second EPG edge service nodes in other clusters, wherein other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node.
[0015] Optionally, the first image repository includes: multiple first image levels, and each first image level is used to store at least one first image metadata.
[0016] Optionally, responding to a request to retrieve first image metadata carrying the image layer to be pulled, and determining the target image metadata corresponding to the image layer to be pulled from the first image repository, includes: determining whether the image layer to be pulled is a first image layer within the first image repository; wherein, if the image layer to be pulled is a first target image layer within the first image repository, retrieving the first target image metadata from the first target image layer, wherein the first target image layer is any one of multiple first image layers within the first image repository; if the image layer to be pulled is not any of the first image layers within the first image repository, determining the second cluster to which the image layer to be pulled belongs based on a preset distributed hash table, and sending a request to retrieve second image metadata carrying the image layer to be pulled to the second EPG central service node of the second cluster, and receiving the second target image metadata fed back by the second EPG central service node, wherein the distributed hash table is used to store the hash values and image layer information corresponding to all image metadata in each cluster of the image distribution system, and the second cluster is a cluster within another cluster.
[0017] Optionally, after determining the other clusters to which the image layer to be pulled belongs based on a preset distributed hash table, the method further includes: sending an image file data acquisition request to the second EPG central service node of the second cluster, and receiving the second target image file data fed back by the second image loader of the second EPG central service node, and storing it in the first image repository, wherein the second image loader is used to pull the second target image metadata file in parallel from the corresponding second EPG edge service node based on the second target image metadata.
[0018] Optionally, sending a target response message carrying target image metadata to the first EPG edge service node includes: sending a first response message carrying first target image metadata to the first EPG edge service node; or sending a second response message carrying second target image metadata to the first EPG center service node.
[0019] According to another aspect of the embodiments of this application, an electronic device is also provided, the electronic device including: a memory and a processor, wherein the memory stores a computer program, and the processor is configured to execute the above-described image distribution method through the computer program.
[0020] In this embodiment, the first EPG edge service node within the first cluster receives an image retrieval request from the terminal device for obtaining a target image, and sends a first image metadata retrieval request carrying the image layer to be pulled to the first EPG central service node within the first cluster. The first EPG central service node then determines the target image metadata corresponding to the image layer to be pulled from the first image repository and sends a target response message carrying the target image metadata back to the first EPG edge service node. The first EPG edge service node then pulls the target image file data from the target node in parallel based on the target image metadata, and sends the pulled target image file data and target image metadata to the terminal device. This enables image distribution between peer-to-peer internet networks, thus solving the technical problem of related technologies that directly obtain all image metadata of the image layer containing the image metadata when transmitting container images, resulting in severe waste of server resources and increased data transmission time. Attached Figure Description
[0021] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:
[0022] Figure 1 This is an architecture diagram of an optional cloud-edge-device three-tier architecture based on relevant technologies;
[0023] Figure 2 This is a schematic diagram of the structure of an optional mirror distribution system 10 according to an embodiment of this application;
[0024] Figure 3 This is a schematic diagram of the structure of any first cluster within an optional image distribution system according to an embodiment of this application;
[0025] Figure 4 This is a schematic diagram illustrating an optional method of pulling image file data from a local cluster according to an embodiment of this application;
[0026] Figure 5 This is a schematic diagram illustrating an optional method of pulling image file data from another cluster according to an embodiment of this application;
[0027] Figure 6 This is a flowchart illustrating an optional image distribution method according to an embodiment of this application;
[0028] Figure 7 This is a flowchart illustrating another optional image distribution method according to an embodiment of this application. Detailed Implementation
[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0031] Furthermore, all information and data (including but not limited to user device information, user personal information, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with the relevant user or organization. Before obtaining relevant information, it needs to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent from the aforementioned user or organization.
[0032] To better understand the embodiments of this application, the technical terms involved in the embodiments of this application are explained below:
[0033] IPTV (Internet Protocol Television) is a new technology that utilizes broadband cable television networks and integrates multiple technologies such as the Internet, multimedia, and communications to provide home users with a variety of interactive services, including digital television.
[0034] EPG (Electronic Programmer Guide) provides IPTV users with a convenient and quick interface for navigating television programs, allowing users to easily search for and select the television channels and programs they want to watch. It is usually presented in the form of an electronic list or graphical interface, which includes the name of the television channel, the currently playing program, the program schedule, program categories, actor information, and other relevant information.
[0035] Container: An open-source engine that packages software into standardized units for development, interaction, and deployment.
[0036] Example 1
[0037] Containers, as a lightweight, portable, and low-cost virtualization method, have become a popular way for many companies to reduce costs and improve efficiency. Starting a container requires a container image file on your hard drive. The container image is the foundation for starting a container instance and contains all the data needed for the container to run. Before starting a container instance, the complete container image must be downloaded locally.
[0038] This application provides an image distribution system, which includes at least one cluster, and each cluster includes an EPG central service node and at least one EPG edge service node. The system deploys container applications in edge computing scenarios, and its container image distribution typically employs a three-tier cloud-edge-device architecture, as detailed below. Figure 1 As shown, Figure 1 This is an architecture diagram of an optional cloud-edge-device three-tier architecture for related technologies, such as... Figure 1 As shown, the cloud-edge-device three-tier architecture includes the EPG central service node (also known as the image repository node) and EPG edge service nodes 1, 2, and 3 connected to the EPG central service node; devices 1, 2, and 3 connected to EPG edge service node 1; devices 4 and 5 connected to EPG edge service node 2; and devices 6 and 7 connected to EPG edge service node 13. Among these, devices 1-3 pull the required image file data from EPG edge service node 1, devices 4-5 pull the required image file data from EPG edge service node 2, and devices 6-7 pull the required image file data from EPG edge service node 3.
[0039] Specifically, Figure 2 This is a schematic diagram of the structure of an optional mirror distribution system 10 according to an embodiment of this application, as shown below. Figure 2As shown, the image distribution system includes multiple clusters 1, 2, ..., M, where M is a positive integer greater than or equal to 1, and each cluster includes an EPG central service node and an EPG edge service node.
[0040] Figure 3 This is a schematic diagram of the structure of any first cluster within an optional image distribution system according to an embodiment of this application, such as... Figure 3 As shown, the first cluster includes a first EPG central service node 31 and multiple EPG edge service nodes 4N, where N is also a positive integer greater than or equal to 1. The following uses... Figure 3 When any one of the multiple EPG edge service nodes 4N in the first cluster shown pulls an image, the interaction functions between the nodes in the cluster are explained to illustrate the image distribution process, wherein:
[0041] The first EPG edge service node 41 can receive an image retrieval request sent by a terminal device to obtain a target image, wherein the target image includes: target image metadata and target image file data; determine the image level to be pulled that stores the target image metadata, and send a first image metadata retrieval request carrying the image level to be pulled to the first EPG center service node 31 in the first cluster.
[0042] The first cluster is any one of the at least one clusters in the image distribution system 10, the first EPG edge service node 41 is any one of the EPG edge service nodes in the first cluster, and the target image includes: target image metadata and target image file data.
[0043] The first EPG central service node 31 can respond to the request to obtain the first image metadata carrying the image layer to be pulled, determine the target image metadata corresponding to the image layer to be pulled from the first image repository, and send back a target response message carrying the target image metadata to the first EPG edge service node 41. The first image repository includes at least one of the following: the first image metadata of each EPG edge service node in the first cluster, and the second image metadata of each second EPG edge service node in other clusters. Other clusters include: clusters in the image distribution system 10 other than the first cluster.
[0044] In addition, the first EPG edge service node 41 can also receive a target response message carrying target image metadata, pull target image file data from the target node according to the target image metadata, and send the pulled target image file data and target image metadata to the terminal device. The target image file data includes at least one of the following: first image file data of other EPG edge service nodes in the first cluster, and second image file data pulled by the first EPG center service node 31 from second EPG edge service nodes in other clusters. Other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node 41.
[0045] Specifically, the aforementioned terminal devices can refer to the demand side of cloud computing services, such as terminals or user terminals in the Internet, or terminals or user terminals in the Internet of Things (IoT). The aforementioned EPG edge service nodes correspond to EPG central service nodes or traditional cloud computing platforms. The EPG edge service node is a relative concept; typically, an edge service node refers to a cloud computing platform relatively close to the terminal. In other words, it is distinct from a central cloud or traditional cloud computing platform. A central cloud or traditional cloud computing platform can include resource-scaled and geographically concentrated data centers, while edge service nodes cover a wider network range and therefore have the characteristic of being closer to the terminal. A single edge service node has a smaller resource scale. The aforementioned EPG central service node can be a device or server with processing capabilities and strong security protection capabilities, ensuring that the image metadata in the central image repository cannot be easily stolen. Image metadata includes image attribute information, such as image size, name, identifier, or image file identifier, while image file data includes data in the image other than image metadata. Therefore, image metadata and image file data can form a usable image.
[0046] During the process of the first EPG edge service node 41 in the first cluster pulling target image file data, the target image file data is indexed based on the target image metadata to determine its image layer information in the first image repository. This allows the first EPG edge service node 41 to retrieve the corresponding target image metadata from the corresponding image layer, enabling it to pull the target image file data from the corresponding edge node based on the target image metadata. The first image repository of the first EPG center service node 31 includes not only the first image metadata of each EPG edge service node within its own first cluster, but also the second image metadata of each EPG edge service node in multiple clusters within the image distribution system 10. This ensures that the first EPG edge service node 41 in the first cluster can quickly obtain the corresponding target image metadata through the first image repository of the first EPG center service node 31. Furthermore, the first EPG center service node 31 has good security protection capabilities; therefore, storing the image metadata in the first image repository reduces the possibility of image metadata theft and improves the security of image distribution.
[0047] Furthermore, the specific components and corresponding functions of the first EPG central service node 31 and the first EPG edge service node 41 will be described in detail below.
[0048] The first EPG edge service node 41 includes an edge image loader 411, which can aggregate the first image metadata of at least one first image on the first EPG edge service node 41, and upload the first image metadata to the corresponding first image level in the first image repository according to the first hash value corresponding to the first image metadata of each first image.
[0049] The aforementioned first image repository includes multiple first image levels, and each first image level is used to store at least one first image metadata.
[0050] In other words, the edge image loader 411 reports all the first image metadata owned by the first EPG edge service node 41 to the first image repository. In order to facilitate the subsequent retrieval of image metadata, a hash function can be used to calculate the first hash value corresponding to each first image metadata, and the first image metadata can be stored in the corresponding first image layer in the first image repository according to the mapping relationship between the first hash value and the image layer.
[0051] In addition, the aforementioned edge image loader 411 can also use a hash function to calculate the target hash value corresponding to the target image metadata corresponding to the target image file data, and determine the image level to be pulled corresponding to the target hash value based on a preset distributed hash table, and then send a first image metadata retrieval request carrying the image level to be pulled to the first image loader 311 of the first EPG center service node 31.
[0052] Each first EPG central service node 31 has a unique first image repository. Therefore, when the first image loader 311 of the first EPG central service node 31 receives a request from the first EPG edge service node 41 to obtain the first image metadata carrying the image layer to be pulled, it will further perform the following steps:
[0053] First, determine whether the image level to be pulled is the first image level within the first image repository.
[0054] When the image to be pulled is the first target image level in the first image repository, the first target image metadata is obtained from the first target image level, and a first response message carrying the first target image metadata is sent back to the first EPG edge service node 41. The first target image metadata refers to one of the multiple first image metadata stored in the first image level.
[0055] If the image layer to be pulled is not one of the first image layers in the first image repository, the second cluster to which the image layer to be pulled belongs is determined according to a preset distributed hash table. The distributed hash table is used to store the hash values and image layer information corresponding to all image metadata in each cluster in the image distribution system. The second cluster is a cluster within another cluster. A request to obtain the second image metadata of the image layer to be pulled is sent to the second EPG center service node of the second cluster, and the second target image metadata is received from the second EPG center service node. A second response message carrying the second target image metadata is also sent to the first EPG edge service node 41.
[0056] The specific function of the first image loader 311 can be understood as follows: First, the first image loader 311 determines whether the image layer to be pulled is the first target image layer in the local first image repository used to store the first image metadata. If so, it means that the target image file data is the first image metadata of other EPG edge nodes in the first cluster, and the first image loader 311 can obtain the first target image metadata from the first image layer and send a first response message carrying the first target image metadata to the first EPG edge service node 41. Otherwise, it means that the target image file data does not belong to the first cluster. At this time, it can be determined that the target image file data is the second target image metadata, and the second cluster to which the second target image metadata belongs is determined according to the distributed hash table. Then, a request to obtain the second image metadata carrying the image layer to be pulled is sent to the second EPG center service node of the second cluster, and the second target image metadata is received from the second EPG center service node. At this time, the first image loader 311 sends a second response message carrying the second target image metadata to the first EPG edge service node 41.
[0057] In addition, to facilitate the first EPG edge service node 41 in quickly and easily acquiring the target image file data, the first image loader 311 can also send an image file data acquisition request to the second EPG center service node of the second cluster. This request is used to acquire the second target image file data corresponding to the second target image metadata. Correspondingly, it receives the second target image file data fed back by the second image loader of the second EPG center service node and stores it in the first image repository. If the first image loader 311 needs to pull the second target image file data from the second image repositories of multiple second EPG center service nodes, it can use a parallel approach to improve the pulling efficiency.
[0058] Furthermore, after the first image loader 311 of the first EPG central service node 31 sends a second response message carrying the second target image metadata of the second cluster to the first EPG edge service node 41, the edge image loader 411 of the first EPG edge service node 41 can also receive the target response message carrying the target image metadata from the first EPG central service node 31, and send the target image metadata to the edge container engine 412 of the first EPG edge service node 41. The edge container engine 412 can then pull the target image file data from the target node based on the target image metadata.
[0059] The edge container engine 412 pulls target image file data in either a parallel or serial manner. In this embodiment, the parallel method is preferred to improve the efficiency of pulling image file data.
[0060] As an optional example, we will briefly explain the two cases: the image file data belongs to the local cluster and the data does not belong to the local cluster.
[0061] Figure 4 This is a schematic diagram illustrating an optional method of pulling image file data from a local cluster according to an embodiment of this application, such as... Figure 4 As shown, EPG edge node A has container image layers 1 and 2, and EPG edge node B has container image layer 3. Both EPG edge nodes send their image metadata to the image repository through their own image loader. When EPG edge node C needs to load all the image metadata within image layers 1, 2, and 3, it first uses a distributed hash table to determine the location of each image layer in its local image repository. The local image repository then sends all the image metadata back to EPG edge node C, allowing EPG edge node C to pull the corresponding image file data from EPG edge node A and EPG edge node B in parallel based on the image metadata, thereby accelerating image acquisition.
[0062] Figure 5 This is a schematic diagram illustrating an optional method of pulling image file data from another cluster according to an embodiment of this application, such as... Figure 5 As shown, EPG nodes A, B, and C share image metadata within their respective image repositories. When an EPG edge node within the same group as EPG node A wants to retrieve image file data that doesn't belong to its own group but to another group, EPG node A can determine the group to which the retrieved image file data belongs using a distributed hash table. If the group consists of EPG nodes B and C, EPG node A's central image loader can initiate image file data retrieval requests to the central image loaders of both EPG nodes B and C. The central image loaders of EPG nodes B and C then retrieve the image file data from their respective EPG edge nodes within their groups. This allows EPG node A to retrieve image file data from both EPG node B's and EPG node C's image repositories in parallel and store it within the image repository. Subsequently, an EPG edge node within the same group as EPG node A can directly retrieve the image file data from EPG node A's image repository. All of the aforementioned image repositories are... Figure 4 The mirror parameters of the EPG node are shown.
[0063] It should be noted that the above Figure 5In the scenario of image distribution between EPG central nodes, each EPG central node mainly communicates with each other in pairs. This method is only convenient when there are a small number of clusters in the image distribution system. However, if the number of clusters in the image distribution system is large, continuing to use the pairwise communication method will undoubtedly complicate the network.
[0064] Therefore, the image distribution system in this application embodiment may also include an image metadata center node, which also includes a unique third image repository, which is used to store all image metadata of multiple clusters in the image distribution system.
[0065] Specifically, each first EPG center service node 31 sends the first image metadata of each EPG edge service node in the first cluster stored in the first image repository to the second image repository of the second EPG center service node in other clusters through its first image loader 311, and receives the second image metadata of multiple second EPG edge service nodes stored in the second image repository of the second EPG center service node in other clusters.
[0066] In other words, the first image loader 311 of the first EPG center service node 31 can realize the interaction of image metadata between various clusters in the image distribution system, and the first EPG center service node 31 can be used as an image metadata center node.
[0067] Therefore, when the first EPG edge service node 41 sends a request to the image metadata center node to obtain the third image metadata carrying the image layer to be pulled, it can respond to the request to obtain the third image metadata carrying the image layer to be pulled, and directly determine the target image metadata corresponding to the image layer to be pulled from the third image repository, and send back a target response message carrying the target image metadata to the first EPG edge service node 41.
[0068] In other words, the image distribution system establishes a more centralized image metadata storage center to store all the image metadata owned by the image repositories of each cluster. This ensures that when the image file data does not belong to the local cluster, the transfer of image files between the local EPG central node and one or more other EPG central nodes is completed through the image metadata central node.
[0069] The image distribution system described in this application embodiment can realize image distribution between peer-to-peer Internet networks, solve the technical problems of low efficiency and success rate of existing image distribution and large waste of computer resources, and effectively alleviate the pressure of downloading image file data directly from the image repository, thus breaking through the performance bottleneck.
[0070] Example 2
[0071] Based on Embodiment 1 of this application, an embodiment of a mirror distribution method is also provided. This method is applied to a first electronic program guide (EPG) edge service node within a first cluster, wherein the first cluster is any one of at least one cluster in the mirror distribution system, and the first EPG edge service node is any one of the EPG edge service nodes within the first cluster.
[0072] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0073] Figure 6 This is a flowchart illustrating an optional image distribution method according to an embodiment of this application, as shown below. Figure 6 As shown, the method includes at least steps S602-S608, wherein:
[0074] Step S602: Receive an image acquisition request sent by the terminal device for obtaining the target image, wherein the target image includes: target image metadata and target image file data.
[0075] Step S604: Determine the image level to be pulled that stores the target image metadata, and send a request to the first EPG center service node in the first cluster to retrieve the first image metadata carrying the image level to be pulled.
[0076] As an optional implementation, in the technical solution provided in step S604 above, the method may include: using a hash function to calculate the target hash value corresponding to the target image metadata corresponding to the target image file data; determining the image level to be pulled corresponding to the target hash value based on a preset distributed hash table, wherein the distributed hash table is used to store the hash values and image level information corresponding to all image metadata in each cluster of the image distribution system.
[0077] Step S606: Receive the target response message carrying the target image metadata from the first EPG center service node.
[0078] The target image metadata is the target image metadata corresponding to the image level to be pulled, which is determined by the first EPG central service node in the first cluster from the first image repository. The first image repository includes at least one of the following: the first image metadata of each EPG edge service node in the first cluster, and the second image metadata of each second EPG edge service node in other clusters. Other clusters include: clusters in the image distribution system other than the first cluster.
[0079] Furthermore, the edge image loader in the first EPG edge service node can aggregate the first image metadata of at least one first image on the first EPG edge service node, and upload the first image metadata to the corresponding first image level in the first image repository based on the first hash value corresponding to the first image metadata of each first image. Therefore, the aforementioned first image repository includes multiple first image levels, and each first image level is used to store at least one first image metadata.
[0080] Step S608: Pull target image file data from the target node in parallel based on the target image metadata, and send the pulled target image file data and target image metadata to the terminal device.
[0081] Specifically, the edge container engine in the first EPG edge service node pulls target image file data from the target node in parallel based on the target image metadata, and combines the pulled target image file data and target image metadata into a target image, which is then fed back to the terminal device.
[0082] Example 3
[0083] Based on Embodiment 1 of this application, an embodiment of a mirror distribution method is also provided. This method is applied to a first electronic program guide (EPG) center service node within a first cluster, wherein the first cluster is any one of at least one cluster in the mirror distribution system.
[0084] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0085] Figure 7 This is a flowchart illustrating an optional image distribution method according to an embodiment of this application, as shown below. Figure 4 As shown, the method includes at least steps S702-S708, wherein:
[0086] Step S702: Obtain the first image metadata retrieval request sent by the first EPG edge service node in the first cluster, carrying the image layer to be pulled.
[0087] In the technical solution provided in step S702, the first image loader of the first EPG central service node obtains a first image metadata acquisition request carrying the image layer to be pulled. The first image metadata acquisition request is generated by the first EPG edge service node receiving an image acquisition request sent by the terminal device for acquiring the target image. The target image includes: target image metadata and target image file data. The image layer to be pulled is used to store the target image metadata. The first EPG edge service node is any EPG edge service node in the first cluster.
[0088] Step S704: Respond to the request to obtain the first image metadata carrying the image layer to be pulled, and determine the target image metadata corresponding to the image layer to be pulled from the first image repository.
[0089] In the technical solution provided in step S704, the first image repository includes at least one of the following: first image metadata of each EPG edge service node in the first cluster, and second image metadata of each second EPG edge service node in other clusters. The other clusters include clusters in the image distribution system other than the first cluster. Furthermore, since the first image repository includes multiple first image levels, and each first image level stores at least one first image metadata level, the first image loader of the first EPG central service node, after receiving the image level to be pulled, can also perform the following steps:
[0090] First, determine whether the image level to be pulled is the first image level within the first image repository.
[0091] When the image to be pulled is the first target image level in the first image repository, the first target image metadata is obtained from the first target image level, wherein the first target image level is any one of the multiple first image levels in the first image repository.
[0092] If the image layer to be pulled is not one of the first image layers in the first image repository, the second cluster to which the image layer to be pulled belongs is determined according to the preset distributed hash table. A request to obtain the second image metadata of the image layer to be pulled is sent to the second EPG center service node of the second cluster, and the second target image metadata fed back by the second EPG center service node is received. The distributed hash table is used to store the hash value and image layer information corresponding to all image metadata in each cluster in the image distribution system. The second cluster is a cluster within another cluster.
[0093] Furthermore, after determining the other clusters to which the image level to be pulled belongs based on the preset distributed hash table, the first image loader of the first EPG center service node can also send an image file data acquisition request to the second EPG center service node of the second cluster, and receive the second target image file data fed back by the second image loader of the second EPG center service node, and store it in the first image repository. The second image loader is used to pull the second target image metadata file in parallel from the corresponding second EPG edge service node based on the second target image metadata.
[0094] Step S706: Send a target response message carrying the target image metadata to the first EPG edge service node.
[0095] In the technical solution provided in step S706, the first EPG edge service node is used to receive a target response message carrying target image metadata, pull target image file data from the target node in parallel according to the target image metadata, and send the pulled target image file data and target image metadata to the terminal device. The target image file data includes at least one of the following: the first image file data of other EPG edge service nodes in the first cluster, and the second image file data pulled by the first EPG center service node from the second EPG edge service node in other clusters. Other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node.
[0096] Specifically, step S706 further includes: sending a first response message carrying the first image metadata to the first EPG edge service node; or sending a second response message carrying the second image metadata to the first EPG center service node.
[0097] Example 4
[0098] According to an embodiment of this application, an electronic device is also provided, wherein the electronic device includes one or more processors; a memory for storing one or more programs, wherein when one or more programs are executed by one or more processors, the one or more processors are configured to run the programs, wherein the programs are configured to execute the image distribution method in embodiment 2 or the image distribution method in embodiment 3 above.
[0099] Optionally, the processor is configured to execute the following steps via a computer program: receiving an image retrieval request sent by a terminal device for retrieving a target image, wherein the target image includes: target image metadata and target image file data; determining the image level to be pulled that stores the target image metadata, and sending a first image metadata retrieval request carrying the image level to be pulled to the first EPG central service node in the first cluster; receiving a target response message carrying the target image metadata from the first EPG central service node, wherein the target image metadata is the target image metadata corresponding to the image level to be pulled determined by the first EPG central service node in the first cluster from the first image repository, and the first image repository includes at least one of the following: each EPG in the first cluster... The first image metadata of the G edge service node, the second image metadata of each second EPG edge service node in other clusters, the other clusters include: clusters in the image distribution system other than the first cluster; according to the target image metadata, target image file data is pulled from the target node in parallel, and the pulled target image file data and target image metadata are sent to the terminal device, wherein the target image file data includes at least one of the following: the first image file data of other EPG edge service nodes in the first cluster, the second image file data pulled by the first EPG center service node from the second EPG edge service nodes in other clusters, the other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node.
[0100] Optionally, the processor is configured to execute the following steps via a computer program: acquiring a first image metadata acquisition request carrying the image layer to be pulled, sent by a first EPG edge service node within a first cluster, wherein the first image metadata acquisition request is generated by the first EPG edge service node receiving an image acquisition request from a terminal device for acquiring a target image, the target image including: target image metadata and target image file data, the image layer to be pulled being used to store the target image metadata, and the first EPG edge service node being any EPG edge service node within the first cluster; responding to the first image metadata acquisition request carrying the image layer to be pulled, determining the target image metadata corresponding to the image layer to be pulled from the first image repository, wherein the first image repository includes at least one of the following: the first image metadata of each EPG edge service node within the first cluster, other sets... The second image metadata of each second EPG edge service node in the group, and other clusters include: clusters in the image distribution system other than the first cluster; feeding back a target response message carrying the target image metadata to the first EPG edge service node, wherein the first EPG edge service node is used to receive the target response message carrying the target image metadata, pull the target image file data from the target node in parallel according to the target image metadata, and send the pulled target image file data and target image metadata to the terminal device, and the target image file data includes at least one of the following: the first image file data of other EPG edge service nodes in the first cluster, and the second image file data pulled by the first EPG center service node from the second EPG edge service nodes in other clusters, and other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node.
[0101] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0102] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0103] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.
[0104] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.
[0107] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.
Claims
1. A mirror distribution system, characterized in that, The image distribution system includes at least one cluster and an image metadata master node, and each cluster includes an Electronic Program Guide (EPG) center service node and at least one EPG edge service node, wherein: The first EPG edge service node within the first cluster is used to receive an image retrieval request sent by a terminal device for obtaining a target image, wherein the target image includes: target image metadata and target image file data; determine the image level to be pulled that stores the target image metadata, and send a first image metadata retrieval request carrying the image level to be pulled to the first EPG center service node within the first cluster, wherein the first cluster is any one of at least one cluster in the image distribution system, the first EPG edge service node is any EPG edge service node within the first cluster, and the target image includes: target image metadata and target image file data; The first EPG central service node is configured to respond to a request for obtaining first image metadata carrying the image layer to be pulled, determine the target image metadata corresponding to the image layer to be pulled from the first image repository, and send a target response message carrying the target image metadata back to the first EPG edge service node. The first image repository includes at least one of the following: the first image metadata of each EPG edge service node in the first cluster, and the second image metadata of each second EPG edge service node in other clusters. The other clusters include: clusters in the image distribution system other than the first cluster. The first EPG edge service node is further configured to receive the target response message carrying the target image metadata, pull the target image file data from the target node in parallel according to the target image metadata, and send the target image file data and the target image metadata to the terminal device. The target image file data includes at least one of the following: first image file data of other EPG edge service nodes in the first cluster, and second image file data pulled by the first EPG center service node from the second EPG edge service node in the other cluster. The other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node. The first EPG central service node includes: a first image loader, wherein the first image loader is further configured to determine whether the image layer to be pulled is a first image layer within the first image repository; if the image layer to be pulled is a first target image layer within the first image repository, it obtains first target image metadata from the first target image layer and sends a first response message carrying the first target image metadata to the first EPG edge service node, wherein the first target image layer is any first image layer within the first image repository; if the image layer to be pulled is not a first image layer within the first image repository, it determines the second cluster to which the image layer to be pulled belongs based on a preset distributed hash table, sends a request to the second EPG central service node of the second cluster to obtain the second image metadata of the image layer to be pulled, receives the second target image metadata from the second EPG central service node, and sends a second response message carrying the second target image metadata to the first EPG edge service node, wherein the distributed hash table is used to store the hash values and image layer information corresponding to all image metadata in each cluster in the image distribution system, and the second cluster is a cluster within the other clusters; The first EPG edge service node is further configured to send a request to the image metadata center node to obtain third image metadata carrying the image layer to be pulled; the image metadata center node is configured to respond to the request to obtain third image metadata carrying the image layer to be pulled, determine the target image metadata corresponding to the image layer to be pulled from the third image repository, and send back a target response message carrying the target image metadata to the first EPG edge service node, wherein the third image repository is used to store all image metadata of each cluster in the image distribution system.
2. The mirror distribution system according to claim 1, characterized in that, The first image repository includes multiple first image levels, and each first image level is used to store at least one first image metadata.
3. The image distribution system according to claim 1, characterized in that, The first image loader is further configured to send an image file data acquisition request to the second EPG central service node of the second cluster, and to receive the second target image file data fed back by the second image loader of the second EPG central service node, and store it in the first image repository. The second image loader is configured to pull the second target image metadata file in parallel from the corresponding second EPG edge service node according to the second target image metadata.
4. The mirror distribution system according to claim 1, characterized in that, The first EPG edge service node includes: an edge image loader, wherein, The edge image loader is used to aggregate the first image metadata of at least one first image on the first EPG edge service node, and upload the first image metadata to the corresponding first image level in the first image repository according to the first hash value corresponding to the first image metadata of each first image.
5. The mirror distribution system according to claim 4, characterized in that, The first EPG edge service node also includes: an edge container engine, wherein, The edge image loader is further configured to use a hash function to calculate the target hash value corresponding to the target image metadata corresponding to the target image file data, and determine the image level to be pulled corresponding to the target hash value according to a preset distributed hash table; receive the target response message carrying the target image metadata from the first EPG central service node, and send the target image metadata to the edge container engine; The edge container engine is used to pull the target image file data from the target node in parallel based on the target image metadata.
6. A mirror distribution method, characterized in that, The first Electronic Program Guide (EPG) edge service node is applied within a first cluster, where the first cluster is any cluster in the mirror distribution system, and the first EPG edge service node is any EPG edge service node within the first cluster, including: The system receives an image acquisition request sent by a terminal device to obtain a target image, wherein the target image includes: target image metadata and target image file data; Determine the image level to be pulled that stores the target image metadata, and send a first image metadata retrieval request carrying the image level to be pulled to the first EPG center service node in the first cluster; The system receives a target response message carrying target image metadata from the first EPG center service node. The target image metadata is the image metadata corresponding to the image level to be pulled, which is determined by the first EPG center service node in the first cluster from the first image repository. The first image repository includes at least one of the following: the first image metadata of each EPG edge service node in the first cluster, and the second image metadata of each second EPG edge service node in other clusters. The other clusters include: clusters in the image distribution system other than the first cluster. The target image file data is pulled from the target node in parallel according to the target image metadata, and the pulled target image file data and the target image metadata are sent to the terminal device. The target image file data includes at least one of the following: first image file data of other EPG edge service nodes in the first cluster, and second image file data pulled by the first EPG center service node from the second EPG edge service node in the other cluster. The other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node. A request to obtain third image metadata of the image layer to be pulled is sent to the image metadata center node in the image distribution system, and a target response message carrying the target image metadata corresponding to the image layer to be pulled is received from the image metadata center node. The target image metadata is the image metadata corresponding to the image layer to be pulled determined by the image metadata center node from the third image repository, and the third image repository is used to store all image metadata of each cluster in the image distribution system.
7. The mirror distribution method according to claim 6, characterized in that, Determining the image level to be pulled, which stores the target image metadata, includes: A hash function is used to calculate the target hash value corresponding to the target image metadata corresponding to the target image file data; The image level to be pulled is determined according to a preset distributed hash table, which is used to store the hash values and image level information corresponding to all image metadata in each cluster of the image distribution system.
8. A mirror distribution method, characterized in that, The first Electronic Program Guide (EPG) center service node is applied within the first cluster, which is any cluster in the mirror distribution system, including: The system obtains a first image metadata retrieval request sent by the first EPG edge service node within the first cluster, which carries the image layer to be pulled. The first image metadata retrieval request is generated by the first EPG edge service node receiving an image retrieval request sent by the terminal device for retrieving the target image. The target image includes: target image metadata and target image file data. The image layer to be pulled is used to store the target image metadata. The first EPG edge service node is any EPG edge service node within the first cluster. In response to a request to retrieve first image metadata carrying the image layer to be pulled, the target image metadata corresponding to the image layer to be pulled is determined from the first image repository. The first image repository includes at least one of the following: first image metadata of each EPG edge service node in the first cluster, and second image metadata of each second EPG edge service node in other clusters. The other clusters include: clusters in the image distribution system other than the first cluster. The first EPG edge service node sends a target response message carrying the target image metadata back to the first EPG edge service node. The first EPG edge service node is used to receive the target response message carrying the target image metadata, pull target image file data from the target node in parallel according to the target image metadata, and send the pulled target image file data and the target image metadata to the terminal device. The target image file data includes at least one of the following: first image file data of other EPG edge service nodes in the first cluster, and second image file data pulled by the first EPG center service node from the second EPG edge service node in the other cluster. The other EPG edge service nodes include EPG edge service nodes in the first cluster other than the first EPG edge service node. The process of responding to a request to retrieve first image metadata carrying the image layer to be retrieved, and determining the target image metadata corresponding to the image layer to be retrieved from the first image repository, includes: determining whether the image layer to be retrieved is a first image layer within the first image repository; if the image layer to be retrieved is a first target image layer within the first image repository, retrieving first target image metadata from the first target image layer, wherein the first target image layer is any first image layer within the first image repository; if the image layer to be retrieved is not a first image layer within the first image repository, determining the second cluster to which the image layer to be retrieved belongs based on a preset distributed hash table, sending a request to retrieve second image metadata carrying the image layer to be retrieved to the second EPG center service node of the second cluster, and receiving the second target image metadata fed back by the second EPG center service node, wherein the distributed hash table is used to store the hash values and image layer information corresponding to all image metadata in each cluster in the image distribution system, and the second cluster is a cluster within the other clusters.
9. The mirror distribution method according to claim 8, characterized in that, The first image repository includes: multiple first image levels, and each first image level is used to store at least one first image metadata.
10. The mirror distribution method according to claim 8, characterized in that, After determining the other clusters to which the image level to be pulled belongs based on a preset distributed hash table, the method further includes: Send an image file data acquisition request to the second EPG center service node of the second cluster, and receive the second target image file data fed back by the second image loader of the second EPG center service node, and store it in the first image repository. The second image loader is used to pull the second target image metadata file in parallel from the corresponding second EPG edge service node according to the second target image metadata.
11. The mirror distribution method according to claim 10, characterized in that, Send a target response message carrying the target image metadata back to the first EPG edge service node, including: The first response message carrying the metadata of the first target image is sent back to the first EPG edge service node; Alternatively, a second response message carrying the metadata of the second target image can be sent back to the first EPG central service node.
12. An electronic device, characterized in that, include: A memory and a processor, the processor being configured to run a program stored in the memory, wherein the program, when running, performs the image distribution method according to any one of claims 6 to 7 or any one of claims 8 to 11.
Citation Information
Patent Citations
Mirror image storage and distribution system for virtual machines
CN104050015A
Mirror image distribution method, mirror image distribution device, electronic equipment and storage medium
CN114238262A
