A method, apparatus and system for mirror distribution in a network element cluster

Abstract

Embodiments of the present application disclose a mirror distribution method, device and system in a network element cluster. The method comprises: separating a mirror file, sending a hash list of each mirror block to all edge nodes for separate maintenance; distributing all mirror blocks to multiple different edge nodes; analyzing a received mirror transmission request for a hash list carried by the mirror transmission request, finding out a set of edge nodes that have corresponding mirror blocks for each hash value in the hash list, determining an edge node with minimum network delay according to network quality of the edge nodes in the set monitored dynamically, forwarding the mirror transmission request to the edge node for transmission of the corresponding mirror block, and updating mirror block information owned by each edge node after the transmission is completed. The present application utilizes bandwidth of each edge node to transmit the mirror, improves mirror download speed, reduces transmission time, improves scalability of mirror distribution, and ensures reliability and success rate of mirror transmission.

Description

Technical Field

[0001] This invention relates to the field of network technology, and more specifically, to a mirror distribution method, apparatus, and system in a network element cluster. Background Technology

[0002] With the development of mobile communication networks, network element clusters have become a common way to deploy computing resources. A network element cluster can be considered to consist of a central storage node and several edge nodes. Due to the large number of nodes in the cluster, manually deploying software and configuring the environment on each node would be extremely cumbersome and time-consuming. Therefore, the automation and standardization of cluster management and deployment have become particularly important.

[0003] An image, also known as an image file, is a form of file storage. An image can be viewed as a copy of the original file. Image distribution technology can help enable rapid deployment and configuration of nodes in a cluster, improving the efficiency and reliability of cluster management.

[0004] The existing image distribution solutions mainly include the following methods:

[0005] The first approach is the traditional image distribution scheme, where the image resides on a central storage node. Each node requests to download the image from the central storage node, which then transmits the image file to each node sequentially. This scheme, where each node downloads the image from the central node, places excessive demands on the central storage node's bandwidth, easily leading to network bottlenecks. Furthermore, centralized downloading results in low distribution efficiency.

[0006] The second approach involves building a P2P network among cluster nodes, then dividing the image into several blocks and publishing them to the P2P network. Each node downloads image blocks from other nodes, and finally reassembles the downloaded image blocks. This scheme can reduce the load and bandwidth consumption of the source server and improve download speed. However, efficient image distribution through the P2P network requires a certain number of nodes; if the number of nodes is too small, the availability and reliability of the image may be affected.

[0007] The third approach is a binary tree-based image distribution scheme. This scheme divides the image file into multiple blocks and distributes them to multiple image nodes, while organizing the image nodes by constructing a binary tree structure. This scheme is more complex to implement, requiring in-depth research and optimization of the binary tree structure and block-level transmission. Furthermore, excessively deep binary trees can lead to a decrease in network transmission efficiency.

[0008] Overall, existing image distribution schemes place high demands on the management of nodes in the network element cluster and also require a certain number of nodes. Furthermore, existing schemes cannot effectively utilize the bandwidth of each node to transmit images, resulting in poor performance under high load conditions. Summary of the Invention

[0009] The embodiments of the present invention provide a mirror distribution method, apparatus and system in a network element cluster, which can solve the problems of existing mirror distribution schemes having high management complexity of nodes in the network element cluster, certain requirements on the number of nodes, and poor performance under high load conditions, while failing to make good use of the bandwidth of each node to transmit images.

[0010] According to a first aspect of the present invention, a mirror distribution method in a network element cluster is provided, the method being applied to a central storage node, comprising:

[0011] The pre-distributed image file is divided into several blocks, the hash value of each image block is calculated, all hash values ​​are constructed into a hash list, and the hash list is sent to all edge nodes for separate maintenance.

[0012] Based on the network quality of the edge nodes, all mirror blocks are distributed to multiple different edge nodes, and the mirror block information of each edge node is recorded after all mirror blocks have been distributed.

[0013] Upon receiving a mirror transmission request from a first edge node, the mirror transmission request carries a first hash list maintained separately by the first edge node. The first hash list only records the hash values ​​corresponding to the mirror blocks that the first edge node is missing. The mirror transmission request is parsed to obtain the first hash list. For each hash value in the first hash list, the set of edge nodes that have the corresponding mirror blocks is found. Based on the network quality of the edge nodes in the dynamically monitored set, the second edge node with the lowest network latency is determined. The mirror transmission request is forwarded to the second edge node, which then transmits the corresponding mirror blocks to the first edge node. After the transmission is completed, the mirror block information of each edge node is updated.

[0014] Preferably, the above method further includes:

[0015] The busy or idle state of an edge node is determined based on the network speed. When the network speed of an edge node is less than a set network speed value, the edge node is determined to be in a busy state; otherwise, it is in an idle state.

[0016] Upon receiving the mirror transmission request from the first edge node, the central storage node first determines the busy or idle status of each edge node in the set of edge nodes that have the corresponding mirror block. If all edge nodes in the set are busy, the central storage node directly transmits the corresponding mirror block to the first edge node and no longer forwards the mirror transmission request to other edge nodes.

[0017] Preferably, the above method further includes:

[0018] When multiple mirror transfer requests from different edge nodes are received simultaneously, the hash lists carried by each of the multiple mirror transfer requests are parsed, and the multiple mirror transfer requests are processed in descending order of the length of the multiple hash lists.

[0019] According to a second aspect of the present invention, another mirroring distribution method in a network element cluster is provided, the method being applied to edge nodes, comprising:

[0020] The receiving center storage node sends a hash list, which records multiple hash values, each hash value corresponding to one of the image blocks in all the image blocks into which the pre-distributed image file is divided;

[0021] When a mirror block is received, a hash value is calculated for the mirror block and compared with a locally stored hash list. If the same hash value exists in the hash list, the corresponding hash value is deleted from the hash list. If the same hash value does not exist in the hash list, the received mirror block is deleted.

[0022] When a complete image file is needed, an image transfer request is sent to the central storage node. This request carries a locally stored hash list, which only records the hash values ​​corresponding to missing image blocks. Furthermore, if the locally stored hash list is empty, all received image blocks are merged into a complete image file.

[0023] When a mirror transmission request forwarded by the central storage node is received, the forwarded mirror transmission request carries the target edge node identifier and hash value. Based on the forwarded mirror transmission request, the mirror block corresponding to the hash value is transmitted to the target edge node.

[0024] According to a third aspect of the present invention, a mirror distribution device in a network element cluster is provided, the device being applied to a central storage node, comprising:

[0025] The network monitoring module is used to dynamically monitor the network quality of all edge nodes;

[0026] The image segmentation module is used to divide the pre-distributed image file into several blocks, calculate the hash value of each image block, construct a hash list from all hash values, and send the hash list to all edge nodes for individual maintenance.

[0027] The image distribution module is used to distribute all image blocks to multiple different edge nodes based on the network quality of the edge nodes;

[0028] The transmission management module is used to record the image block information of each edge node after all image blocks have been distributed, and when it receives an image transmission request from a first edge node, wherein the image transmission request carries a first hash list maintained separately by the first edge node, the first hash list only records the hash values ​​corresponding to the image blocks missing by the first edge node. The module parses the image transmission request to obtain the first hash list, searches for each hash value in the first hash list to find the set of edge nodes that have the corresponding image blocks, determines the second edge node with the lowest network latency based on the network quality of the edge nodes in the set, forwards the image transmission request to the second edge node, and the second edge node transmits the corresponding image blocks to the first edge node. After the transmission is completed, the module updates the image block information of each edge node.

[0029] Preferably, the network monitoring module is further configured to determine the busy or idle state of the edge node based on the network speed, wherein the edge node is determined to be in a busy state when the network speed of the edge node is less than a set network speed value, and otherwise in an idle state.

[0030] The transmission management module is further configured to, upon receiving a mirror transmission request from the first edge node, first determine the busy or idle status of each edge node in the set of edge nodes that have the corresponding mirror block. If all edge nodes in the set are busy, the central storage node directly transmits the corresponding mirror block to the first edge node and no longer forwards the mirror transmission request to other edge nodes.

[0031] The transmission management module is also used to, when receiving multiple mirror transmission requests from different edge nodes at the same time, parse the hash list carried by each of the multiple mirror transmission requests, and process the multiple mirror transmission requests in descending order of the length of the multiple hash lists.

[0032] According to a fourth aspect of the present invention, another mirror distribution device in a network element cluster is provided, the device being applied to an edge node, comprising:

[0033] The hash list maintenance module is used to receive a hash list sent by the central storage node. The hash list records multiple hash values, each hash value corresponding to one of the image blocks in the pre-distributed image file. When a certain image block is received, the module calculates the hash value of the image block and compares it with the hash list stored locally. If the hash list contains the same hash value, the corresponding hash value is deleted from the hash list. If the hash list does not contain the same hash value, the received image block is deleted.

[0034] The image merging module is used to send an image transfer request to the central storage node when a complete image file is needed. The image transfer request carries a locally stored hash list, which only records the hash values ​​corresponding to missing image blocks. Furthermore, when the locally stored hash list is empty, it merges all received image blocks into a complete image file.

[0035] The mirror transmission module is used to transmit a mirror block corresponding to the hash value to the target edge node when it receives a mirror transmission request forwarded by the central storage node, wherein the forwarded mirror transmission request carries a target edge node identifier and a hash value; according to the forwarded mirror transmission request, the module transmits the mirror block to the target edge node.

[0036] According to a fifth aspect of the present invention, a mirror distribution system in a network element cluster is provided, the system comprising a central storage node and several edge nodes, wherein,

[0037] The central storage node includes a network monitoring module, an image segmentation module, an image distribution module, and a transmission management module, which are used to execute the aforementioned image distribution method;

[0038] Each edge node includes a hash list receiving module, a hash list maintenance module, a mirror merging module, and a mirror transmission module, used to execute the aforementioned mirror distribution method.

[0039] The beneficial effects of the various embodiments of the present invention are as follows:

[0040] The image distribution method, apparatus, and system in a network element cluster provided by this invention, at the central storage node side, divides the image file into several blocks, calculates the hash value of each image block, constructs a hash list of all hash values, and sends it to all edge nodes. Each edge node maintains its own locally stored hash list and verifies the received image blocks based on the hash values, ensuring the reliability of image transmission. By distributing all image blocks to multiple different edge nodes based on their network quality, all image blocks can be quickly distributed to multiple edge nodes, utilizing the network bandwidth of multiple edge nodes to improve image download speed and reduce transmission time. By recording and updating the image block information owned by each edge node, image transmission requests can be redirected to other edge nodes, improving the scalability of image distribution. Furthermore, by dynamically monitoring the network quality of edge nodes, it can be ensured that the selected edge node has good network quality when choosing an edge node for image transmission, improving the success rate of image transmission.

[0041] As can be seen, the solutions provided by the various embodiments of the present invention do not require complex management of nodes in the network element cluster, nor do they have requirements on the number of nodes. Under high load conditions of the central storage node, the bandwidth of each edge node can be effectively utilized to transmit images, improve image download speed, reduce transmission time, and improve the scalability of image distribution, ensuring the reliability and success rate of image transmission. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention, and those skilled in the art can obtain other drawings based on these drawings. In the drawings:

[0043] Figure 1 A flowchart illustrating a mirror distribution method in a network element cluster according to an embodiment of the present invention is shown.

[0044] Figure 2 This diagram illustrates the process of constructing a hash list according to an embodiment of the present invention.

[0045] Figure 3 A flowchart illustrating a mirror distribution method in a network element cluster provided by another embodiment of the present invention is shown.

[0046] Figure 4 This invention illustrates a schematic diagram of the structure of a mirror distribution device in a network element cluster according to an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of the structure of a mirror distribution device in a network element cluster according to another embodiment of the present invention is shown;

[0048] Figure 6 A schematic diagram of the structure of a mirror distribution system in a network element cluster provided by an embodiment of the present invention is shown. Detailed Implementation

[0049] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. These embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Although exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein.

[0050] Figure 1 The diagram illustrates a flow chart of a mirror distribution method in a network element cluster provided by an embodiment of the present invention, which is applied to a central storage node. Figure 1The image distribution method in the network element cluster shown includes the following steps S110 to S130:

[0051] In step S110, the central storage node divides the pre-distributed image file into several blocks, calculates the hash value of each image block, constructs a hash list from all hash values, and sends the hash list to all edge nodes for individual maintenance.

[0052] Image files are typically large, reaching gigabytes (GB) in size. To achieve rapid distribution, the pre-distributed image file needs to be divided into multiple blocks, for example, N blocks, where N is an integer greater than or equal to 2. The division can be either equal or unequal.

[0053] Figure 2 A schematic diagram illustrating the process of constructing a hash list according to an embodiment of the present invention is shown. Figure 2 As shown, firstly, a hash value is calculated for each mirror block, and then all the obtained hash values ​​Hash1, Hash2, ..., Hashn are constructed into a hash list [Hash1, Hash2, ..., Hashn]. There are various ways to calculate the hash value, such as the mid-square method, the remainder method, the integer multiplication method, and the random number method. This embodiment of the invention does not limit the method used to calculate the hash value.

[0054] A hash value serves as the identifier for a mirror block, uniquely identifying it. A list is constructed from the hash values ​​corresponding to all mirror blocks of the image file, resulting in the image file's hash list. This hash list is then sent to all edge nodes. Each edge node can maintain its own locally stored hash list, ensuring, for example, that the locally stored hash list only records the hash values ​​of missing mirror blocks.

[0055] Step S120: Distribute all mirror blocks to multiple different edge nodes according to the network quality of the edge nodes, and record the mirror block information of each edge node after all mirror blocks have been distributed.

[0056] The central storage node dynamically monitors the network quality of all edge nodes, with the monitoring frequency pre-set. Before image distribution, the central storage node needs to obtain the network quality information of all edge nodes in order to distribute all image blocks to multiple different edge nodes based on the network quality of the edge nodes.

[0057] The distribution methods may include:

[0058] 1) Distribution one by one: Each time, select an edge node with the lowest network latency, distribute a mirror block to that edge node, and after the transmission is complete, select another edge node with the lowest network latency, and distribute another mirror block to the newly selected edge node, until all mirror blocks are distributed to multiple different edge nodes. The newly selected edge node is determined by dynamically monitored network quality and can be the same as or different from the previously selected edge nodes.

[0059] 2) Group Distribution: Select the top N edge nodes with the lowest network latency, equal to the number of mirror blocks N. If the number of edge nodes is less than N, and assuming there are K mirror blocks remaining after the first round of distribution, then distribute these K mirror blocks in a new round. Specifically,

[0060] If the number of edge nodes is not less than the number of mirror blocks N, select the top N edge nodes with low network latency that are equal to the number of mirror blocks N, and issue a mirror block to each of the selected edge nodes. After all mirror blocks are issued, these top N edge nodes with low network latency will each have a different mirror block.

[0061] When the number of edge nodes is less than the number of mirror blocks N, a predetermined number of edge nodes with low network latency are selected in two or more rounds. A mirror block is issued to each of the selected edge nodes. After the previous round of issuance is completed, the next round of edge node selection and issuance is carried out until all mirror blocks are issued so that multiple different edge nodes have one or more mirror blocks.

[0062] After all the image blocks are distributed to multiple different edge nodes, the central storage node records the image block information of each edge node so that the transmission request for a certain image block can be redirected to the appropriate node that owns the image block.

[0063] Step S130: Upon receiving a mirror transmission request from a first edge node, wherein the mirror transmission request carries a first hash list maintained separately by the first edge node, the first hash list only records the hash values ​​corresponding to the mirror blocks missing by the first edge node; the central storage node parses the mirror transmission request to obtain the first hash list, searches for each hash value in the first hash list to find the set of edge nodes that have the corresponding mirror blocks, determines the second edge node with the lowest network latency based on the network quality of the edge nodes in the dynamically monitored set, forwards the mirror transmission request to the second edge node, and the second edge node transmits the corresponding mirror blocks to the first edge node, and updates the mirror block information of each edge node after the transmission is completed.

[0064] When an edge node needs a complete image file, it sends an image transfer request to the central storage node. This request includes a hash list of the requested image blocks. Upon receiving the request, the central storage node parses it to obtain the hash list from the edge node, allocates a suitable transfer node according to certain rules, and updates the image block information of each edge node after the transfer is complete. This ensures that subsequent requests can be accurately redirected to the appropriate node, improving the scalability of image distribution.

[0065] Taking a mirror transfer request sent by a first edge node to a central storage node as an example, the mirror transfer request carries a first hash list maintained separately by the first edge node. This first hash list only records the hash values ​​corresponding to the missing mirror blocks of the first edge node. When the central storage node receives the mirror transfer request from the first edge node, it parses the request to obtain the first hash list, and then allocates a suitable node for the mirror transfer request according to the first processing rule defined in the embodiments of this invention.

[0066] The central storage node searches for the set of edge nodes that have the corresponding mirror block for each hash value in the first hash list. Based on the network quality of the edge nodes in the dynamically monitored set, the second edge node with the smallest network latency is determined. The mirror transmission request is forwarded to the second edge node, and the second edge node transmits the corresponding mirror block to the first edge node.

[0067] Specifically, for the first hash value in the first hash list, a set of edge nodes possessing the corresponding mirror block is found. Then, based on the dynamically monitored network quality of the edge nodes in the set, the edge node with the lowest network latency is determined, such as the second edge node. The central storage node forwards the request to the second edge node, which then transmits the mirror block corresponding to the first hash value to the first edge node. Next, for the second hash value in the first hash list, a set of edge nodes possessing the corresponding mirror block is found. Then, based on the dynamically monitored network quality of the edge nodes in the set, the edge node with the lowest network latency is determined, such as the third edge node. The central storage node forwards the request to the third edge node, which then transmits the mirror block corresponding to the second hash value to the first edge node. ...This process continues until the mirror transmission request for the last hash value in the first hash list is also forwarded by the central storage node to a suitable edge node.

[0068] However, following only the first processing rule presents a problem: when the central storage node processes a hash value from the first hash list, if the network quality of all edge nodes in the set containing the corresponding mirror block is poor, even if the edge node with the lowest network latency (e.g., the fourth edge node) is determined based on dynamically monitored network quality, its current network quality is insufficient to complete the transmission of the corresponding mirror block. Therefore, it must abandon the transmission of the mirror block, resulting in transmission failure, and the first edge node cannot receive all the mirror blocks in a timely manner.

[0069] To address the aforementioned issues, one embodiment of the present invention further establishes a second processing rule for allocating a suitable node for the mirror transfer request:

[0070] The busy or idle state of an edge node is determined based on the network speed. When the network speed of an edge node is less than a set network speed value, the edge node is determined to be in a busy state; otherwise, it is in an idle state.

[0071] When the central storage node receives the mirror transmission request from the first edge node, it first determines the busy or idle status of each edge node in the set of edge nodes that have the corresponding mirror block. If all edge nodes in the set are busy, the central storage node directly transmits the corresponding mirror block to the first edge node and does not forward the mirror transmission request to other edge nodes.

[0072] Taking the above scenario as an example, when the central storage node processes a hash value in the first hash list, if the network quality of all edge nodes in the set containing the corresponding mirror block is poor, then according to the second processing rule, for the set of edge nodes containing the corresponding mirror block, the busy or idle status of each edge node in the set is first determined. If all edge nodes in the set are busy, the central storage node directly transmits the corresponding mirror block to the first edge node, without forwarding the mirror transmission request to other edge nodes. Therefore, in the above scenario, instead of solely following the first processing rule to forward the mirror transmission request to the determined fourth edge node with the lowest network latency, the transmission failure of the corresponding mirror block is effectively avoided. Even when the network quality of all edge nodes in the set is poor, it can still ensure that the first edge node receives all the mirror blocks in a timely manner.

[0073] In reality, the central storage node may simultaneously receive mirror transfer requests from multiple different edge nodes. To address this, one embodiment of the present invention further defines a third processing rule for allocating suitable nodes to mirror transfer requests:

[0074] When multiple mirror transfer requests are received simultaneously from different edge nodes, the central storage node parses the hash list carried by each of the multiple mirror transfer requests and processes the multiple mirror transfer requests in descending order of the length of the multiple hash lists.

[0075] A longer hash list indicates a larger number of image block files being sent in the request. This third processing rule prioritizes requests from edge nodes with longer hash lists, which can ensure load balancing to some extent.

[0076] Figure 3 A flowchart illustrating a mirror distribution method in a network element cluster provided by another embodiment of the present invention is shown, which is applied to edge nodes. Figure 3 The image distribution method in the network element cluster shown includes the following steps S310 to S340:

[0077] In step S310, the edge node receives a hash list sent by the central storage node. The hash list records multiple hash values, and each hash value corresponds to one of the image blocks in all the image blocks divided into the pre-distributed image file.

[0078] As mentioned earlier, the central storage node divides the pre-distributed image files into several blocks, calculates the hash value of each image block, constructs a hash list from all hash values, and sends the hash list to all edge nodes so that each edge node can maintain its own locally stored hash list, for example, ensuring that the hash list in the local storage only records the hash values ​​of missing image blocks.

[0079] Step S320: When a certain mirror block is received, calculate the hash value of the mirror block and compare it with the hash list stored locally. If there is a matching hash value in the hash list, delete the corresponding hash value in the hash list. If there is no matching hash value in the hash list, delete the received mirror block.

[0080] The edge node uses step S320 to verify the received image block using a hash value, and at the same time maintains the hash list stored locally separately.

[0081] The verification method is as follows: Calculate the hash value for each received image block and compare it with the locally stored hash list. If the hash list contains the same hash value, it means that the received image block is complete and belongs to one of the image blocks in the image file. To mark that the image block has been completely received, the corresponding hash value in the hash list is deleted. If the hash list does not contain the same hash value, this corresponds to two situations: one is that the received image block has been received before and is a duplicate image block; the other is that the received image block is incomplete or incorrect and does not belong to one of the image blocks in the image file. In both of these situations, the received image block needs to be deleted.

[0082] Step S330: When a complete image file is needed, an image transfer request is sent to the central storage node. The image transfer request carries a hash list stored locally. The hash list only records the hash values ​​corresponding to the missing image blocks. And when the hash list stored locally is empty, all received image blocks are merged into a complete image file.

[0083] Edge nodes maintain a separate hash list in their local storage to ensure that the hash list only records the hash values ​​corresponding to the missing image blocks. Thus, when a complete image file is needed, the edge nodes only send a transmission request for the missing image blocks to the central storage node, avoiding the repeated receipt of image blocks.

[0084] When the hash list stored locally is empty, it means that all the image blocks of the image file have been received and each received image block has passed the integrity and duplication checks. At this point, all the received image blocks can be merged into a complete image file.

[0085] Step S340: When a mirror transmission request forwarded by the central storage node is received, the forwarded mirror transmission request carries the target edge node identifier and hash value. According to the forwarded mirror transmission request, the mirror block corresponding to the hash value is transmitted to the target edge node.

[0086] As mentioned earlier, when the central storage node receives a mirror transmission request from the first edge node, it determines the second edge node with the lowest network latency based on the network quality of the edge nodes in the dynamically monitored set, and forwards the mirror transmission request to the second edge node. Once the second edge node receives the mirror transmission request forwarded by the central storage node, it transmits the mirror block corresponding to the hash value to the first edge node according to the first edge node identifier and hash value carried in the forwarded mirror transmission request. This achieves the utilization of the network bandwidth of the edge nodes, improves the mirror download speed, and reduces the transmission time.

[0087] It should be noted that there is no specific order among the steps S320, S330, and S340. When the conditions for a certain step are met, the corresponding step will be executed.

[0088] In summary, the image distribution method in the network element cluster provided by this invention, on the central storage node side, divides the image file into several blocks, calculates the hash value of each image block, constructs a hash list of all hash values, and sends it to all edge nodes. Each edge node maintains its own locally stored hash list and verifies the received image blocks based on the hash values, ensuring the reliability of image transmission. By distributing all image blocks to multiple different edge nodes based on their network quality, all image blocks can be quickly distributed to multiple edge nodes, utilizing the network bandwidth of multiple edge nodes to improve image download speed and reduce transmission time. By recording and updating the image block information owned by each edge node, image transmission requests can be redirected to other edge nodes, improving the scalability of image distribution. Furthermore, by dynamically monitoring the network quality of edge nodes, it can be ensured that the selected edge node has good network quality when choosing an edge node for image transmission, improving the success rate of image transmission.

[0089] As can be seen, the solutions provided by the various embodiments of the present invention do not require complex management of nodes in the network element cluster, nor do they have requirements on the number of nodes. Under high load conditions of the central storage node, the bandwidth of each edge node can be effectively utilized to transmit images, improve image download speed, reduce transmission time, and improve the scalability of image distribution, ensuring the reliability and success rate of image transmission.

[0090] and Figure 1 The mirror distribution methods in the network element cluster shown belong to the same technical concept. An embodiment of the present invention also provides a mirror distribution device in the network element cluster. Figure 4 This diagram illustrates the structure of a mirror distribution device in a network element cluster according to an embodiment of the present invention. Figure 4 The mirror distribution device 400 in the network element cluster shown is applied to the central storage node and includes:

[0091] The network monitoring module 401 is used to dynamically monitor the network quality of all edge nodes;

[0092] The image segmentation module 402 is used to divide the pre-distributed image file into several blocks, calculate the hash value of each image block, construct a hash list from all hash values, and send the hash list to all edge nodes for individual maintenance.

[0093] The image distribution module 403 is used to distribute all image blocks to multiple different edge nodes according to the network quality of the edge nodes;

[0094] The transmission management module 404 is used to record the image block information owned by each edge node after all image blocks have been distributed; and when receiving an image transmission request from a first edge node, wherein the image transmission request carries a first hash list maintained separately by the first edge node, the first hash list only records the hash values ​​corresponding to the image blocks missing by the first edge node, the image transmission request is parsed to obtain the first hash list, and for each hash value in the first hash list, the set of edge nodes that own the corresponding image block is found, and the second edge node with the smallest network latency is determined according to the network quality of the edge nodes in the set, the image transmission request is forwarded to the second edge node, the second edge node transmits the corresponding image block to the first edge node, and the image block information owned by each edge node is updated after the transmission is completed.

[0095] In a preferred embodiment, the network monitoring module 401 is further configured to determine the busy or idle state of the edge node based on the network speed, wherein the edge node is determined to be in a busy state when the network speed of the edge node is less than a set network speed value, and otherwise in an idle state.

[0096] The transmission management module 404 is further configured to, upon receiving a mirror transmission request from the first edge node, first determine the busy or idle state of each edge node in the set of edge nodes that have the corresponding mirror block. If all edge nodes in the set are busy, the central storage node directly transmits the corresponding mirror block to the first edge node and no longer forwards the mirror transmission request to other edge nodes.

[0097] In another preferred embodiment, the transmission management module 404 is further configured to, when receiving multiple mirror transmission requests from different edge nodes simultaneously, parse the hash lists carried by each of the multiple mirror transmission requests, and process the multiple mirror transmission requests in descending order of the length of the multiple hash lists.

[0098] Figure 4 The implementation process of each module in the illustrated device can be found in [reference needed]. Figure 1 The method embodiments shown will not be described again here.

[0099] and Figure 3 The mirror distribution methods in the network element cluster shown belong to the same technical concept. One embodiment of the present invention also provides another mirror distribution device in the network element cluster. Figure 5 A schematic diagram of the structure of a mirror distribution device in a network element cluster according to another embodiment of the present invention is shown. Figure 5 The mirror distribution device 500 in the network element cluster shown is applied to edge nodes and includes:

[0100] The hash list maintenance module 501 is used to receive a hash list sent by the central storage node. The hash list records multiple hash values, each hash value corresponding to one of the image blocks in the pre-distributed image file. When a certain image block is received, the module calculates the hash value of the image block and compares it with the hash list stored locally. If the hash list contains the same hash value, the corresponding hash value is deleted from the hash list. If the hash list does not contain the same hash value, the received image block is deleted.

[0101] The image merging module 502 is used to send an image transmission request to the central storage node when a complete image file is needed. The image transmission request carries a hash list stored locally. The hash list only records the hash values ​​corresponding to the missing image blocks. When the hash list stored locally is empty, all received image blocks are merged into a complete image file.

[0102] The mirror transmission module 503 is used to transmit a mirror block corresponding to the hash value to the target edge node when it receives a mirror transmission request forwarded by the central storage node, wherein the forwarded mirror transmission request carries a target edge node identifier and a hash value; according to the forwarded mirror transmission request, the module transmits the mirror block to the target edge node.

[0103] Figure 5 The implementation process of each module in the illustrated device can be found in [reference needed]. Figure 3 The method embodiments shown will not be described again here.

[0104] The mirror distribution method and apparatus in the aforementioned network element cluster belong to the same technical concept. One embodiment of the present invention also provides a mirror distribution system in the network element cluster. Figure 6 A schematic diagram of the structure of a mirror distribution system in a network element cluster provided by an embodiment of the present invention is shown. Figure 6 The mirror distribution system in the network element cluster shown includes a central storage node 610 and several edge nodes 620. The central storage node 610 includes a network monitoring module, a mirror segmentation module, a mirror distribution module, and a transmission management module, used to execute... Figure 1 The image distribution method shown; each edge node 620 includes a hash list maintenance module, an image merging module, and an image transmission module, used to perform... Figure 3 The image distribution method shown. Specifically,

[0105] On the central storage node 610 side: The network monitoring module dynamically monitors the network quality of all edge nodes. The image partitioning module divides the pre-distributed image files into several blocks, calculates the hash value of each image block, constructs a hash list from all hash values, and sends the hash list to all edge nodes for individual maintenance. The image distribution module distributes all image blocks to multiple different edge nodes based on the network quality of the edge nodes. The transmission management module records the image block information of each edge node after all image blocks have been distributed. When receiving an image transmission request from the first edge node, which carries a first hash list maintained separately by the first edge node (the first hash list only records the hash values ​​corresponding to the image blocks missing by the first edge node), the module parses the image transmission request to obtain the first hash list. For each hash value in the first hash list, it searches for the set of edge nodes that possess the corresponding image block. Based on the network quality of the edge nodes in the set, it determines the second edge node with the lowest network latency, forwards the image transmission request to the second edge node, and the second edge node transmits the corresponding image block to the first edge node. After the transmission is completed, the module updates the image block information of each edge node.

[0106] Preferably, the network monitoring module is further configured to determine the busy or idle state of an edge node based on the network speed. Specifically, if the network speed of an edge node is less than a set network speed value, the edge node is determined to be in a busy state; otherwise, it is in an idle state. The transmission management module is further configured to, upon receiving a mirror transmission request from a first edge node, first determine the busy or idle state of each edge node in the set of edge nodes possessing the corresponding mirror block. If all edge nodes in the set are busy, the central storage node directly transmits the corresponding mirror block to the first edge node, without forwarding the mirror transmission request to other edge nodes.

[0107] Preferably, the transmission management module described above is further configured to, when receiving mirror transmission requests from multiple different edge nodes simultaneously, parse the hash lists carried by each of the multiple mirror transmission requests, and process the multiple mirror transmission requests in descending order of the length of the multiple hash lists.

[0108] On edge node 620: The hash list maintenance module receives a hash list sent by the central storage node. This hash list records multiple hash values, each corresponding to one of the image blocks in a pre-distributed image file. When an image block is received, the module calculates its hash value and compares it with the locally stored hash list. If a matching hash value exists in the hash list, the corresponding hash value is deleted. If no matching hash value exists, the received image block is deleted. The image merging module sends an image transfer request to the central storage node when a complete image file is needed. This request carries a locally stored hash list, which only records the hash values ​​corresponding to missing image blocks. When the locally stored hash list is empty, all received image blocks are merged into a complete image file. The image transfer module receives an image transfer request forwarded by the central storage node, which carries a target edge node identifier and a hash value. Based on the forwarded image transfer request, the module transfers the image block corresponding to the hash value to the target edge node.

[0109] Figure 6 The implementation process of each module in the system shown can be found in [reference needed]. Figure 1 and Figure 3 The method embodiments shown will not be described again here.

[0110] Finally, it should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0111] The above are merely embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for mirror distribution in a cluster of network elements, characterized in that, The method is applied to a central storage node, and comprises: dividing a pre-distributed mirror image file into a plurality of blocks, calculating a hash value of each mirror image block, constructing a hash list from all the hash values, and sending the hash list to all edge nodes for individual maintenance; distributing all the mirror image blocks to a plurality of different edge nodes according to network quality of the edge nodes, and recording information of the mirror image blocks owned by each edge node after the distribution of all the mirror image blocks is completed; when a mirror image transmission request of a first edge node is received, analyzing the mirror image transmission request to obtain a first hash list, finding out a set of edge nodes owning corresponding mirror image blocks for each hash value in the first hash list one by one, determining a second edge node with the minimum network delay according to network quality of the edge nodes in the set dynamically monitored, forwarding the mirror image transmission request to the second edge node, and transmitting corresponding mirror image blocks from the second edge node to the first edge node, and updating information of the mirror image blocks owned by each edge node after the transmission is completed, wherein the mirror image transmission request carries the first hash list maintained by the first edge node individually, and the first hash list only records hash values corresponding to mirror image blocks lacking in the first edge node.

2. The method of claim 1, wherein, The method further comprises: determining a busy state or an idle state of an edge node according to a network rate, wherein the edge node is determined to be in the busy state when the network rate of the edge node is less than a set network rate value, and otherwise in the idle state; when the mirror image transmission request of the first edge node is received, firstly judging the busy state or the idle state of each edge node in the found set of edge nodes owning corresponding mirror image blocks, and if all the edge nodes in the set are in the busy state, directly transmitting corresponding mirror image blocks from the central storage node to the first edge node, and not forwarding the mirror image transmission request to other edge nodes.

3. The method of claim 1 or 2, wherein, The method further comprises: when a plurality of mirror image transmission requests of different edge nodes are received simultaneously, analyzing hash lists carried by the plurality of mirror image transmission requests respectively, and processing the plurality of mirror image transmission requests in a descending order of lengths of the plurality of hash lists.

4. A method for mirror distribution in a cluster of network elements, characterized in that, The method is applied to an edge node, and comprises: receiving a hash list sent by a central storage node, wherein the hash list records a plurality of hash values, and each hash value corresponds to one of all mirror image blocks divided from a pre-distributed mirror image file; when a certain mirror image block is received, calculating a hash value of the mirror image block, and comparing the hash value with a locally stored hash list, if there is a same hash value in the hash list, deleting the corresponding hash value in the hash list, and if there is no same hash value in the hash list, deleting the received mirror image block; when a complete mirror image file is needed, sending a mirror image transmission request to the central storage node, wherein the mirror image transmission request carries the locally stored hash list, and the hash list only records hash values corresponding to lacking mirror image blocks; and when the locally stored hash list is empty, combining all the received mirror image blocks into the complete mirror image file; and When receiving the mirrored transmission request forwarded by the center storage node, the mirrored transmission request carries a target edge node identifier and a hash value, and according to the mirrored transmission request, the corresponding mirror block of the hash value is transmitted to the target edge node.

5. An apparatus for mirror distribution in a cluster of network elements, the apparatus comprising: The device is applied to a center storage node and comprises: a network monitoring module for dynamically monitoring the network quality of all edge nodes; a mirror segmentation module for separating a pre-distributed mirror file into a plurality of blocks, calculating the hash value of each mirror block, constructing a hash list from all hash values, and sending the hash list to all edge nodes for separate maintenance; a mirror distribution module for distributing all mirror blocks to a plurality of different edge nodes according to the network quality of the edge nodes; a transmission management module for, when receiving a mirror transmission request of a first edge node, analyzing the mirror transmission request to obtain a first hash list, finding out a set of edge nodes that own the corresponding mirror block for each hash value in the first hash list, determining a second edge node with the minimum network delay according to the dynamically monitored network quality of the edge nodes in the set, forwarding the mirror transmission request to the second edge node, transmitting the corresponding mirror block from the second edge node to the first edge node, and updating the mirror block information owned by each edge node after the transmission is completed, wherein the mirror transmission request carries a first hash list maintained by the first edge node alone, and the first hash list only records the hash values corresponding to the missing mirror blocks of the first edge node.

6. The device according to claim 5, wherein the network monitoring module is further configured to determine the busy state or idle state of an edge node according to the network rate, wherein the edge node is determined to be in the busy state when the network rate of the edge node is less than a set network rate value, and otherwise in the idle state; the transmission management module is further configured to, when receiving the mirror transmission request of the first edge node, first determine the busy state or idle state of each edge node in the found set of edge nodes that own the corresponding mirror block, and if all edge nodes in the set are in the busy state, directly transmit the corresponding mirror block from the center storage node to the first edge node, and no longer forward the mirror transmission request to other edge nodes.

7. The apparatus of claim 5 or 6, wherein, the transmission management module is further configured to, when simultaneously receiving a plurality of mirror transmission requests of different edge nodes, analyze the hash lists carried by the plurality of mirror transmission requests respectively, and process the plurality of mirror transmission requests in the order from large to small according to the lengths of the plurality of hash lists.

8. An apparatus for mirror distribution in a cluster of network elements, the apparatus comprising: The device is applied to an edge node and comprises: a hash list maintenance module configured to receive a hash list sent by the central storage node, the hash list recording a plurality of hash values, each hash value corresponding to one of all image blocks separated from a pre-distributed image file, and when receiving a certain image block, calculating a hash value of the image block and comparing the hash value with a locally stored hash list, if there is a same hash value in the hash list, deleting the corresponding hash value in the hash list, and if there is no same hash value in the hash list, deleting the received image block; an image merging module configured to, when a complete image file is needed, send an image transmission request to the central storage node, the image transmission request carrying a locally stored hash list, the hash list recording only hash values corresponding to missing image blocks, and when the locally stored hash list is empty, merging all received image blocks into a complete image file; and an image transmission module configured to, when receiving an image transmission request forwarded by the central storage node, the forwarded image transmission request carrying a target edge node identifier and a hash value, according to the forwarded image transmission request, transmitting an image block corresponding to the hash value to the target edge node. 9.A mirror distribution system in a network element cluster, the system comprising a central storage node and a plurality of edge nodes, characterized in that the central storage node comprises a network monitoring module, an image segmentation module, an image distribution module and a transmission management module, and is configured to perform the mirror distribution method of any one of claims 1 to 3; each edge node comprises a hash list maintenance module, an image merging module and an image transmission module, and is configured to perform the mirror distribution method of claim 4.

Images