File distribution method, device and system, electronic equipment and storage medium

Abstract

The invention provides a file distribution method, device and system, electronic equipment and a storage medium, and the method comprises the steps: determining an issuing mode of a current task when a task distribution command is received; when the issuing mode of the current task is a large-scale issuing mode, marking the issuing mode of the current task as the large-scale issuing mode, and storing file data of the current task to a file server; and distributing the marked command of the distribution task to a first scheduling center in a scheduling center cluster, so that the first scheduling center distributes the command of the distribution task to a corresponding distribution transfer station in each transfer cluster for scheduling according to an edge node corresponding to the distribution task when receiving the command of the distribution task. According to the invention, the distribution command is issued to each transfer cluster for forwarding, and the files are quickly distributed to the specified nodes in a large scale, so that the network transmission time is effectively saved, and the distribution pressure of the central service node is reduced.

Description

Technical Field

[0001] The present invention relates to the field of file distribution technology, and in particular to a file distribution method, device, system, electronic device and computer-readable storage medium. Background Art

[0002] With the rapid development of cloud computing, resources around the world are closely connected through cloud technology. Edge machines realize unified management and scheduling of resources through Kubernetes and kubeedge technology, forming a huge content distribution network (CDN, Content Delivery Network).

[0003] In the related technology, when CDN nodes provide services externally, they often need to quickly send various files, such as configuration files, installation packages, and image files, to meet the needs of different nodes to provide services externally. However, since edge machines are geographically far apart, belong to different operators and are in different network segments, if files are distributed directly from the central node to the distant edge machines, the transmission channel becomes lengthy, which not only increases the transmission delay, but also consumes a lot of traffic and causes problems such as untimely task feedback. At the same time, the file service source station will be under tremendous pressure when the edge machines frequently download files.

[0004] Therefore, how to achieve fast and efficient file distribution between large-scale CDN edge nodes and reduce the huge pressure on the file service source station caused by edge machines downloading files is a problem that needs to be solved. Summary of the invention

[0005] The present invention provides a file distribution method, device, system, electronic device and computer-readable storage medium to at least solve the problem in the related art that the central node distributes files to the edge machines at a distance, resulting in a large amount of traffic consumption, serious cross-network transmission delay, untimely task feedback, and increased download pressure on the file service source station. The technical solution of the present invention is as follows:

[0006] According to a first aspect of an embodiment of the present invention, a file distribution method is provided, the method being applied to a central service node, comprising:

[0007] When receiving a command to distribute a task, determine the distribution mode of the current task;

[0008] When the delivery mode of the current task is a large-scale delivery mode, marking the delivery mode of the current task as a large-scale delivery mode, and storing the file data of the current task on a file server;

[0009] The marked distribution task command is distributed to the first dispatch center in the dispatch center cluster, so that when the first dispatch center receives the distribution task command, it distributes the distribution task command to the corresponding distribution transfer stations in each transfer cluster for scheduling according to the edge nodes corresponding to the distribution task.

[0010] Optionally, the method further includes:

[0011] When the delivery mode of the current task is a small-scale direct push mode, marking the delivery mode of the current task as a small-scale direct push mode;

[0012] The marked task distribution command and the file data of the current task are sent to the second scheduling center in the scheduling center cluster, so that the second scheduling center directly distributes the file data of the current task to the corresponding edge node when receiving the task distribution command.

[0013] Optionally, when receiving a command to distribute a task, determining a distribution mode of the current task includes:

[0014] When receiving the command to distribute tasks, obtain the file size of the current task, the number of edge nodes of the current task, and the number of tasks per unit time;

[0015] Determine the size of the current traffic according to the file size of the current task, the number of edge nodes of the current task, and the number of tasks per unit time;

[0016] According to the size of the current traffic, it is determined that the delivery mode of the current task is a large-scale delivery mode or a small-scale direct push mode.

[0017] Optionally, determining, according to the size of the current traffic, whether the delivery mode of the current task is a large-scale delivery mode or a small-scale direct push mode includes:

[0018] Determine whether the current flow rate exceeds the set threshold;

[0019] When it is determined that the current flow exceeds the set threshold, determining that the delivery mode of the current task is a large-scale delivery mode; or

[0020] When it is determined that the size of the current traffic does not exceed the set threshold, the delivery mode of the current task is determined to be a small-scale direct push mode.

[0021] According to a second aspect of an embodiment of the present invention, a file distribution method is provided, the method being applied to each scheduling center in a scheduling center cluster, comprising:

[0022] When receiving the task distribution command sent by the central service node, determine the distribution mode of the current task;

[0023] When the delivery mode of the current task is a large-scale delivery mode, determining a first delivery transfer station in a transfer cluster to which an edge node to which the current task is delivered belongs;

[0024] Calling the preheating interface of the file cache server of the first distribution transfer station, obtaining the file data of the current task from the file server, and caching it on the file cache server;

[0025] The command of the distribution task is distributed to the first distribution transfer station, so that the first distribution transfer station distributes the command of the distribution task to the corresponding edge node, and the edge node obtains the file data of the current task from the file cache server corresponding to the first distribution transfer station.

[0026] Optionally, the method further includes:

[0027] When the delivery mode of the current task is a small-scale direct push mode, determine a second distribution transfer station in the transfer cluster to which the edge node of the direct push of the current task belongs;

[0028] The command of the distribution task and the file data are directly pushed to the second distribution transfer station, so that the second distribution transfer station distributes the command of the distribution task and the file data directly to the corresponding edge node, and the edge node directly stores the received file data of the current task in the specified directory.

[0029] Optionally, determining the first distribution transfer station in the transfer cluster to which the edge node to which the current task is distributed belongs includes:

[0030] According to the attribute information of the edge node distributed by the current task, the distribution transfer stations of the transfer cluster where the edge node is located are grouped, and the distribution transfer station with healthy heartbeat in each group is determined as the first distribution transfer station in the transfer cluster to which the edge node of the corresponding group belongs, wherein the attribute information includes: the IP address of the edge node and / or the location information of the edge node;

[0031] The step of determining a second distribution transfer station in a transfer cluster to which an edge node for current task distribution belongs includes:

[0032] According to the attribute information of the edge nodes distributed for the current task, the distribution transit stations of the transit cluster where the edge nodes are located are grouped, and the distribution transit station with the healthiest center jump in each group is determined as the second distribution transit station in the transit cluster to which the edge nodes of the corresponding group belong, wherein the attribute information includes: the IP address of the edge node and / or the location information of the edge node.

[0033] According to a third aspect of an embodiment of the present invention, a file distribution device is provided, the device being applied to a central service node, comprising:

[0034] A determination module, used to determine the distribution mode of the current task when receiving a command to distribute the task;

[0035] A first marking module, used for marking the delivery mode of the current task as the large-scale delivery mode when the delivery mode of the current task is the large-scale delivery mode, and storing the file data of the current task on the file server;

[0036] The first distribution module is used to distribute the marked distribution task command to the first dispatching center in the dispatching center cluster, so that when the first dispatching center receives the distribution task command, it distributes the distribution task command to the distribution transfer stations in each transfer cluster for scheduling according to the edge nodes corresponding to the distribution task.

[0037] Optionally, the device further comprises:

[0038] A second marking module is used to mark the delivery mode of the current task as the small-scale direct push mode when the delivery mode of the current task is the small-scale direct push mode;

[0039] The second distribution module is used to send the marked distribution task command and the file data of the current task to the second scheduling center in the scheduling center cluster, so that when the second scheduling center receives the distribution task command, it directly distributes the file data of the current task to the corresponding edge node.

[0040] Optionally, the determining module includes:

[0041] An acquisition module is used to obtain the file size of the current task, the number of edge nodes of the current task, and the number of tasks per unit time when receiving a command to distribute a task;

[0042] A traffic determination module, used to determine the size of the current traffic according to the file size of the current task, the number of edge nodes of the current task and the number of tasks per unit time;

[0043] The mode determination module is used to determine whether the delivery mode of the current task is a large-scale delivery mode or a small-scale direct push mode according to the size of the current traffic.

[0044] Optionally, the mode determination module includes:

[0045] A judgment module is used to judge whether the current flow exceeds a set threshold;

[0046] A first delivery mode determination module, configured to determine that the delivery mode of the current task is a large-scale delivery mode when the judgment module determines that the current flow exceeds a set threshold;

[0047] The second sending mode determination module is used to determine that the sending mode of the current task is a small-scale direct push mode when the judgment module determines that the current traffic size does not exceed the set threshold.

[0048] According to a fourth aspect of an embodiment of the present invention, a file distribution device is provided, the device being applied to each scheduling center in a scheduling center cluster, comprising:

[0049] A mode determination module is used to determine the delivery mode of the current task when receiving a command to distribute the task sent by the central service node;

[0050] A first transfer determination module, configured to determine, when the mode determination module determines that the delivery mode of the current task is a large-scale delivery mode, a first distribution transfer station in a transfer cluster to which an edge node for the current task distribution belongs;

[0051] A calling module, used for calling a preheating interface of a file cache server of the first distribution transfer station, obtaining file data of a current task from the file server, and caching the file data on the file cache server;

[0052] The first distribution module is used to distribute the command of the distribution task to the first distribution transfer station, so that the first distribution transfer station distributes the command of the distribution task to the corresponding edge node, and the edge node obtains the file data of the current task from the file cache server corresponding to the first distribution transfer station.

[0053] Optionally, the device further comprises:

[0054] A second transfer determination module is used to determine a second distribution transfer station in the transfer cluster to which the edge node of the current task direct push belongs when the mode determination module determines that the task delivery mode is a small-scale direct push mode;

[0055] The second distribution module is used to push the command of the distribution task and the file data directly to the second distribution transfer station, so that the second distribution transfer station distributes the command of the distribution task and the file data directly to the corresponding edge node, and the edge node directly stores the received file data of the current task in the specified directory.

[0056] Optionally, the first transit determination module is specifically used to group the distribution transit stations of the transit cluster where the edge nodes are located according to the attribute information of the edge nodes distributed by the current task when the mode determination module determines that the delivery mode of the current task is a large-scale delivery mode, and determine the distribution transit station with a healthy center jump in each group as the first distribution transit station in the transit cluster to which the edge nodes of the corresponding group belong, wherein the attribute information includes: the IP address of the edge node and / or the location information of the edge node;

[0057] The second transit determination module is specifically used to group the distribution transit stations of the transit cluster where the edge nodes are located according to the attribute information of the edge nodes distributed for the current task when the mode determination module determines that the delivery mode of the task is a small-scale direct push mode, and determine the distribution transit station with a healthy heartbeat in each group as the second distribution transit station in the transit cluster to which the edge nodes of the corresponding group belong, wherein the attribute information includes: the IP address of the edge node and / or the location information of the edge node.

[0058] According to a fifth aspect of an embodiment of the present invention, a file distribution system is provided, the system comprising: a central service node, a file server, a scheduling center cluster, a distribution transfer cluster and a file cache service cluster, wherein:

[0059] The central service node is used to determine the delivery mode of the current task when receiving the command to distribute the task, and when the delivery mode of the current task is the large-scale delivery mode, mark the delivery mode of the current task as the large-scale delivery mode, store the file data of the current task on the file server, and distribute the marked delivery task command to the first scheduling center in the scheduling center cluster; or when it is determined that the delivery mode of the current task is the small-scale direct push mode, mark the delivery mode of the current task as the small-scale direct push mode in the command; and deliver the marked delivery task command and the file data of the current task to the second scheduling center in the scheduling center cluster;

[0060] The file server is used to store the file data sent by the central service node and provide an external download function;

[0061] Each dispatch center in the dispatch center cluster is used to determine the dispatch mode of the current task when receiving the command of distributing the task sent by the central service node; when the dispatch mode of the current task is a large-scale dispatch mode, determine the first distribution transfer station in the distribution transfer cluster to which the edge node for distributing the current task belongs; call the preheating interface of the file cache server of the first distribution transfer station, obtain the file data of the current task from the file server, and cache it on the file cache server; distribute the command of the distribution task to the first distribution transfer station; or, when it is determined that the dispatch mode of the current task is a small-scale direct push mode, determine the second distribution transfer station in the distribution transfer cluster to which the edge node for direct pushing the current task belongs; push the command of the distribution task and the file data directly to the second distribution transfer station;

[0062] Each file cache server in the file cache service cluster is used to provide a preheating interface for the distribution transfer station in the corresponding transfer cluster, temporarily cache the file data of the current task obtained from the file server, and provide download services;

[0063] Each distribution transfer station in the distribution transfer cluster is used to distribute the command of the distribution task to the corresponding edge node when receiving the command of the distribution task sent by the corresponding scheduling center, so that the edge node obtains the file data of the current task from the file cache server of the corresponding first distribution transfer station; or, when receiving the command of the distribution task and the file data sent by the corresponding scheduling center, directly distribute the command of the distribution task and the file data to the corresponding edge node, and the edge node directly stores the received file data of the current task in the designated directory.

[0064] According to a sixth aspect of an embodiment of the present invention, there is provided an electronic device, comprising:

[0065] processor;

[0066] a memory for storing instructions executable by the processor;

[0067] The processor is configured to execute the instructions to implement the file distribution method as described above.

[0068] According to a seventh aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the file distribution method as described above.

[0069] According to an eighth aspect of an embodiment of the present invention, there is provided a computer program product, comprising a computer program or instructions, which implement the file distribution method as described above when executed by a processor of an electronic device.

[0070] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects:

[0071] In the embodiment of the present invention, when receiving the command of the distribution task, the issuing mode of the current task is determined; when the issuing mode of the current task is the large-scale issuing mode, the issuing mode of the current task is marked as the large-scale issuing mode, and the file data of the current task is stored on the file server; the marked command of the distribution task is distributed to the first dispatch center in the dispatch center cluster, so that when the first dispatch center receives the command of the distribution task, it distributes the command of the distribution task to the corresponding distribution transfer station in each transfer cluster according to the edge node corresponding to the distribution task for scheduling. That is to say, in the embodiment of the present invention, the service of the file distribution task command is sunk to the distribution transfer station in the transfer cluster to reduce the pressure of the central service node to distribute the command and shorten the network link. Since the distribution transfer station and the managed edge node are usually in the same network segment, the embodiment of the present invention utilizes this feature to distribute the task distribution command to the distribution transfer station in each transfer cluster, and the nearest distribution transfer station is responsible for the distribution of the file download command, which effectively saves network transmission time, reduces the distribution pressure of the central service node, and enhances the reliability and efficiency of the system.

[0072] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention, and do not constitute an improper limitation of the present invention. In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0074] Figure 1 The present invention provides a flowchart of a method for distributing files.

[0075] Figure 2 This is another flow chart of a file distribution method provided by an embodiment of the present invention.

[0076] Figure 2A It is a schematic diagram of group scheduling of target machines based on current task distribution provided by an embodiment of the present invention.

[0077] Figure 3 This is an application example diagram of a file distribution method provided by an embodiment of the present invention.

[0078] Figure 4 The present invention provides a block diagram of a file distribution device.

[0079] Figure 5 It is another block diagram of a file distribution device provided by an embodiment of the present invention.

[0080] Figure 6 The present invention provides a file distribution system block diagram.

[0081] Figure 7 It is an application block diagram of a file distribution system provided by an embodiment of the present invention.

[0082] Fig. 7A This is a schematic diagram of the distribution of machines in the entire network provided by an embodiment of the present invention.

[0083] Figure 7B This is a schematic diagram of edge cluster machine distribution provided by an embodiment of the present invention.

[0084] Figure 8 It is a block diagram of an electronic device provided by an embodiment of the present invention.

[0085] Fig. 9 The invention is a block diagram of a device for distributing files provided by an embodiment of the invention. DETAILED DESCRIPTION

[0086] In order to enable ordinary persons in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0087] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0088] Figure 1 is a flowchart of a file distribution method provided by an embodiment of the present invention, wherein the method is applied to a central service node, such as Figure 1 As shown, the method comprises the following steps:

[0089] Step 101: upon receiving a command to distribute a task, determining a distribution mode of the current task.

[0090] Step 102: When the delivery mode of the current task is a large-scale delivery mode, mark the delivery mode of the current task as a large-scale delivery mode, and store the file data of the current task on a file server.

[0091] Step 103: Distribute the marked distribution task command to the first dispatch center of the dispatch center cluster, so that when the first dispatch center receives the distribution task command, it distributes the distribution task command to the corresponding distribution transfer stations in each cluster for scheduling according to the edge nodes corresponding to the distribution task.

[0092] In the embodiment of the present invention, the received distribution command is sent to the distribution transfer station (i.e., management machine) of each transfer cluster, and the distribution transfer station in each cluster is responsible for the distribution of the file download command, that is, the distribution transfer station of each cluster distributes the command of the distribution task to the corresponding edge node in the edge cluster, and the edge node can download the file data of the current task from the file cache server of the distribution transfer station in the cluster, which effectively saves network transmission time, reduces the distribution pressure of the central service node, realizes the rapid large-scale distribution of files to the designated node, and reduces the pressure of the source station (file server) to download the file service. It also enhances the reliability and efficiency of the system.

[0093] The file distribution method described in the present invention can be applied to terminals, servers, etc., without limitation herein. The terminal implementation device can be an electronic device such as a smart phone, a laptop computer, a tablet computer, a desktop computer, a personal digital assistant (PDA, Personal Digital Assistant) and a wearable device. The server can be an independent server or a server cluster, or a server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, intermediate services, domain name services, security services, content distribution networks, or a big data and artificial intelligence platform, etc., without limitation herein.

[0094] Combine the following Figure 1 , the specific implementation steps of a file distribution method provided by an embodiment of the present invention are described in detail.

[0095] In step 101, when a command to distribute a task is received, the distribution mode of the current task is determined.

[0096] In this step, after receiving the command of the file distribution task, the central service node first determines the delivery mode of the current task, specifically including: obtaining the size of the current file, the number of edge nodes of the current task, and the number of tasks per unit time; according to the file size of the current task, the number of edge nodes of the current task and the number of tasks per unit time, determining the size of the current traffic; according to the size of the current traffic, determining whether the delivery mode of the current task is a large-scale delivery mode or a small-scale direct push mode.

[0097] Wherein, determining the size of the current traffic according to the file size of the current task, the number of edge nodes of the current task and the number of tasks per unit time includes:

[0098] After receiving the command for the file distribution task, the central service calculates the current flow Q based on the file size, the scale of the edge nodes issued (the number of edge nodes), and the number of tasks within 1 minute of the current system. The calculation formula is:

[0099]

[0100] Among them: F now Indicates the file size of the current task, M now represents the target machine number of the current task, t represents the number of tasks within 1 minute, and F i represents the file size of the i-th task, M i represents the number of machines for the i-th task, where the number of task machines can be understood as the number of edge nodes.

[0101] Afterwards, the central service node determines whether the current traffic size exceeds the set threshold; when it is determined that the current traffic size exceeds the set threshold, it determines that the delivery mode of the current task is a large-scale delivery mode; or when it is determined that the current traffic size does not exceed the set threshold, it determines that the delivery mode of the current task is a small-scale direct push mode.

[0102] In step 102, when the delivery mode of the current task is the large-scale delivery mode, the delivery mode of the current task is marked as the large-scale delivery mode, and the file data of the current task is stored in a file server.

[0103] In this step, when the delivery mode of the current task is a large-scale delivery mode, the central service node pushes the file data of the delivery task to the file server for storage first, and marks the task as a large-scale delivery mode in the command; of course, the task can also be marked as a large-scale delivery mode in the command first, and then the file data of the delivery task is pushed to the file server for storage.

[0104] In step 103, the marked distribution task command is distributed to the first dispatching center in the dispatching center cluster, so that when the first dispatching center receives the distribution task command, it distributes the distribution task command to the corresponding distribution transfer stations in each transfer cluster for scheduling according to the edge nodes corresponding to the distribution task.

[0105] In this step, the central service node sends the marked command to the first dispatch center in the dispatch center cluster, for example, to the North China dispatch center in the dispatch center cluster. When the first dispatch center receives the command to distribute the task, it distributes the command to the corresponding distribution transfer stations in each transfer cluster according to the edge nodes corresponding to the distribution task for dispatching.

[0106] Among them, the dispatching center cluster is a general term for the central clusters of each major region, among which the central clusters of each major region can include the North China cluster, South China cluster, Northeast cluster, East China cluster, Central China cluster, Southwest cluster, Northwest cluster, etc.

[0107] In an embodiment of the present invention, when a command for distributing a task is received, the issuing mode of the current task is determined; when the issuing mode of the current task is a large-scale issuing mode, the issuing mode of the current task is marked as a large-scale issuing mode, and the file data of the current task is stored on a file server; the marked command for distributing the task is distributed to the first dispatch center in the dispatch center cluster, so that when the first dispatch center receives the command for distributing the task, it distributes the command for distributing the task to the corresponding distribution transfer stations in each transit cluster for scheduling according to the edge nodes corresponding to the distribution task. That is to say, in an embodiment of the present invention, the service of the file distribution task command is sunk to the distribution transfer station in the transit cluster (the distribution transfer station is also called the management machine of the edge cluster, usually composed of two primary and standby management machines) to reduce the pressure of the central service node in distributing commands and shorten the network link. Since the distribution transfer station and the managed edge nodes are usually in the same network segment, the embodiment of the present invention utilizes this feature to distribute the task distribution commands to the distribution transfer stations in each transfer cluster. The nearest distribution transfer station is responsible for the distribution of file download commands, which effectively saves network transmission time, reduces the distribution pressure of the central service node, and enhances the reliability and efficiency of the system.

[0108] Optionally, in another embodiment, based on the above embodiment, the method may further include: when the delivery mode of the current task is a small-scale direct push mode, marking the delivery mode of the current task as a small-scale direct push mode in the command; sending the marked distribution task command and the file data of the current task to the second scheduling center in the scheduling center cluster, so that the second scheduling center will directly distribute the file data of the current task to the corresponding edge node when receiving the distribution task command.

[0109] In this embodiment, when the central service node determines that the delivery mode of the current task is a small-scale direct push mode, it marks the task as a small-scale direct push mode in the command, and sends the marked command to the corresponding scheduling center in the scheduling center cluster (such as the second scheduling center, etc.), and carries the file data, that is, the command and the file data are sent together to the scheduling center of the corresponding cluster.

[0110] That is to say, in this embodiment, when the central service node determines that the size of the current traffic Q exceeds the set threshold, it determines that the delivery mode of the current task is a large-scale delivery mode, and first pushes the file data of the delivery task to the file server for storage, and then marks the task as a large-scale delivery mode in the command, and finally, sends the marked command to the scheduling center of the corresponding cluster; otherwise, that is, it is determined that the size of the current traffic Q does not exceed the set threshold, then the task is marked as a small-scale direct push mode in the command, and when the marked command is sent to the scheduling center of the corresponding cluster, the file data is carried, that is, the command and the file data are sent to the scheduling center of the corresponding cluster together.

[0111] The embodiment of the present invention is based on the distribution of edge nodes of the content distribution network CND, combined with the needs of CDN business to distribute files on a large scale across the entire network, and the needs of some newly listed clusters to distribute configuration files, and proposes a method of direct push distribution for small files and small-scale machines, and preheating and nearby scheduling of large-scale files across the entire network to achieve rapid distribution of files, that is, by sinking the service of distributing download instructions to each distribution transfer station (management machine), and scheduling by the corresponding scheduling center, the link distance for the instruction to arrive is reduced, the network transmission time is effectively saved, the speed of issuing download commands is improved, and the distribution pressure of the central service node is reduced.

[0112] See also Figure 2 , is another flow chart of a file distribution method provided by an embodiment of the present invention, the method is used for each scheduling center in a scheduling center cluster, and includes:

[0113] Step 201: upon receiving a task distribution command sent by a central service node, determining a distribution mode of the current task.

[0114] In this step, when the dispatch center receives the command to distribute the task sent by the central service node, it can determine the delivery mode of the current task based on the mark in the command to distribute the task, for example, determine whether the delivery mode of the current task is a large-scale delivery mode or a small-scale direct push mode.

[0115] Step 202: When it is determined that the delivery mode of the current task is a large-scale delivery mode, determine the first delivery transfer station in the transfer cluster to which the edge node to which the current task is distributed belongs.

[0116] In this step, the dispatch center can group the distribution transfer stations of the transit cluster where the edge nodes are located according to the attribute information of the edge nodes of the current task distribution, and determine a healthy distribution transfer station in each group of center jumps as the first distribution transfer station in the transit cluster to which the edge nodes of the corresponding group belong, wherein the first distribution transfer station can also be understood as the distribution transfer station in the first transit cluster, and other similar descriptions are the same. Among them, the attribute information includes: the IP address of the edge node and / or the location information of the edge node, such as which province, which municipality, etc. Of course, the attribute information can also adaptively include other information, which is not limited in this embodiment. Among them, in this embodiment, the transit cluster is a general term for provincial administrative division clusters, wherein the provincial administrative division clusters can include: Beijing, Shanghai, Guangdong, Inner Mongolia, Fujian, Qinghai, Gansu, etc., and this embodiment does not give examples one by one.

[0117] Among them, in this embodiment, the scheduling center will group (or classify) the distribution transfer stations of the transfer cluster where the edge node is located according to the attribute information of the edge node (target machine) of the current task distribution, and determine a distribution transfer station with a healthy center jump in each group for scheduling. The schematic diagram of the group scheduling is as follows Figure 2A As shown, Figure 2A A schematic diagram of group scheduling based on a target machine (i.e., an edge node) for current task distribution provided by an embodiment of the present invention specifically includes:

[0118] like Figure 2AAs shown, the dispatch center will group the target machines (i.e. edge nodes) to which the tasks are to be sent according to their IP addresses (here, this implementation takes IP addresses as examples, such as 1.1.12; 1.1.1.3; 2.2.2.2; 2.2.2.3; 2.2.3.2; 11.2.3.2, etc.) according to the management machines where they are located (the management machines are deployed with distribution transfer stations and file cache services). Then, it is determined whether the heartbeat of the grouped management machines is healthy. If it is healthy, it means that the management machine service is normal, and the command is distributed to the distribution transfer station of the management machine. If the heartbeat is not healthy, it means that the management machine service is not normal, and it is necessary to select a transfer distribution station from the nearby management machines (for example: the computer room in the adjacent city) for transfer service; if there is no suitable management machine nearby, a random one is selected from the existing, healthy management machines as the transfer service; finally, the command to distribute the task is sent to the corresponding management machine. In addition, in this embodiment, the file data is carried while sending the distribution command in the direct push mode, and for the large-scale mode, the file cache service where a distribution transfer station in the transfer cluster is located will be called to preheat the file.

[0119] In this embodiment, when the dispatch center receives the command to distribute the task (i.e., the file distribution command), if the distribution mode of this task is a large-scale distribution mode, it will first call the preheating interface of the file cache server to control the file cache server to pull the file from the source station (file server or file service database, etc.), and the pulling method can be P2P, etc. to perform sharding or block pulling, and store the sharded or block pulled files on the local disk, and provide the file download function to the outside through nginx load balancing. In this way, when the edge execution agent (i.e., the edge execution agent service of the edge node, etc.) receives the command to distribute the task, it can directly pull the file from the file cache server in the cluster without having to pull the file from the file server, thereby avoiding all machines from pulling files from the file server, thereby effectively reducing the distribution pressure on the file server.

[0120] Step 203: calling the preheating interface of the file cache server of the first distribution transfer station, obtaining the file data of the current task from the file server, and caching it on the file cache server.

[0121] In this step, when the dispatch center determines the distribution transfer station in the transfer cluster to which the edge node to which the current task is distributed belongs, it calls the preheating interface of the file cache server of the first distribution transfer station, obtains the file data of the current task from the file server, and caches it on the file cache server.

[0122] Among them, the method of obtaining the file data of the current task from the file server can adopt the P2P method to pull files in pieces or blocks. The specific P2P downloading method is a well-known technology and will not be repeated here. Of course, other downloading methods can also be adopted, and this embodiment does not limit it.

[0123] Step 204: Distribute the command of the distribution task to the first distribution transfer station, so that the first distribution transfer station distributes the command of the distribution task to the corresponding edge node, and the edge node obtains the file data of the current task from the file cache server corresponding to the first distribution transfer station.

[0124] In this step, the dispatch center distributes the command of the distribution task to the first distribution transfer station, and the first distribution transfer station distributes the command of the distribution task to the corresponding edge node. The edge execution agent of the edge node obtains the file data of the current task from the file cache server of the corresponding first distribution transfer station, and stores the obtained file data of the current task in the specified directory.

[0125] The distribution transfer station in this embodiment is responsible for distributing the received distribution file command to the edge execution agent service on the designated edge node, and the edge execution agent service collects and reports the task status, etc.

[0126] That is to say, after the edge execution agent service receives the command to distribute the task, it will directly save the file data according to the task distribution mode. If the task distribution mode is direct push mode, it will directly save the file data. If the task distribution mode is large-scale distribution mode, it will pull the file from the file cache server and save the pulled file in the specified directory.

[0127] Optionally, in another embodiment, based on the above embodiment, after downloading the file, the edge execution agent will report the download result and completion time to the distribution transfer station, and the distribution transfer station will record the result in redis, where Redis is an open source memory data structure storage system, which can also be used as a database, cache and message agent, etc. Then, the dispatch center will pass the task completion data to the central service node, and finally, the central service will record the received results in the database and provide them to the user for query. In addition, the system provides a complete link tracking function and a callback function for completing tasks, which can timely discover and troubleshoot the failure of sending.

[0128] Optionally, in another embodiment, based on the above embodiment, the method may also include: when it is determined that the delivery mode of the current task is a small-scale direct push mode, determining a distribution transfer station in the transfer cluster to which the edge node to which the current task is directly pushed belongs, referred to as a second distribution transfer station. The second distribution transfer station may also be understood as a distribution transfer station in the second transfer cluster, and other similar descriptions are the same; directly pushing the command of the distribution task and the file data to the second distribution transfer station, so that the second distribution transfer station distributes the command of the distribution task and the file data directly to the corresponding edge node, and the edge node directly stores the received file data of the current task in the specified directory.

[0129] Among them, the distribution transit station in the transit cluster to which the edge node to which the current task distribution belongs is determined is called the second distribution transit station, including: according to the attribute information of the edge node for the current task distribution, the distribution transit station of the transit cluster where the edge node is located is grouped according to the attribute information of the edge node for the current task distribution, and the distribution transit station with a healthy center jump in each group is determined as the second distribution transit station in the transit cluster to which the edge node of the corresponding group belongs, wherein the attribute information includes: the IP address of the edge node and / or the location information of the edge node.

[0130] Optionally, in another embodiment, based on the above embodiment, a re-issuance mechanism may be executed when a service fails or is upgraded.

[0131] When the dispatch center or distribution transfer station is upgraded or restarted due to a failure, it will read the relevant data from the database at startup and re-issue all unissued tasks to the corresponding edge execution agent. The edge execution agent will also use a retry mechanism every time it issues a task to ensure that the file can be accurately downloaded on the corresponding machine.

[0132] The file distribution method provided by the embodiment of the present invention adopts a hierarchical distribution architecture and has flexible scalability. For the access of a small cluster or a single machine in a newly added computer room, as long as the cluster information of the access cluster is stored in the CDN platform that specifically manages machine data, it can be quickly accessed to the system to achieve machine (such as edge nodes, etc.) scheduling and file distribution. In addition, the migration of machines in different edge clusters and the re-listing after a failure are all well compatible, thereby improving the flexibility and scalability of the system.

[0133] See also Figure 3 , is an application example diagram of a file distribution method provided by an embodiment of the present invention, the method comprising:

[0134] Step 301: The central service node receives a command for a file distribution task.

[0135] Step 302: The central service node obtains the file size of the current task, the number of edge nodes of the current task, and the number of tasks per unit time.

[0136] Step 303: The central service node determines the size of the current traffic according to the file size of the current task, the number of edge nodes of the current task and the number of tasks per unit time.

[0137] Step 304: The central service node determines whether the current traffic volume exceeds the set threshold. If so, execute steps 305 to 311; otherwise, execute step 312.

[0138] Step 305: The central service node determines that the delivery mode of the current task is a large-scale delivery mode.

[0139] Step 306: When the delivery mode of the current task is the large-scale delivery mode, the central service node marks the delivery mode of the current task as the large-scale delivery mode, and stores the file data of the current task on the file server.

[0140] Step 307: The central service node distributes the marked distribution task command to the first scheduling center in the scheduling center cluster.

[0141] Step 308: When the first scheduling center receives the task distribution command sent by the central service node, it determines the distribution mode of the current task;

[0142] Step 309: When the first dispatching center determines that the delivery mode of the current task is a large-scale delivery mode, the first dispatching center determines the first delivery transfer station in the transfer cluster to which the edge node to which the current task is delivered belongs;

[0143] Step 310: The first dispatch center calls the preheating interface of the file cache server of the first distribution transfer station, obtains the file data of the current task from the file server, and caches the file data on the file cache server.

[0144] Step 311: The first dispatching center distributes the command of the distribution task to the first distribution transfer station, so that the first distribution transfer station distributes the command of the distribution task to the corresponding edge node, and the execution agent service of the edge node (i.e., edge execution agent) obtains the file data of the current task from the file cache server corresponding to the first distribution transfer station, and saves the file data to the specified directory.

[0145] Step 312: When the delivery mode of the current task is the small-scale direct push mode, the central service node marks the delivery mode of the current task as the small-scale direct push mode.

[0146] Step 313: The central service node sends the marked command for distributing the task and the file data of the current task to the second scheduling center in the scheduling center cluster.

[0147] Step 314: When the second scheduling center receives the command to distribute the task and the file data of the current task sent by the central service node, it determines that the distribution mode of the current task is the small-scale direct push mode.

[0148] Step 315: The second dispatch center determines a distribution transfer station in the transfer cluster to which the edge node to which the current task is directly pushed belongs, which is called the second distribution transfer station.

[0149] Step 316: The second dispatching center pushes the command of the distribution task and the file data directly to the second distribution transfer station, so that the second distribution transfer station distributes the command of the distribution task and the file data directly to the corresponding edge node, and the execution agent service of the edge node (i.e., edge execution agent) saves the received file data directly to the designated directory.

[0150] In this embodiment, the implementation process of each step is detailed in the above embodiment and will not be repeated here.

[0151] In the embodiment of the present invention, large-scale file distribution across the entire network is supported. By sinking the service of distributing download instructions (distribution transfer station) to the management machine and scheduling by the scheduling center, the link distance for the instruction to arrive is reduced and the speed of issuing download commands is improved.

[0152] In the embodiment of the present invention, through the hierarchical architecture design of file distribution, the edge node (i.e., the target machine) can be flexibly and conveniently connected to the system, and the newly listed node file (i.e., machine file) is initialized for download, providing a convenient file distribution channel.

[0153] In an embodiment of the present invention, the command to distribute files is sent to the distribution transfer station (i.e., the management machine) of the transfer cluster, thereby realizing the machine's nearby message sending scheduling technology. The command can be sent to a distribution transfer station (i.e., the management machine) that is closer according to the machine's geographical location or network topology relationship, thereby reducing the delay in message transmission and improving the efficiency and real-time performance of file sending.

[0154] In the embodiments of the present invention, file distribution can flexibly distribute files of different sizes and machine tasks. According to the distribution mode of the distributed files, file preheating or file direct push is adopted, which can effectively reduce the distribution pressure of the file server, improve the concurrency and stability of file distribution, and speed up the file distribution, thereby improving the overall performance and reliability of the system.

[0155] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that this implementation disclosure is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0156] Figure 4 4 is a block diagram of a file distribution device provided by an embodiment of the present invention. The device is applied to a central service node and includes: a determination module 401, a first marking module 402 and a first distribution module 403, wherein:

[0157] The determination module 401 is used to determine the distribution mode of the current task when receiving the command to distribute the task;

[0158] A first marking module 402, configured to mark the delivery mode of the current task as the large-scale delivery mode when the delivery mode of the current task is the large-scale delivery mode, and store the file data of the current task on a file server;

[0159] The first distribution module 403 is used to distribute the marked distribution task command to the first scheduling center in the scheduling center cluster, so that when the first scheduling center receives the distribution task command, it distributes the distribution task command to the distribution transfer stations in each transfer cluster for scheduling according to the edge nodes corresponding to the distribution task.

[0160] Optionally, in another embodiment, based on the above embodiment, the device further includes:

[0161] A second marking module is used to mark the delivery mode of the current task as the small-scale direct push mode when the delivery mode of the current task is the small-scale direct push mode;

[0162] The second distribution module is used to send the marked distribution task command and the file data of the current task to the second scheduling center in the scheduling center cluster, so that when the second scheduling center receives the distribution task command, it directly distributes the file data of the current task to the corresponding edge node.

[0163] Optionally, in another embodiment, based on the above embodiment, the determining module includes:

[0164] An acquisition module is used to obtain the file size of the current task, the number of edge nodes of the current task, and the number of tasks per unit time when receiving a command to distribute a task;

[0165] A traffic determination module, used to determine the size of the current traffic according to the file size of the current task, the number of edge nodes of the current task and the number of tasks per unit time;

[0166] The mode determination module is used to determine whether the delivery mode of the current task is a large-scale delivery mode or a small-scale direct push mode according to the size of the current traffic.

[0167] Optionally, in another embodiment, based on the above embodiment, the mode determination module includes:

[0168] A judgment module is used to judge whether the current flow exceeds a set threshold;

[0169] A first delivery mode determination module, configured to determine that the delivery mode of the current task is a large-scale delivery mode when the judgment module determines that the current flow exceeds a set threshold;

[0170] The second sending mode determination module is used to determine that the sending mode of the current task is a small-scale direct push mode when the judgment module determines that the current traffic size does not exceed the set threshold.

[0171] See also Figure 5 , another block diagram of a file distribution device provided by an embodiment of the present invention, the device is applied to each dispatch center in the dispatch center cluster, including: a mode determination module 501, a first transfer determination module 502, a calling module 503 and a first distribution module 504, wherein,

[0172] The mode determination module 501 is used to determine the delivery mode of the current task when receiving the command for distributing the task sent by the central service node;

[0173] The first transfer determination module 502 is used to determine the first distribution transfer station in the transfer cluster to which the edge node of the current task distribution belongs when the mode determination module determines that the distribution mode of the current task is a large-scale distribution mode;

[0174] The calling module 503 is used to call the preheating interface of the file cache server of the first distribution transfer station, obtain the file data of the current task from the file server, and cache it on the file cache server;

[0175] The first distribution module 504 is used to distribute the command of the distribution task to the first distribution transfer station, so that the first distribution transfer station distributes the command of the distribution task to the corresponding edge node, and the edge node obtains the file data of the current task from the file cache server corresponding to the first distribution transfer station.

[0176] Optionally, in another embodiment, based on the above embodiment, the device further includes:

[0177] A second transfer determination module is used to determine a second distribution transfer station in the transfer cluster to which the edge node of the current task direct push belongs when the mode determination module determines that the task delivery mode is a small-scale direct push mode;

[0178] The second distribution module is used to push the command of the distribution task and the file data directly to the second distribution transfer station, so that the second distribution transfer station distributes the command of the distribution task and the file data directly to the corresponding edge node, and the edge node directly stores the received file data of the current task in the specified directory.

[0179] Optionally, in another embodiment, based on the above embodiment, the first transit determination module is specifically used to group the distribution transit stations of the transit cluster where the edge nodes are located according to the attribute information of the edge nodes distributed by the current task when the mode determination module determines that the delivery mode of the current task is a large-scale delivery mode, and determine the distribution transit station with a healthy heartbeat in each group as the first distribution transit station in the transit cluster to which the edge nodes of the corresponding group belong, wherein the attribute information includes: the IP address of the edge node and / or the location information of the edge node;

[0180] The second transit determination module is specifically used to group the distribution transit stations of the transit cluster where the edge nodes are located according to the attribute information of the edge nodes distributed for the current task when the mode determination module determines that the delivery mode of the task is a small-scale direct push mode, and determine the distribution transit station with a healthy heartbeat in each group as the second distribution transit station in the transit cluster to which the edge nodes of the corresponding group belong, wherein the attribute information includes: the IP address of the edge node and / or the location information of the edge node.

[0181] See also Figure 6 , a block diagram of a file distribution system is provided for an embodiment of the present invention, the system comprising: a central service node 601, a file server 602, a scheduling center cluster 603 (including multiple scheduling centers), a file cache service cluster 604 (including multiple file cache servers) and a distribution transfer cluster 605 (including multiple distribution transfer stations), wherein,

[0182] The central service node 601 is used to determine the delivery mode of the current task when receiving the command to distribute the task, and when the delivery mode of the current task is the large-scale delivery mode, mark the delivery mode of the current task as the large-scale delivery mode, store the file data of the current task on the file server 602, and distribute the marked delivery task command to the first scheduling center in the scheduling center cluster; or when it is determined that the delivery mode of the current task is the small-scale direct push mode, mark the delivery mode of the current task as the small-scale direct push mode in the command; and deliver the marked delivery task command and the file data of the current task to the second scheduling center in the scheduling center cluster;

[0183] The file server 602 is used to store the file data sent by the central service node and provide an external download function;

[0184] Each dispatch center in the dispatch center cluster 603 is used to determine the dispatch mode of the current task when receiving the command of the distribution task sent by the central service node; when the dispatch mode of the current task is a large-scale dispatch mode, determine the first distribution transfer station in the distribution transfer cluster to which the edge node for distributing the current task belongs; call the preheating interface of the file cache server of the first distribution transfer station, obtain the file data of the current task from the file server, and cache it on the file cache server; distribute the command of the distribution task to the first distribution transfer station; or, when it is determined that the dispatch mode of the current task is a small-scale direct push mode, determine the second distribution transfer station in the distribution transfer cluster to which the edge node for direct pushing the current task belongs; push the command of the distribution task and the file data directly to the second distribution transfer station;

[0185] Each file cache server in the file cache service cluster 604 is used to provide a preheating interface for the distribution transfer station in the corresponding transfer cluster, temporarily cache the file data of the current task obtained from the file server, and provide download services;

[0186] Each distribution transfer station in the distribution transfer cluster 605 is used to distribute the command of the distribution task to the corresponding edge node when receiving the command of the distribution task sent by the corresponding scheduling center, so that the edge node obtains the file data of the current task from the file cache server of the corresponding first distribution transfer station and stores the file data of the current task in a designated directory; or, when receiving the command of the distribution task and the file data sent by the corresponding scheduling center, directly distribute the command of the distribution task and the file data to the corresponding edge node, and the edge node directly stores the received file data of the current task in the designated directory.

[0187] The embodiments of the present invention support the rapid access of new CDN machines and clusters to the file distribution system, and can retry tasks for restarting abnormal machines and migrate machines between different clusters, thereby greatly improving flexibility and scalability.

[0188] See also Figure 7 , is an application block diagram of a file distribution system provided by an embodiment of the present invention, such as Figure 7 As shown, it includes: a central cluster, taking the South China cluster and the North China cluster connected to the central service as an example, and taking the Fujian cluster connected to the South China cluster as an example, and taking the Beijing cluster connected to the North China cluster as an example.

[0189] Among them, the central cluster includes the central service (i.e., the central service node), the South China cluster includes the dispatch center, and the North China cluster also includes the dispatch center. In actual applications, in order to distinguish different dispatch centers, they can be set as the first dispatch center and the second dispatch center, etc., the same below. Before issuing a command, each dispatch center will determine whether the heartbeat of the distribution transit station is healthy. If it is healthy, it will issue a distribution command. If it is not healthy, it will select a distribution transit station from an adjacent city as a transit service.

[0190] Among them, the file cache server in the Fujian cluster and the file cache server in the Beijing cluster can both pull (i.e. download) file data from the file server. The distribution transfer station in the Fujian cluster is connected to the dispatch center in the South China cluster, and the distribution transfer station in the Beijing cluster is connected to the dispatch center in the North China cluster. The file cache server and distribution transfer station in the Fujian cluster are deployed on the management machine of the corresponding computer room, and the file cache server and distribution transfer station in the Beijing cluster are also deployed on the management machine of the corresponding computer room. The file server is deployed on the file source station to store the file data of the Chinese service.

[0191] The distribution transfer station in the Fujian cluster takes the edge execution agent (agent) that manages two edge nodes as an example. The two edge execution agents can be the same or different. Each edge execution agent can download file data from the file cache server in the Fujian cluster. The edge execution agent can also be understood as a service deployed on the edge node. The distribution transfer station in the Beijing cluster also takes the edge execution agent that manages two edge nodes as an example. The two edge execution agents can be the same or different. Each edge execution agent can download file data from the file cache server in the Beijing cluster.

[0192] Of course, the file distribution system is not limited to this in practical applications. In another embodiment, the system includes six modules: central service (i.e., central service node), dispatch center (i.e., dispatch center node), distribution transfer station, file server, edge execution agent, and file cache server. The functions of each module are as follows:

[0193] Among them, the central service is used to provide interface services for users to call, including: interface verification, authentication; task creation, task distribution to corresponding regions, task query, task callback; file writing to persistent storage function, etc.

[0194] The dispatch center is responsible for the dispatching of files sent by machines, file preheating, collection and reporting of task completion status, etc.

[0195] The distribution transit station manages the edge execution agent on the edge nodes in the edge cluster and is deployed on the corresponding management machine. It is a transit station for issuing commands and is responsible for issuing commands to the edge execution agent (agent) of the edge machines (i.e. edge nodes) of the managed cluster.

[0196] The edge execution agent is responsible for downloading files and saving the downloaded files to a specified directory. The edge execution agent can also be understood as a service deployed on the edge node.

[0197] File cache server, used to cache temporary files and provide file download services, etc.

[0198] File server: responsible for persistent storage of issued files and providing third-party call services.

[0199] The file distribution system provided in this embodiment provides an API interface service to the outside world and provides it to third parties for calling. This function is mainly responsible for the central service. The API interface service also includes the following functions:

[0200] (1) Interface verification and authentication services: ensuring the legitimacy and security of user interface calls;

[0201] (2) Task link status tracking: task creation, task issuance, and task completion;

[0202] (3) Task result callback function: According to the callback interface set by the user, the result will be fed back to the user at the specified time or after the task is completed.

[0203] The file distribution system provided by the embodiment of the present invention, through the above functions, provides services to the outside while ensuring the security and reliability of the system.

[0204] The file distribution system provided by the embodiment of the present invention sinks the service of issuing file instructions to the management machine of the edge cluster (composed of two management machines, the primary and the backup), so as to reduce the pressure of the central service and shorten the network link. Since the management machine and the managed edge machine are in the same network segment, this feature is used to sink the command of the edge machine to the management machine, and the nearest management machine is responsible for the distribution of the file download command, which effectively saves network transmission time, reduces the pressure of the central service, and enhances the reliability and efficiency of the system.

[0205] It should be noted that the embodiment of the present invention provides a hierarchical, large-scale, network-wide file distribution system based on the existing CND edge node distribution situation, combined with the CDN business needs for large-scale network-wide file distribution and the distribution of configuration files in some newly listed clusters. It proposes a direct push method for small files and small-scale machines, and a method for preheating and nearby scheduling of large-scale network-wide files to achieve rapid file distribution.

[0206] Among them, based on the current management and distribution characteristics of CDN edge machines across the entire network, such as Fig. 7A and Figure 7B As shown, Fig. 7A is a schematic diagram of the distribution of machines in the entire network provided by an embodiment of the present invention. Figure 7B This is a schematic diagram of edge cluster machine distribution provided by an embodiment of the present invention.

[0207] like Fig. 7A and Figure 7B As shown in the figure, the distribution of CDN machines is to set up central clusters according to each region, and small clusters are set up within the region according to the province. All edge machines are managed in the small clusters, and there are two active and standby management machines in the cluster as the service center. Specifically,

[0208] like Fig. 7A As shown, it includes central clusters, regional central clusters, and provincial administrative division clusters. The central cluster includes the Guizhou central cluster as an example. The regional central clusters include: North China cluster, South China cluster, Northeast China cluster, East China cluster, Central China cluster, Southwest China cluster, and Northwest China cluster as examples. The provincial administrative division clusters include: North China cluster includes Beijing and Inner Mongolia as an example, South China cluster includes Fujian and Guangdong as an example, Northwest cluster includes Qinghai and Gansu as an example, etc. This implementation does not give examples one by one.

[0209] like Figure 7B As shown, this embodiment takes the Fujian cluster as an example, including a main management machine (i.e., the main management machine) and a backup management machine (i.e., the backup management machine), as well as multiple edge machines responsible for management (i.e., the edge execution agent of the boundary node). Normally, each cluster includes two management machines. For example, when the main management machine fails, the backup management machine is enabled, etc., but in actual applications, it is not limited to this.

[0210] The embodiment of the present invention combines the characteristics of CDN machine nodes to improve a hierarchical file distribution system, sinks the service of distribution instructions to the management machine of each cluster, realizes hierarchical command issuance, optimizes the transmission link of distribution commands, greatly reduces the pressure of central node distribution commands, realizes rapid large-scale distribution of files to designated nodes, reduces the service pressure of source stations, saves transmission time, and reduces network consumption. The modified file distribution system architecture can flexibly expand distribution machines and ensure system stability. The issuance speed of system files is improved, and the consumption of network bandwidth is reduced. The file distribution system provides a file preheating function for large files and large-scale machine issuance, and adopts a small file direct push function for small files and a small number of machines. In combination with the nearby scheduling strategy for issuing instructions, the instruction link and file download link are reduced, and the file issuance speed is significantly improved.

[0211] Optionally, an embodiment of the present invention further provides an electronic device, including:

[0212] processor;

[0213] a memory for storing instructions executable by the processor;

[0214] The processor is configured to execute the instructions to implement the file distribution method as described above.

[0215] Optionally, an embodiment of the present invention further provides a computer-readable storage medium, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the file distribution method as described above.

[0216] Optionally, an embodiment of the present invention further provides a computer program product, including a computer program or instructions, and when the computer program or instructions are executed by a processor of an electronic device, the file distribution method as described above is implemented.

[0217] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0218] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0219] Figure 8 800 is a block diagram of an electronic device 800 provided in an embodiment of the present invention. For example, the electronic device 800 may be a mobile terminal or a server, and the electronic device is a mobile terminal as an example for explanation in the embodiment of the present invention. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0220] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0221] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0222] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0223] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0224] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0225] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0226] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0227] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0228] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0229] In an embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the file distribution method shown above.

[0230] In an embodiment, a computer-readable storage medium is also provided, and when the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device 800 can execute the file distribution method shown above. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0231] In an embodiment, a computer program product is also provided, including a computer program or instructions. When the computer program or instructions are executed by the processor 820 of the electronic device 800, the electronic device 800 executes the above-mentioned file distribution method.

[0232] Fig. 9 is a block diagram of a device 900 for file distribution provided by an embodiment of the present invention. For example, the device 900 may be provided as a server. Fig. 9 , the apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions, such as an application, that can be executed by the processing component 922. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute the instructions to perform the above method.

[0233] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0234] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0235] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A file distribution method, characterized in that: The method is applied to a central service node, comprising: When receiving a command to distribute a task, determine the distribution mode of the current task; When the delivery mode of the current task is a large-scale delivery mode, marking the delivery mode of the current task as a large-scale delivery mode, and storing the file data of the current task on a file server; The marked distribution task command is distributed to the first dispatch center in the dispatch center cluster, so that when the first dispatch center receives the distribution task command, it distributes the distribution task command to the corresponding distribution transfer stations in each transfer cluster for scheduling according to the edge nodes corresponding to the distribution task.

2. The file distribution method according to claim 1, characterized in that: The method further comprises: When the delivery mode of the current task is a small-scale direct push mode, marking the delivery mode of the current task as a small-scale direct push mode; The marked task distribution command and the file data of the current task are sent to the second scheduling center in the scheduling center cluster, so that the second scheduling center directly distributes the file data of the current task to the corresponding edge node when receiving the task distribution command.

3. The file distribution method according to claim 2, characterized in that: When receiving a command to distribute a task, determining a distribution mode of the current task includes: When receiving the command to distribute tasks, obtain the file size of the current task, the number of edge nodes of the current task, and the number of tasks per unit time; Determine the size of the current traffic according to the file size of the current task, the number of edge nodes of the current task, and the number of tasks per unit time; According to the size of the current traffic, it is determined that the delivery mode of the current task is a large-scale delivery mode or a small-scale direct push mode.

4. A file distribution method, characterized in that: The method is applied to each dispatch center in the dispatch center cluster, and includes: When receiving the task distribution command sent by the central service node, determine the distribution mode of the current task; When the delivery mode of the current task is a large-scale delivery mode, determining a first delivery transfer station in a transfer cluster to which an edge node to which the current task is delivered belongs; Calling the preheating interface of the file cache server of the first distribution transfer station, obtaining the file data of the current task from the file server, and caching it on the file cache server; The command of the distribution task is distributed to the first distribution transfer station, so that the first distribution transfer station distributes the command of the distribution task to the corresponding edge node, and the edge node obtains the file data of the current task from the file cache server corresponding to the first distribution transfer station.

5. The file distribution method according to claim 4, characterized in that: The method further comprises: When the delivery mode of the current task is a small-scale direct push mode, determine a second distribution transfer station in the transfer cluster to which the edge node of the direct push of the current task belongs; The command of the distribution task and the file data are directly pushed to the second distribution transfer station, so that the second distribution transfer station distributes the command of the distribution task and the file data directly to the corresponding edge node, and the edge node directly stores the received file data of the current task in the specified directory.

6. A file distribution device, characterized in that: The device is applied to a central service node, and includes: A determination module, used to determine the distribution mode of the current task when receiving a command to distribute the task; A first marking module, used for marking the delivery mode of the current task as the large-scale delivery mode when the delivery mode of the current task is the large-scale delivery mode, and storing the file data of the current task on the file server; The first distribution module is used to distribute the marked distribution task command to the first dispatching center in the dispatching center cluster, so that when the first dispatching center receives the distribution task command, it distributes the distribution task command to the distribution transfer stations in each transfer cluster for scheduling according to the edge nodes corresponding to the distribution task.

7. A file distribution device, characterized in that: The device is applied to each dispatch center in the dispatch center cluster, and includes: A mode determination module is used to determine the delivery mode of the current task when receiving a command to distribute the task sent by the central service node; A first transfer determination module, configured to determine, when the mode determination module determines that the delivery mode of the current task is a large-scale delivery mode, a first distribution transfer station in a transfer cluster to which an edge node for the current task distribution belongs; A calling module, used for calling a preheating interface of a file cache server of the first distribution transfer station, obtaining file data of a current task from the file server, and caching the file data on the file cache server; The first distribution module is used to distribute the command of the distribution task to the first distribution transfer station, so that the first distribution transfer station distributes the command of the distribution task to the corresponding edge node, and the edge node obtains the file data of the current task from the file cache server corresponding to the first distribution transfer station.

8. A file distribution system, characterized in that: The system includes: a central service node, a file server, a scheduling center cluster, a distribution transfer cluster and a file cache service cluster, wherein: The central service node is used to determine the delivery mode of the current task when receiving the command to distribute the task, and when the delivery mode of the current task is the large-scale delivery mode, mark the delivery mode of the current task as the large-scale delivery mode, store the file data of the current task on the file server, and distribute the marked delivery task command to the first scheduling center in the scheduling center cluster; or when it is determined that the delivery mode of the current task is the small-scale direct push mode, mark the delivery mode of the current task as the small-scale direct push mode in the command; and deliver the marked delivery task command and the file data of the current task to the second scheduling center in the scheduling center cluster; The file server is used to store the file data sent by the central service node and provide an external download function; Each dispatch center in the dispatch center cluster is used to determine the dispatch mode of the current task when receiving the command of distributing the task sent by the central service node; when the dispatch mode of the current task is a large-scale dispatch mode, determine the first distribution transfer station in the distribution transfer cluster to which the edge node for distributing the current task belongs; call the preheating interface of the file cache server of the first distribution transfer station to obtain the file data of the current task from the file server; distribute the command of distributing the task to the first distribution transfer station; or, when it is determined that the dispatch mode of the current task is a small-scale direct push mode, determine the second distribution transfer station in the distribution transfer cluster to which the edge node for direct pushing the current task belongs; and push the command of the distribution task and the file data directly to the second distribution transfer station; Each file cache server in the file cache service cluster is used to provide a preheating interface for the distribution transfer station in the corresponding transfer cluster, temporarily cache the file data of the current task obtained from the file server, and provide download services; Each distribution transfer station in the distribution transfer cluster is used to distribute the command of the distribution task to the corresponding edge node when receiving the command of the distribution task sent by the corresponding scheduling center, so that the edge node obtains the file data of the current task from the file cache server of the corresponding first distribution transfer station; or, when receiving the command of the distribution task and the file data sent by the corresponding scheduling center, directly distribute the command of the distribution task and the file data to the corresponding edge node, and the edge node directly stores the received file data of the current task in the designated directory.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the file distribution method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the file distribution method according to any one of claims 1 to 5.

Images