A configuration file repairing method and device for a CDN system

Abstract

The application discloses a configuration file repairing method and device for a CDN system, and the file repairing device comprises a resource management system, a global configuration management center, a detection scheduling module, a regional configuration agent, an edge agent and a task system. Compared with the existing configuration file repairing method, the method of the application completes the collection, comparison and repair of the edge machine configuration file information through the regional configuration agent, and uses a scheduling strategy which comprehensively considers the health value of the regional configuration agent and the optimal route to distribute the edge node detection task to the optimal regional configuration agent, so that the repair efficiency is improved. Meanwhile, the rapid expansion and the elimination of the faulty regional configuration agent can be realized, and the robustness of the system is improved. Furthermore, the configuration file rapid repair in the environment of different networks and cross-networks can be realized by deploying the regional configuration agent to the BGP machine room and the multiple-instance deployment of each region.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of configuration file repair devices of CDN systems, and particularly relates to a configuration file repair method and device for CDN systems. BACKGROUND

[0002] A CDN (Content Delivery Network) is a distributed network architecture aiming to improve user experience and network performance by distributing content to nodes closer to users, and a CDN service provider will install corresponding software on machines according to node resource characteristics, business requirements, customer customization and other factors, and the normal operation of different software relies on different configuration files, and the accuracy of software configuration plays a crucial role in the reliability of CDN services. With the continuous growth of Internet content data, the scale and complexity of CDN systems are also increasing. In actual production applications, the software configuration of machines in the whole network is usually planned and issued by the CDN support center. Due to network fluctuations, machine failures, external component abnormalities and other factors, it is often difficult to ensure that all machines in the network receive correct configurations when software configuration is issued. At the same time, the generation of some configuration files needs to rely on real-time data such as machine service status, disk, network card information, etc. Once these data change, the corresponding configuration files need to be regenerated and issued.

[0003] The prior art mainly adopts a method of issuing file lists and hash values to machines at the same time, and machines compare them. If they are inconsistent, they are reported to the center server. When the number of configuration files to be repaired is large and a large number of machines need to be repaired, the file list and hash value need to be issued to the machine frequently to ensure the accuracy and timeliness of the report. This process will cause waste of bandwidth resources and will put a lot of pressure on the file distribution system, which is prone to resource bottlenecks. At the same time, the existing method does not address the poor communication quality caused by different networks and cross-regional communication in large-scale cluster environments. SUMMARY

[0004] This section is intended to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification of the present application to avoid obscuring the purpose of the abstract and title of the specification. Such simplifications or omissions cannot be used to limit the scope of the present application.

[0005] In view of the above problems of the prior art, the present application is proposed.

[0006] Therefore, the application aims to provide a configuration file repair method and device for a CDN system, which is suitable for solving the problem that the existing method cannot solve the poor communication quality caused by different networks and cross-regional areas in a large-scale cluster environment.

[0007] To solve the above technical problems, the application provides the following technical solutions: a configuration file repair device for a CDN system, which comprises:

[0008] a resource management system, a global configuration management center, a detection scheduling module, a regional configuration agent, an edge agent and a task system.

[0009] The resource management system mainly maintains server information, logically plans on the basis of the server information, plans multiple servers into a host group according to the server characteristics, and provides services externally through load balancing.

[0010] The global configuration management center maintains the software configuration commonly used in the CDN system, renders software configuration files on the planned host group according to machine information, planning information, customer information and a software configuration template, and distributes the configuration files to edge nodes.

[0011] As a preferred scheme of the configuration file repair device for the CDN system, the detection scheduling module comprises a suitable regional configuration agent for collecting edge node data, the regional configuration agent comprises a regional configuration agent deployed according to the granularity of a province or a large area, judges whether repair is needed by comparing the edge node configuration file information with the platform configuration file information according to the platform configuration file list and the platform summary MD5 distributed by the global configuration management center, and calls a center interface to distribute correct configuration files to the edge nodes if repair is needed.

[0012] As a preferred scheme of the configuration file repair device for the CDN system, the edge agent comprises a configuration agent deployed on an edge node, is responsible for the deployment of configuration files, collects configuration file information on the machine according to the platform configuration file list distributed by the global configuration management center, and provides a corresponding query interface.

[0013] As a preferred scheme of the configuration file repairing device for the CDN system, the task system comprises a configuration center, and the configuration center can distribute configuration files to specified regional configuration agents or edge nodes via the task system; the global configuration management center synchronizes basic data from a resource management system in a time manner, including but not limited to machine information, planning information, host groups, LVS groups, resource labels, customer information, customer IDs and acceleration domain names; different software configuration files are generated according to a software configuration template; the configuration files are distributed to edge machines; and a platform configuration file list is generated by the configuration center according to historical distribution records, and the platform configuration file list comprises paths of the configuration files, each configuration file is a line of records, all the records are sorted according to a dictionary order, and the platform configuration file list is written into the platform configuration file list; and the machine information comprises an IP, memory capacity, hard disk capacity and rated service capability.

[0014] As a preferred scheme of the configuration file repairing device for the CDN system, the configuration center renders a software configuration template in a time manner; when machine information, planning information and customer information are changed, the software configuration files are re-distributed, and the platform configuration file list and platform summary MD5 are updated and distributed to the regional configuration agents; and when the edge nodes add or delete configuration files, the platform configuration file list is updated and distributed to the corresponding edge nodes.

[0015] As a preferred scheme of the configuration file repairing device for the CDN system, the platform configuration file list is generated by the configuration center according to historical distribution records, and the platform configuration file list comprises paths of the configuration files and MD5 of contents of the configuration files recorded by the configuration center when the configuration files are distributed; a hash value of the configuration file is obtained by performing a hash operation on the configuration file; each configuration file is a line of records; the MD5 of the contents of each configuration file is appended to the platform configuration file list on the basis of the platform configuration file list; and the platform configuration file list MD5 is obtained by performing a hash operation on the contents of the platform configuration file list.

[0016] As a preferred scheme of the configuration file repairing device for the CDN system, the edge nodes receive information reported by the regional agents, and generate corresponding repairing tasks and distribute the repairing tasks to edge machine probe scheduling modules; the configuration center in the probe scheduling module collects machine indexes of the regional configuration agents in a time manner, and then calculates an overall health value by using the machine indexes; and the probe scheduling module comprises a health value and an optimal link.

[0017] As a preferred scheme of the configuration file repair method for CDN system, wherein: the health value calculation method is adapted to any one of the repair devices, and the health value calculation method comprises the following steps:

[0018] S1. Smooth and standardize the collected indicators;

[0019] S2. Single indicator: set maximum and minimum threshold values, if the single indicator exceeds the maximum threshold value, then Q=1; if the single indicator is less than the minimum threshold value, then Q=0; if the single indicator is between the threshold values, then a normalization algorithm is used for normalization;

[0020] S3. Health value algorithm:

[0021] s3.1. When any of the four indicators is greater than the maximum threshold value, Q=1

[0022] s3.2. When all four indicators are less than the minimum threshold value, Q=0

[0023] s3.3. Other cases: Q=(CPU+MEM+WID+RT80) / 4;

[0024] S4. Health value weight H=1-Q weight ∈[0,1], when the health value is small, the weight allocated is larger.

[0025] As a preferred scheme of the configuration file repair method for CDN system, wherein: the optimal link allocation method comprises the following steps:

[0026] S1. When the regional proxy center calls the edge node to collect data, record the response time of this request, and when the interface call fails, the response time is recorded as NaN;

[0027] S2. When initialized, set a default experience value s for the response time of the edge node to each regional configuration proxy. With the center continuously assigning edge nodes to regional configuration proxies for detection, the real response time of the edge node to each regional configuration proxy will be obtained. Record the response time of each edge node to the regional configuration proxy detected in the same period of time, and calculate the average response time of the low(25%) response time. Normalize all average response times to obtain R value;

[0028] S3. The response time weight W of the edge node to each regional configuration proxy is 1-R value, and the weight ∈[0,1], when the average response time is small, it means that the communication link quality of the regional proxy to the edge node is better, and the weight allocated is larger;

[0029] S4. When the edge node to all regional configuration proxies is NaN, then the allocation weights of all regional proxies are consistent.

[0030] As a preferred scheme of the configuration file repair method for the CDN system, wherein: the configuration file repair method for the CDN system can eliminate the regional proxy with a health value of 1 and a mean response time of NaN for each edge node, and the final scheduling decision of the health value weight H and the response time weight W is realized by the following formula, and the specific formula of the configuration file repair method for the CDN system is as follows:

[0031]

[0032] Wherein, l is the number of regional configuration proxy nodes that are not eliminated.

[0033] The beneficial effects of the present application are:

[0034] Compared with the existing configuration file repair method: the method of the present application completes the collection, comparison and repair of the edge machine configuration file information through the regional configuration proxy, and uses a scheduling strategy that comprehensively considers the health value of the regional configuration proxy and the optimal line to distribute the edge node detection task to the optimal regional configuration proxy, thereby improving the repair efficiency; at the same time, the method can realize rapid expansion and elimination of faulty regional configuration proxies, thereby improving the robustness of the system; and the method can realize rapid repair of the configuration file in the environment of different networks and cross-networks by deploying the regional configuration proxy to the BGP machine room and deploying multiple instances in each region.

[0035] By comparing and summarizing the inconsistent MD5 and comparing the configuration file list again, the configuration file list is only distributed to the edge node when the configuration file list changes, thereby saving the consumption of bandwidth resources. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0037] Fig. 1 A main structure diagram of a configuration file repair device for the CDN system is provided for the present application.

[0038] Fig. 2 A resource management flowchart of a configuration file repair method for the CDN system is provided for the present application. DETAILED DESCRIPTION

[0039] In order to make the above-mentioned objectives, features and advantages of the present application more obvious and comprehensible, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0040] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without the specific details and other implementations can be employed. Therefore, the present application is not limited to the details described herein and can be practiced with variations that are within the scope and spirit of the present application.

[0041] Secondly, the "one embodiment" or "embodiment" referred to herein means a specific feature or characteristic that can be included in at least one implementation of the present application. "In one embodiment" appearing in different places in the specification does not mean the same embodiment, nor does it mean an embodiment that is mutually exclusive of other embodiments, alone or selectively.

[0042] Thirdly, the present application is described in detail in conjunction with the schematic diagram. In the detailed description of the embodiments of the present application, the cross-sectional view of the device structure is locally enlarged without the general proportion for the convenience of description, and the schematic diagram is only an example, which should not limit the scope of protection of the present application herein. In addition, the three-dimensional spatial dimensions of length, width and depth should be included in actual manufacture.

[0043] Reference Figs. 1-2 For one embodiment of the present application, a configuration file repair method and device for a CDN system are provided, and the file repair device comprises:

[0044] a resource management system, a global configuration management center, a detection scheduling module, a regional configuration agent, an edge agent and a task system;

[0045] The resource management system mainly maintains server information, logically plans on the basis, plans multiple servers as a host group according to server characteristics, and provides services externally through load balancing.

[0046] The global configuration management center maintains the software configuration commonly used in the CDN system, renders the software configuration file on the planned host group according to machine information, planning information, customer information and software configuration templates, and issues the configuration file to the edge node.

[0047] Further, the probe scheduling module includes selecting a suitable area configuration agent to complete the collection of edge node data, the area configuration agent includes an area configuration agent deployed according to a province or a large area granularity, according to a platform configuration file list and a platform summary MD5 issued by a global configuration management center, the area agent probes edge node configuration file information in a timely manner, and compares with the platform configuration file information to judge whether repair is needed, if repair is needed, a center interface is called to issue a correct configuration file to the edge node.

[0048] Further, the edge agent includes a configuration agent deployed on the edge node, responsible for the deployment of the configuration file, according to the platform configuration file list issued by the global configuration management center, the configuration file information on the machine is collected, and a corresponding query interface is provided.

[0049] Further, the task system includes a configuration center which can issue a configuration file to a specified area configuration agent or edge node via the task system, the global configuration management center synchronizes basic data from a resource management system in a timely manner, including but not limited to: machine information, planning information, host group, LVS group, resource label, customer information, customer ID and acceleration domain name, different software configuration files are generated according to a software configuration template, the configuration file is issued to the edge machine, and the platform configuration file list: the configuration center generates the expected platform configuration file list of each machine according to the historical issue record, the content is the path of the configuration file, each configuration file is a line record, all records are sorted according to the dictionary order, and written into the platform configuration file list, the machine information includes IP, memory capacity, hard disk capacity and rated service capacity.

[0050] Further, the configuration center renders the software configuration template in a timely manner, when the machine information, planning information and customer information change, the software configuration file is reissued, and the platform configuration file list and the platform summary MD5 are updated and issued to the area configuration agent, when the edge node adds or deletes the configuration file, the platform configuration file list is updated and issued to the corresponding edge node.

[0051] Further, the platform configuration file list: the configuration center generates the expected platform configuration file list of each machine according to the historical delivery record, the content is the path of the configuration file and the MD5 of the configuration file content recorded by the configuration center when delivering the configuration, the hash value obtained by performing hash operation on the configuration file, each configuration file is a line record, on the basis of the platform configuration file list, the MD5 of each configuration file content is appended and written into the platform configuration file list, the platform configuration file list MD5: the MD5 obtained by performing hash operation on the content of the platform configuration file list, the configuration center delivers the detection task, the platform configuration file list and the list MD5 to the specified regional configuration agent according to the decision result of the detection scheduling module.

[0052] Further, the edge node receives the information reported by the regional agent and generates a corresponding repair task, which is delivered to the edge machine detection scheduling module, the configuration center in the detection scheduling module collects the machine indicators of each regional configuration agent regularly, and then calculates the overall health value by using the machine indicators, the detection scheduling module contains the health value and the optimal link.

[0053] Further, the regional configuration agent comprises the following steps:

[0054] [1] Receive the detection task, the platform configuration file list, the platform summary MD5 delivered by the configuration center and cache them, the cache content is mainly used to record the platform configuration file list of each machine, the platform summary MD5, the collection summary MD5 and the number of exceptions, the cache content is as follows:

[0055] a) 1.1.1.1: [platform configuration file list, platform summary MD5, collection summary MD5, number of exceptions]

[0056] b) 1.1.1.2: [platform configuration file list, platform summary MD5, collection summary MD5, number of exceptions]

[0057] [2] The regional agent regularly pulls the collection summary MD5 from the edge machine and compares it with the platform summary MD5, if they are inconsistent, the number of exceptions is increased by 1, if the collection summary MD5 information is inconsistent with the platform summary MD5 in a plurality of periods (such as three periods), the edge collection configuration file list is obtained, the difference between the platform configuration file list and the collection configuration file list is compared, the inconsistent configuration file MD5 is found, and the interface of the configuration center is called to complete the delivery of the correct configuration file.

[0058] Further, the health value calculation method is suitable for any one of the above repair devices, and the health value calculation method comprises the following steps:

[0059] S1. Smooth and standardize the collected indicators;

[0060] S2. Single index: set maximum and minimum threshold values, if exceeding the maximum threshold value, the single index Q = 1; if less than the minimum threshold value, Q = 0; if between the threshold values, the normalization algorithm is used for normalization;

[0061] S3. Health value algorithm:

[0062] s3.1. When any of the four indexes is greater than the maximum threshold value, Q = 1

[0063] s3.2. When all four indexes are less than the minimum threshold value, Q = 0

[0064] s3.3. Other cases: Q = (CPU + MEM + WID + RT80) / 4;

[0065] S4. Health value weight H = 1 - Q weight ∈ [0, 1], when the health value is small, the weight allocated is larger;

[0066] Among them, the configuration center regularly collects the main machine indicators of each regional configuration agent as follows:

[0067]

[0068] Further, the optimal link allocation method comprises the following steps:

[0069] S1. When the regional agent center calls the edge node to collect data, record the response time of this request, and when the interface call fails, the response time is recorded as NaN;

[0070] S2. When initialized, set a default experience value s for the response time of the edge node to each regional configuration agent. With the center continuously assigning edge nodes to regional configuration agents for detection, the real response time of the edge node to each regional configuration agent will be obtained. Record the response time of each edge node to the regional configuration agent detected in the same period of time recently, and calculate the average response time of the low (25%) response time. Normalize all average response times to obtain R value;

[0071] S3. The response time weight W of the edge node to each regional configuration agent = 1 - R value, weight ∈ [0, 1], when the average response time is small, it means that the communication link quality of the regional agent to the edge node is better, and the weight allocated is larger;

[0072] S4. When the edge node to all regional configuration agents is NaN, the weight of all regional agents is consistent.

[0073] Further, the method for repairing the configuration file of the CDN system can eliminate the regional agent with a health value of 1 and an average response time of NaN for each edge node, and the final scheduling decision of the health value weight H and the response time weight W is realized by the following formula, and the specific formula of the method for repairing the configuration file of the CDN system is as follows:

[0074]

[0075] In the formula, l is the number of regional configuration agent nodes that are not eliminated.

[0076] The specific working principle is as follows:

[0077] Synchronize the basic data from the resource management system in time, including but not limited to: machine information (including IP, memory capacity, hard disk capacity, rated service capacity, etc.), planning information (host group, LVS group, resource label, etc.), customer information (customer ID, acceleration domain name, etc.), render different software configuration files according to the software configuration template, and download the configuration files to the edge machine.

[0078] Platform configuration file list: the configuration center generates the platform configuration file list expected by each machine according to the historical download record, the content is the path of the configuration file, each configuration file is a line record, and all records are sorted according to the dictionary order and written into the platform configuration file list. For example:

[0079] # Configuration file

[0080] / dync_acc_ccs / conf / ccs.yaml

[0081] / etc / cluster_conf / accid.conf

[0082] / etc / cluster_conf / area.conf

[0083] Platform configuration file list: the configuration center generates the platform configuration file list expected by each machine according to the historical download record, the content is the path of the configuration file, each configuration file is a line record, and all records are sorted according to the dictionary order and written into the platform configuration file list. For example:

[0084] # Configuration file platform configuration file MD5

[0085] / dync_acc_ccs / conf / ccs.yaml d5f3d625fcc7a154e4ab24fa2184b8f1

[0086] / etc / cluster_conf / accid.conf 3b67dff001c8fcbfdb1c923c43fe6a47

[0087] / etc / cluster_conf / area.conf 74940f465e3796eae7d38e9e1f507113

[0088] Platform configuration file list MD5: MD5 obtained by hashing the content of the platform configuration file list;

[0089] The configuration center sends the detection task, the platform configuration file list and the list MD5 to the specified area configuration agent according to the decision result of the detection scheduling module.

[0090] The configuration center renders the software configuration template regularly, reissues the software configuration file when the machine information, planning information and customer information change, updates the platform configuration file list and the platform summary MD5, and issues them to the area configuration agent. When an edge node adds or deletes a configuration file, the platform configuration file list is updated and issued to the corresponding edge node. The main process includes:

[0091] The global configuration management center generates the platform configuration file list and the platform summary MD5 according to the configuration issuance history, distributes them to the area configuration agent, and distributes the platform configuration file list to the edge node. The edge node generates the collection configuration file list and the collection summary MD5 according to the platform configuration file list, and provides related query interfaces for the area configuration agent to call. The detection scheduling module distributes the edge node detection task to the optimal area configuration agent according to the area configuration agent node health value condition and the historical detection situation. The area configuration agent obtains the collection summary MD5 of the edge node and compares it with the platform summary MD5. If they are inconsistent, the collection configuration file list and the platform configuration file list are compared. For inconsistent configuration files, the global configuration management center interface is called to complete the configuration file repair.

[0092] Specifically, the purposes of the present application include:

[0093] Improve the reliability of CDN node service: through the method of the application, the software configuration file of the whole network managed machine can be quickly and efficiently repaired, when a large number of new edge nodes are put on the shelf, new regional configuration agent can be deployed to realize rapid expansion, and the regional configuration agent can be deployed to BGP machine room, multiple instance deployment in each area, etc. The method can realize the rapid repair of configuration file in the environment of different network and cross network.

[0094] Reduce the maintenance cost of CDN system manpower: through the method of the application, the failed regional configuration agent can be adaptively removed and the optimal line selection can be realized, without investing a large amount of manpower cost for maintenance.

[0095] Save network resource overhead: by comparing and summarizing the inconsistent MD5, and then comparing the configuration file list, the configuration file list needs to be distributed to the edge node only when the configuration file list is changed.

[0096] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limited. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A configuration file repair apparatus for a CDN system, characterized by, The file repair device comprises: A resource management system, a global configuration management center, a detection scheduling module, a regional configuration agent, an edge agent and a task system; The resource management system: maintains server information, logically plans on the basis, plans multiple servers as a host group according to server characteristics, and provides services externally through load balancing; The global configuration management center: maintains software configuration commonly used in the CDN system, renders software configuration files on the basis of machine information, planning information, customer information and software configuration templates, and distributes the configuration files to edge nodes; The detection scheduling module comprises a selected regional configuration agent for collecting data of the edge nodes, the regional configuration agent comprises a regional configuration agent deployed according to the granularity of provinces or large areas, judges whether repair is needed by comparing the platform configuration file information distributed by the global configuration management center with the edge node configuration file information, and if repair is needed, calls the center interface and distributes correct configuration files to the edge nodes; The edge agent comprises a configuration agent deployed on the edge nodes, is responsible for the deployment of configuration files, collects configuration file information on the machine according to the platform configuration file list distributed by the global configuration management center, and provides a corresponding query interface; The task system comprises a configuration center, can distribute configuration files to specified regional configuration agents or edge nodes through the task system, synchronizes basic data from the resource management system at regular time, including but not limited to: machine information, planning information, host group, LVS group, resource label, customer information, customer ID and acceleration domain name, renders different software configuration files according to the software configuration template, distributes the configuration files to edge machines, and the platform configuration file list: the configuration center generates the expected platform configuration file list of each machine according to historical distribution records, the content is the path of the configuration file, each configuration file is a line of record, all records are sorted according to the dictionary order, and written into the platform configuration file list, the machine information includes IP, memory capacity, hard disk capacity and rated service capacity; The configuration center renders the software configuration template at regular time, re-distributes the software configuration file when the machine information, planning information and customer information change, updates the platform configuration file list and the platform summary MD5, and distributes them to the regional configuration agent, and when the edge node adds or deletes a configuration file, updates the platform configuration file list and distributes it to the corresponding edge node; The platform configuration file list: the configuration center generates the expected platform configuration file list of each machine according to the historical delivery record, the content is the path of the configuration file and the MD5 of the configuration file content recorded when the configuration center delivers the configuration, the hash value obtained by performing hash operation on the configuration file, each configuration file is a line record, on the basis of the platform configuration file list, the MD5 of each configuration file content is appended and written into the platform configuration file list, the platform configuration file list MD5: the MD5 obtained by performing hash operation on the content of the platform configuration file list, the configuration center delivers the detection task, the platform configuration file list and the list MD5 to the specified regional configuration agent according to the decision result of the detection scheduling module; The edge node receives the information reported by the regional agent, generates the corresponding repair task, and delivers it to the edge machine detection scheduling module, the configuration center in the detection scheduling module collects the machine indexes of each regional configuration agent regularly, and then calculates the overall health value by using the machine indexes, the detection scheduling module contains the health value and the optimal link.

Images