Unified control management implementation method and system based on multi-data center calculation

Abstract

The invention provides a unified control management method based on multi-data center calculation. A central controller sends task request information sent by a user to a registration center through a registration center client; the registration center matches the task request information with the distributed coordination and configuration management tool according to the type of the data center and returns a matching result to the central controller through the registration center client; the center controller splices the service information of the data center of the successfully matched task request information in combination with the data center type, packages the service information into a calculation request and sends the calculation request to a calculation service; and the calculation service matches the calculation request to a specific executor of the unified calculation entry according to the executor type to complete a calculation task, and the specific executor returns a result of the calculation task to the user through the calculation service and the central controller. According to the method, the data resources of the multiple clusters are managed and calculated in a unified mode through the unified management end, the data of the distributed clusters can be managed and utilized more efficiently, and the processing efficiency and quality are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data centers, and in particular to a unified control management method and system based on multi-data center computing. Background Art

[0002] Currently, many enterprises or organizations use multiple distributed clusters to process massive data in business environments. The data between different clusters are isolated and scattered in different places. It is difficult for enterprises or organizations to uniformly manage and calculate the scattered data resources, and processing data from different clusters requires additional work. This also adds complexity to data integration, data conversion, data cleaning, and data calculation tasks, making it challenging to ensure data consistency and synchronization.

[0003] When data is replicated and synchronized between different clusters, delays or inconsistencies may occur, which may affect the accuracy and real-time calculation of data. Data interaction between multiple distributed clusters through the network will also cause network communication delays and bottlenecks, especially when communicating across regions or over long distances. High latency problems will also occur, affecting data access and calculation.

[0004] In addition, maintaining multiple distributed clusters requires additional hardware, software, and human resources. Each cluster needs to be managed and maintained independently, which increases resource management and maintenance costs and resource consumption. At the same time, multiple distributed clusters are more prone to more failure points. When one or more clusters fail or become unavailable, data access and computing in the cluster will be interrupted. Therefore, how to ensure the high availability and fault tolerance of the cluster becomes more important and complex. Summary of the invention

[0005] The present invention proposes the following technical solutions to address one or more technical deficiencies in the above-mentioned prior art.

[0006] Based on the first aspect of the present invention, a unified control management method based on multi-data center computing is proposed, comprising:

[0007] S1: The central controller CC sends the task request information TKINFO sent by the user to the registration center RC through the registration center client RCC;

[0008] S2: The registration center RC matches the task request information TKINFO with the distributed coordination and configuration management tool according to the type of the data center in the task request information TKINFO, and returns the matching result to the central controller CC through the registration center client RCC;

[0009] Specifically, the registration center RC obtains the IP and port of the target service from the distributed coordination and configuration management tool according to the type of the data center of the task request information TKINFO and sends the task request information TKINFO to the target service for matching according to the IP and port of the target service;

[0010] S3: The central controller CC combines the service information corresponding to the data center of the successfully matched task request information TKINFO with the type of the data center, and encapsulates it into a computing request and sends it to the computing service;

[0011] S4: The computing service matches the computing request to the specific executor of the unified computing portal UCE according to the executor type to complete the computing task. The specific executor returns the result of the computing task to the user through the computing service and the central controller CC in sequence.

[0012] The present invention uses a unified management terminal to uniformly manage and calculate the data resources of multiple clusters, which can minimize the correlation and dependence between clusters and ensure that the expansion of each cluster will not have a negative impact on the performance and stability of the entire system. Users can more efficiently manage and utilize the data of distributed clusters and improve processing efficiency and quality, ensure data consistency and synchronization, reduce delays in replication and synchronization between data in different clusters, ensure high availability and fault tolerance of distributed clusters, and reduce cluster maintenance costs and resource consumption.

[0013] Furthermore, the computing service is started simultaneously with starting the central controller CC, and the computing service stores the address and port of the data center in the distributed coordination and configuration management tool specified by the registration center RC.

[0014] The central controller has good data collection, processing and scheduling functions, and can achieve rapid response and linkage of multiple devices in a short time, achieving the goals of energy saving, high efficiency and intelligent control.

[0015] Furthermore, the task request information TKINFO includes a task request of a data center type, a task engine type and specific task information.

[0016] Furthermore, the distributed collaboration and configuration management tool is initialized before matching.

[0017] Furthermore, in step S2, if the task request information TKINFO successfully matches the distributed coordination and configuration management tool, the service information corresponding to the data center of the task request information TKINFO is sent to the central controller CC, otherwise, if the match fails, an exception is returned to the central controller CC and the task is interrupted.

[0018] By calling a unified control and management system for multi-data computing, the problem of system complexity is solved to a certain extent, ensuring the coordination of data synchronization, consistency and fault recovery between clusters, minimizing the problem of the normal operation of other clusters affected by the failure of one cluster, and ensuring the reliability and fault tolerance of the system.

[0019] Furthermore, the specific executors include session mode executors, independent mode executors and interactive executors.

[0020] Furthermore, the session mode executor is used for the calculation of online tasks, the standalone mode executor is used for the calculation of offline tasks, and the interactive executor is used for the calculation of ad hoc tasks.

[0021] Matching computing requests to a unified computing entry and assigning them to specific executors for computing can speed up response, reduce delays in copying and synchronizing data between different clusters, avoid data conflicts and inconsistencies, and improve data processing efficiency and quality.

[0022] Based on the second aspect of the present invention, a unified control and management system based on multi-data center computing is also proposed, including:

[0023] Sending module: The central controller CC sends the task request information TKINFO sent by the user to the registration center RC through the registration center client RCC;

[0024] Matching module: the registration center RC matches the task request information TKINFO with the distributed coordination and configuration management tool according to the type of data center in the task request information TKINFO, and returns the matching result to the central controller CC through the registration center client RCC;

[0025] Specifically, the registration center RC obtains the IP and port of the target service from the distributed coordination and configuration management tool according to the type of the data center of the task request information TKINFO and sends the task request information TKINFO to the target service for matching according to the IP and port of the target service;

[0026] Packaging module: The central controller CC combines the type of the data center, splices the service information corresponding to the data center of the successfully matched task request information TKINFO, and packs it into a computing request and sends it to the computing service;

[0027] Computing module: The computing service matches the computing request to the specific executor of the unified computing entry UCE according to the executor type to complete the computing task. The specific executor returns the result of the computing task to the user through the computing service and the central controller CC in sequence.

[0028] Based on the third aspect of the present invention, a computer storable medium is also proposed, on which one or more computer programs are stored, and when the one or more computer programs are executed by a computer processor, any of the methods described above is implemented.

[0029] The technical effect of the present invention is that: the present invention uses a unified management terminal to uniformly manage and calculate the data resources of multiple clusters, easily realizes unified management and calculation of the data resources of multiple clusters, and users can more efficiently manage and utilize the data of distributed clusters and improve processing efficiency and quality, ensure data consistency and synchronization, reduce delays in replication and synchronization between data of different clusters, perform real-time calculations on data, reduce high latency problems in long-distance communication and cross-regional communication, ensure high availability and fault tolerance of distributed clusters, and reduce cluster maintenance costs and resource consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0031] Figure 1 It is an overall flow chart of a unified control management method based on multi-data center computing provided according to an embodiment of the present invention.

[0032] Figure 2 It is an overall framework diagram of a unified control management method based on multi-data center computing provided according to an embodiment of the present invention.

[0033] Figure 3 It is a module diagram of a unified control and management system based on multi-data center computing provided according to an embodiment of the present invention.

[0034] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0035] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0036] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] Figure 1 A data synchronization method based on a graph neural network of the present invention is shown, and the method comprises:

[0038] S1: The central controller CC sends the task request information TKINFO sent by the user to the registration center RC through the registration center client RCC;

[0039] S2: The registration center RC matches the task request information TKINFO with the distributed coordination and configuration management tool according to the type of the data center in the task request information TKINFO, and returns the matching result to the central controller CC through the registration center client RCC;

[0040] Specifically, the registration center RC obtains the IP and port of the target service from the distributed coordination and configuration management tool according to the type of the data center of the task request information TKINFO and sends the task request information TKINFO to the target service for matching according to the IP and port of the target service;

[0041] S3: The central controller CC combines the service information corresponding to the data center of the successfully matched task request information TKINFO with the type of the data center, and encapsulates it into a computing request and sends it to the computing service;

[0042] S4: The computing service matches the computing request to the specific executor of the unified computing portal UCE according to the executor type to complete the computing task. The specific executor returns the result of the computing task to the user through the computing service and the central controller CC in sequence.

[0043] Specifically, Figure 2 The overall framework schematic diagram of the present invention is shown, and the specific method is as follows:

[0044] In the first step, the user sends a task request TKINFO to the central controller CC;

[0045] In the second step, the central controller CC will call the registration center client RCC to send the task request TKINFO to the registration center RC and match it with the distributed collaboration and configuration management tool that has been initialized through the data center type;

[0046] Step 3: If the data type matches successfully, the corresponding service information of the data center is returned. If the match fails, an exception is returned, and the result is returned to the registration center client RCC through the registration center RC, and then returned to the central controller CC through the registration center client RCC;

[0047] Step 4: If the match fails and the returned result is abnormal, the task is interrupted. If the match succeeds, the central controller CC combines the corresponding task information with the data center type, encapsulates it into a computing request and sends it to the computing service.

[0048] Step 5: The computing service matches the task request to the specific executor in the unified computing entry according to the computing request and the transmitted executor type;

[0049] Step 6: After completing the computing task, the specific executor returns the result or exception to the central controller CC via the computing service, and the central controller CC returns it to the user.

[0050] It should be noted that the computing service is started at the same time as the central controller CC is started, and the computing service stores the address and port of the data center in the distributed coordination and configuration management tool specified by the registration center RC.

[0051] It should be noted that the central controller analyzes and determines the operation results through a preset program, sends commands to the execution device so that the device can work according to the instructions, and can also automatically control the operation of the device according to the set time, trigger conditions and execution tasks.

[0052] It should be noted that the computing service is a service that provides computing power, provides a safe, reliable, flexible and highly usable environment, ensures the long-term and stable operation of the service, and improves operation and maintenance efficiency.

[0053] It should be noted that the registration center is an institution in the network that is responsible for verifying user digital authentication requests and informing the authentication center of the results for issuing authentication. It is a core basic service in the distributed system that can realize data storage, data synchronization and data change, and perform service governance and centralized service registration.

[0054] It should be noted that the distributed coordination and configuration management tool can improve the data consistency, load balancing and cluster management functions of the application.

[0055] In a specific embodiment, the distributed collaboration and configuration management tools include Zookeeper, Nacos and Eureka.

[0056] It should be noted that the task request information TKINFO includes a task request of a data center type, a task engine type and specific task information.

[0057] It should be noted that the distributed collaboration and configuration management tool is initialized before matching.

[0058] It should be noted that the specific executors include session mode executors, independent mode executors and interactive executors.

[0059] It should be noted that the session mode executor is used for the calculation of online tasks, the independent mode executor is used for the calculation of offline tasks, and the interactive executor is used for the calculation of ad hoc tasks.

[0060] It should be noted that the present invention has been implemented in the Tiansuan application of the Qiankun big data operating system, and has been verified many times in the practical modeling work of smart cities, epidemic prevention and control, and anti-fraud at the ministerial and provincial levels in recent years. In the actual modeling process, by calling the unified control and management system for multi-data center calculations, the problem of system complexity has been solved to a certain extent, ensuring data synchronization, consistency, and coordination of fault recovery between clusters.

[0061] It should be noted that by calling a unified control and management system for multi-data center computing, the correlation and dependency between clusters can be minimized to the greatest extent, ensuring that the expansion of each cluster will not have a negative impact on the performance and stability of the entire system.

[0062] It should be noted that each cluster is managed by a unified management terminal. Therefore, by calling the unified control and management system for multi-data computing, the problem of affecting the normal operation of other clusters due to the failure of one cluster can be minimized, ensuring the reliability and fault tolerance of the system.

[0063] Figure 3 A module diagram of a unified control and management system based on multi-data center computing is shown, including:

[0064] Sending module a: The central controller CC sends the task request information TKINFO sent by the user to the registration center RC through the registration center client RCC;

[0065] Matching module b: the registration center RC matches the task request information TKINFO with the distributed coordination and configuration management tool according to the type of data center in the task request information TKINFO, and returns the matching result to the central controller CC through the registration center client RCC;

[0066] Specifically, the registration center RC obtains the IP and port of the target service from the distributed coordination and configuration management tool according to the type of the data center of the task request information TKINFO and sends the task request information TKINFO to the target service for matching according to the IP and port of the target service;

[0067] Packaging module c: the central controller CC combines the service information corresponding to the data center of the successfully matched task request information TKINFO with the type of the data center, and packs it into a computing request and sends it to the computing service;

[0068] Computing module d: The computing service matches the computing request to the specific executor of the unified computing entry UCE according to the executor type to complete the computing task, and the specific executor returns the result of the computing task to the user through the computing service and the central controller CC in sequence.

[0069] In a specific embodiment, the computing service is started simultaneously with starting the central controller CC, and the computing service stores the address and port of the data center in the distributed coordination and configuration management tool specified by the registration center RC.

[0070] In a specific embodiment, the task request information TKINFO includes a task request of a data center type, a task engine type, and specific task information.

[0071] In a specific embodiment, the distributed collaboration and configuration management tool is initialized before matching.

[0072] In a specific embodiment, in the matching module b, if the task request information TKINFO successfully matches the distributed coordination and configuration management tool, the service information corresponding to the data center of the task request information TKINFO is sent to the central controller CC; otherwise, if the match fails, an exception is returned to the central controller CC and the task is interrupted.

[0073] In a specific embodiment, the specific executors include a session mode executor, an independent mode executor and an interactive executor.

[0074] In a specific embodiment, the session mode executor is used for the calculation of online tasks, the standalone mode executor is used for the calculation of offline tasks, and the interactive executor is used for the calculation of ad hoc tasks.

[0075] Reference below Figure 4 , which shows a schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application. Figure 4The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0076] like Figure 4 As shown, the computer system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage part 408 into a random access memory (RAM) 403. Various programs and data required for system operation are also stored in the RAM 403. The CPU 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0077] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0078] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 409, and / or installed from the removable medium 411. When the computer program is executed by the central processing unit (CPU) 401, the above functions defined in the method of the present application are executed. It should be noted that the computer-readable storage medium of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, - but not limited to - a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

[0079] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0080] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0081] The modules involved in the embodiments of the present application may be implemented by software or by hardware.

[0082] As another aspect, the present application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiment; or it may exist independently and not be assembled into the electronic device. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device: the central controller CC sends the task request information TKINFO sent by the user to the registration center RC through the registration center client RCC; the registration center RC matches the task request information TKINFO with the distributed coordination and configuration management tool according to the type of the data center in the task request information TKINFO, and returns the matching result to the central controller CC through the registration center client RCC; the central controller CC combines the type of the data center, splices the service information corresponding to the data center of the successfully matched task request information TKINFO, and encapsulates it into a computing request and sends it to the computing service; the computing service matches the computing request to the specific executor of the unified computing entry UCE according to the executor type to complete the computing task, and the specific executor returns the result of the computing task to the user through the computing service and the central controller CC in sequence.

[0083] Finally, it should be noted that the above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above invention concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A unified control and management method based on multi-data center computing, characterized in that: include: S1: The central controller CC sends the task request information TKINFO sent by the user to the registration center RC through the registration center client RCC; S2: The registration center RC matches the task request information TKINFO with the distributed coordination and configuration management tool according to the type of the data center in the task request information TKINFO, and returns the matching result to the central controller CC through the registration center client RCC; Specifically, the registration center RC obtains the IP and port of the target service from the distributed coordination and configuration management tool according to the type of the data center of the task request information TKINFO and sends the task request information TKINFO to the target service for matching according to the IP and port of the target service; S3: The central controller CC combines the service information corresponding to the data center of the successfully matched task request information TKINFO with the type of the data center, and encapsulates it into a computing request and sends it to the computing service; S4: The computing service matches the computing request to the specific executor of the unified computing portal UCE according to the executor type to complete the computing task. The specific executor returns the result of the computing task to the user through the computing service and the central controller CC in sequence.

2. The method according to claim 1, characterized in that The computing service is started at the same time as the central controller CC is started, and the computing service stores the address and port of the data center in the distributed coordination and configuration management tool specified by the registration center RC.

3. The method according to claim 1, characterized in that The task request information TKINFO includes a task request of a data center type, a task engine type and specific task information.

4. The method according to claim 1, characterized in that: The distributed collaboration and configuration management tool is initialized before matching.

5. The method according to claim 1, characterized in that In step S2, if the task request information TKINFO successfully matches the distributed coordination and configuration management tool, the service information corresponding to the data center of the task request information TKINFO is sent to the central controller CC. Otherwise, if the match fails, an exception is returned to the central controller CC and the task is interrupted.

6. The method according to claim 1, characterized in that The specific executors include a session mode executor, an independent mode executor and an interactive executor.

7. The method according to claim 6, characterized in that The session mode executor is used for the calculation of online tasks, the independent mode executor is used for the calculation of offline tasks, and the interactive executor is used for the calculation of ad hoc tasks.

8. A unified control and management system based on multi-data center computing, characterized in that: include: Sending module: The central controller CC sends the task request information TKINFO sent by the user to the registration center RC through the registration center client RCC; Matching module: the registration center RC matches the task request information TKINFO with the distributed coordination and configuration management tool according to the type of data center in the task request information TKINFO, and returns the matching result to the central controller CC through the registration center client RCC; Specifically, the registration center RC obtains the IP and port of the target service from the distributed coordination and configuration management tool according to the type of the data center of the task request information TKINFO and sends the task request information TKINFO to the target service for matching according to the IP and port of the target service; Packaging module: The central controller CC combines the type of the data center, splices the service information corresponding to the data center of the successfully matched task request information TKINFO, and packs it into a computing request and sends it to the computing service; Computing module: The computing service matches the computing request to the specific executor of the unified computing entry UCE according to the executor type to complete the computing task. The specific executor returns the result of the computing task to the user through the computing service and the central controller CC in sequence.

9. A computer storable medium having one or more computer programs thereon, characterized in that: The method according to any one of claims 1 to 7 is implemented when the one or more computer programs are executed by a computer processor.

Images