Cluster data processing method, device and cluster data processing architecture

Abstract

The present application provides a cluster data processing method, device and cluster data processing architecture. The method is applied to a first configuration module, and the method includes: monitoring cache data of a cache unit in at least one proxy module, the cache data is obtained by the cache unit loading data in a storage unit located in the management layer, and the cache data includes cache cluster metadata with different identifiers, and different identifiers correspond to different configuration modules; if the monitored cache data includes updated first cache cluster metadata, then the corresponding target cluster is deployed based on the first cache cluster metadata, and the identifier corresponding to the first cache cluster metadata is the target identifier corresponding to the first configuration module.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a cluster data processing method, device and cluster data processing architecture. Background Art

[0002] In business cloud computing, traditional applications are migrated to the cloud platform so that applications are centralized in the cloud market, making it easier to manage and update application clusters in the cloud platform.

[0003] During the migration of some applications, cluster metadata is stored on the cloud host and cluster deployment is achieved through the cluster metadata in the cloud host. However, in this method, the reliability and security of cluster creation are low. If the cloud host is overloaded or the network fluctuates, the cluster metadata stored in the cloud host cannot be provided, resulting in the inability to create a cluster. Summary of the invention

[0004] The present application provides a cluster data processing method, device and cluster data processing architecture to solve the problem of low security in cluster data creation.

[0005] In a first aspect, the present application provides a cluster data processing method, which is applied to a first configuration module, and the method includes:

[0006] Monitor cache data of a cache unit in at least one proxy module, where the cache data is obtained by the cache unit loading data in a storage unit in a management layer, and the cache data includes cache cluster metadata with different identifiers, where different identifiers correspond to different configuration modules;

[0007] If it is monitored that the cache data includes updated first cache cluster metadata, a corresponding target cluster is deployed based on the first cache cluster metadata, and an identifier corresponding to the first cache cluster metadata is a target identifier corresponding to the first configuration module.

[0008] In one possible implementation, the method further includes:

[0009] Acquire indicator pulling information sent by an indicator acquisition unit of at least one proxy module, wherein the indicator pulling information is used to indicate a target indicator, and the target indicator is obtained by an indicator receiving module located in the management layer;

[0010] Based on the indicator pulling information, pull indicator data corresponding to the target indicator from the business data corresponding to the target cluster;

[0011] The indicator data is sent to the indicator acquisition unit.

[0012] In this manner, the indicator pulling information indicating the target indicator is transmitted through the indicator receiving module and the indicator acquiring unit, thereby breaking the firewall between the indicator receiving module and the first configuration module, and pulling the indicator data in each container can be achieved.

[0013] In a second aspect, the present application provides a cluster data processing method, which is applied to a first agent module, and the method includes:

[0014] Based on the cache unit, data in the storage unit located in the management layer is loaded to obtain cache data, wherein the cache data includes cache cluster metadata with different identifiers, and different configuration modules correspond to cache cluster metadata with different identifiers. The updated cache cluster metadata in the cache data is used to instruct the corresponding configuration module to deploy the cluster based on the updated cache cluster metadata.

[0015] In one possible implementation, the method further includes:

[0016] Receiving indicator pulling information sent by an indicator receiving module located in the management layer, wherein the indicator pulling information is used to indicate a target indicator, and the target indicator is obtained by the indicator obtaining receiving module located in the management layer;

[0017] Sending the indicator pulling information to the configuration module, the indicator pulling information is also used to instruct the configuration module to pull the indicator data corresponding to the target indicator from the business data of the cluster deployed by the configuration module based on the indicator pulling information;

[0018] Receiving the indicator data returned by the configuration module;

[0019] The indicator data is sent to the indicator receiving module.

[0020] In a third aspect, the present application provides a cluster data processing device for a distributed system, applied to a first configuration module, the device comprising:

[0021] A monitoring module, used to monitor cache data of a cache unit in at least one proxy module, wherein the cache data is obtained by the cache unit loading data in a storage unit located in a management layer, and the cache data includes cache cluster metadata with different identifiers, and different identifiers correspond to different configuration modules;

[0022] The deployment module is configured to deploy a corresponding target cluster based on the first cache cluster metadata if the cache data is monitored to include updated first cache cluster metadata, wherein the identifier corresponding to the first cache cluster metadata is the target identifier correspondingly monitored by the first configuration module.

[0023] In a fourth aspect, the present application provides a cluster data processing device of a distributed system, which is applied to a first agent module, and the device includes:

[0024] A cache module is used to load data in a storage unit located in a management layer based on a cache unit to obtain cache data, wherein the cache data includes cache cluster metadata with different identifiers, and different configuration modules correspond to cache cluster metadata with different identifiers. The updated cache cluster metadata in the cache data is used to instruct the corresponding configuration module to deploy the cluster based on the updated cache cluster metadata.

[0025] In a fifth aspect, the present application provides a cluster data processing architecture, including at least one proxy module, at least one configuration module, and a storage unit: wherein,

[0026] The storage unit is arranged at the management layer, and the storage unit stores cluster metadata;

[0027] Each proxy module includes a cache unit, in which cache data is stored. The cache data is obtained by loading data in the storage unit. The cache data includes cache cluster metadata with different identifiers, and different identifiers correspond to different configuration modules.

[0028] The at least one proxy module and the at least one configuration module are both communicatively connected, the at least one configuration module is used to monitor cache data of a cache unit in the at least one proxy module, the at least one configuration module creates a cluster based on updated cache cluster metadata of a corresponding identifier monitored by the cache data, and different configuration modules monitor cache cluster metadata of different identifiers.

[0029] In a sixth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0030] The memory stores computer-executable instructions;

[0031] The processor executes the computer-executable instructions stored in the memory to implement the method according to the first aspect or the second aspect.

[0032] In a seventh aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect or the second aspect.

[0033] In an eighth aspect, the present application provides a computer program product, including a computer program, which implements the method described in the first aspect or the second aspect when executed by a processor.

[0034] The cluster data processing method, device and cluster data processing architecture provided by the present application, the storage unit is set in the management layer, the cache unit in the proxy module loads the data in the storage unit, the excessive load or network communication abnormality in one proxy module affects the cache unit in the proxy module, the configuration module can monitor the data in the cache units of other proxy modules to deploy the cluster, thereby improving the security and reliability of cluster data processing; at the same time, different configuration modules correspond to cache cluster metadata with different identifiers, which can realize logical isolation between cache cluster metadata, improve the security of cache cluster metadata, and further improve the security and reliability of cluster data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0036] Figure 1 is a structural diagram of a cluster data processing architecture shown in an exemplary embodiment;

[0037] Figure 2 is a flow chart of a cluster data processing method shown in an exemplary embodiment;

[0038] Figure 3 is a flow chart of a cluster data processing method shown in another exemplary embodiment;

[0039] Figure 4 is a structural diagram of a cluster data processing device shown in an exemplary embodiment;

[0040] Figure 5 is a structural diagram of a cluster data processing device shown in another exemplary embodiment;

[0041] Figure 6 It is a block diagram of an electronic device shown in an exemplary embodiment.

[0042] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0043] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0044] Technical name explanation:

[0045] High Availability (HA): refers to improving the availability of systems and applications by minimizing downtime caused by routine maintenance operations (planned) and sudden system crashes (unplanned). It is different from fault-tolerant technology, which is considered to be uninterrupted operation. HA system is the most effective means for enterprises to prevent core computer systems from downtime due to failures.

[0046] Cloud Application Management: Aims to solve application lifecycle management issues in complex cloud environments, providing a unified interface and powerful functions, including application creation, version control, review, release, deployment, upgrade and rollback operations to meet the growing demand for cloud application management.

[0047] Cloud computing platform: also known as cloud platform, refers to services based on hardware resources and software resources, providing computing, network and storage capabilities. Cloud computing platforms can be roughly divided into three categories: storage-based cloud platforms that focus on data storage, computing-based cloud platforms that focus on data processing, and comprehensive cloud computing platforms that take into account both computing and data storage processing.

[0048] Application services based on cloud computing have been widely used, and more and more traditional applications have been ported to cloud platforms. Usually, these applications are concentrated in the cloud market, and it only takes a very small cost to put them on the cloud. Application services on the cloud are required to be agile, scalable, stable, and monitorable. How to ensure that the deployed application clusters and services can run continuously and stably will become a key issue to be solved in the cloud market.

[0049] OpenPitrix (an open source platform) proposes a cluster management platform, which includes multiple components. The Drone component can automatically monitor the updated cluster metadata, perform cluster configuration, and communicate with the Frontgate component. The Frontgate component can realize the communication between the Pilot component and the Drone component, and can store cluster metadata. The Drone component monitors the cluster metadata stored by the Frontgate component and performs cluster updates.

[0050] However, in the OpenPitrix platform, the proxy module only stores the cluster metadata of the cluster assigned to the proxy module. When the proxy module is overloaded or the network is abnormal, the cluster metadata stored in the proxy module cannot provide support for application cluster creation, update and other operations, and other proxy modules cannot provide cluster metadata assigned to the proxy module. Therefore, the OpenPitrix platform cannot provide high availability support for application creation.

[0051] The cluster data processing method, device and cluster data processing architecture provided in this application are intended to solve the above technical problems in the prior art.

[0052] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0053] The cluster data processing method provided in this application can be implemented in Figure 1 The cluster data processing architecture shown.

[0054] like Figure 1 As shown, the cluster data processing architecture includes at least one proxy module, at least one configuration module, and a storage unit.

[0055] The storage unit is set at the management layer, and the storage unit stores cluster metadata.

[0056] At least one proxy module and at least one configuration module are arranged at the business layer.

[0057] Each proxy module can communicate with the storage unit. Each proxy module includes a cache unit. The cache unit stores cache data. The cache data is obtained by loading data in the storage unit. The cache data includes cache cluster metadata with different identifiers. Different identifiers correspond to different proxy modules.

[0058] although Figure 1 It is not shown in the figure, but it can be understood that the storage unit is connected to each proxy module, specifically, the cache unit is connected to the storage unit.

[0059] The cache unit can monitor the data in the storage unit, and when the data in the storage unit is updated, the data in the storage unit is updated synchronously.

[0060] In the cache unit, each virtual private cloud (VPC) has a corresponding identifier, and different VPCs correspond to different identifiers.

[0061] One VPC may correspond to one or more proxy modules, and one VPC may correspond to one configuration module, which creates a cluster on the virtual machine (VM) defined by the VPC.

[0062] In this way, different VPCs correspond to cache cluster metadata with different identifiers in the cache data, different configuration modules correspond to cache cluster metadata with different identifiers in the cache data, and different identifiers correspond to different configuration modules.

[0063] At least one proxy module and at least one configuration module are communicatively connected, at least one configuration module is used to monitor cache data of a cache unit in at least one proxy module, at least one configuration module creates a cluster based on updated cache cluster metadata of a corresponding identifier monitored by the cache data, and different configuration modules monitor cache cluster metadata of different identifiers.

[0064] A configuration module corresponds to a virtual machine (VM) that it deploys. For a configuration module, a cluster can be created on the corresponding virtual machine.

[0065] In some embodiments, the relevant functions of the configuration module can refer to the Drone component of OpenPitrix.

[0066] In some embodiments, the configuration module includes a deployment unit. The relevant functions of the deployment unit can refer to the functions of Confd in the Drone component. The difference is that the deployment unit monitors the cache data in the cache unit.

[0067] In some embodiments, Figure 1 As shown, the cluster data processing architecture also includes an indicator receiving unit, the proxy module may also be provided with an indicator acquisition unit, the deployment unit is further provided with an indicator pulling unit, the indicator receiving unit and the indicator acquisition unit are communicatively connected, the indicator pulling unit and the indicator acquisition unit run in the same network or are not in the same network but the single network communication is opened, that is, the indicator pulling unit and the indicator acquisition unit are communicatively connected.

[0068] The indicator receiving unit can receive the target indicator through the interface and generate indicator pulling information. The indicator pulling information is transmitted to the indicator acquisition module. The indicator acquisition module transmits the indicator pulling information to the indicator pulling unit. The indicator pulling unit pulls the indicator data corresponding to the target indicator from the business data of the cluster deployed by the corresponding configuration module, and sends the pulled indicator data to the indicator acquisition module. The indicator acquisition module sends the indicator data to the indicator receiving unit. The indicator receiving unit stores the indicator data or can display the indicator data through the connected visualization interface.

[0069] In some embodiments, the indicator receiving unit is disposed at the management layer.

[0070] In some implementations, the indicator receiving unit may be prometheus (an open source tool), the indicator acquisition module may be prometheus-proxy (an open source tool), and the indicator pulling unit may be prometheus-agent (an open source tool).

[0071] Prometheus can use the pull model to collect metrics, but Figure 1 In the cluster data processing architecture, Prometheus is set at the management level. When there is a firewall between it and the business data of the cluster, the pull model will have problems and cannot pull the indicator data. In this embodiment, prometheus-proxy and prometheus-agent are used to enable Prometheus to reach the measurement endpoint (the business data of the cluster) running behind the firewall, and retain the pull model to pull the indicator data.

[0072] In some embodiments, the cluster data processing architecture further includes a control module, which is used to receive cluster creation data, control data, etc. Each proxy module may also include an agent unit, which implements communication between the control module and the configuration module.

[0073] In some embodiments, the functions implemented by the control module may refer to the Pilot component of OpenPitrix.

[0074] In some embodiments, the control module receives cluster creation information, and the basic creation information can be forwarded to the agent unit, which is connected to the storage unit. After receiving the cluster creation information, the agent unit registers the basic metadata corresponding to the cluster creation information in the storage unit.

[0075] In some embodiments, the configuration module may also include a business processing unit. The control module may receive control data, such as data for starting a cluster or data for performing business control on the cluster. The control data is transmitted to the business processing unit through the proxy module, and the business processing unit executes the business through the control data.

[0076] In some embodiments, the implementation of related functions of the business processing unit may refer to the agent in Drone of OpenPitrix.

[0077] In some embodiments, the cluster data processing architecture further includes a load balancing module, which can perform load balancing distribution when any two structures among the control module, the configuration module, the proxy module, and the storage unit perform data transmission.

[0078] For example, in one embodiment, the control module receives cluster creation information, and the load balancing module can determine one or more proxy modules among multiple proxy modules through built-in load balancing rules to load the cluster metadata stored in the basic creation information storage unit. The implementation method of the relevant load balancing module can refer to the implementation method of load balancing in OpenPitrix, and no specific restrictions are made here.

[0079] Figure 2 is a flow chart of a cluster data processing method shown in an exemplary embodiment, which is applied to Figure 1 Configuration modules in Figure 2 As shown, the method includes steps S201 to S203, which are described in detail as follows:

[0080] S201. Monitor cache data of a cache unit in at least one proxy module. The cache data is obtained by loading data from a storage unit in a management layer by the cache unit. The cache data includes cache cluster metadata with different identifiers. Different identifiers correspond to different configuration modules.

[0081] In some embodiments, the first configuration target is Figure 1 Any configuration module in .

[0082] In some embodiments, the deployment unit in the configuration module monitors cache data of the cache unit in at least one proxy module.

[0083] In some embodiments, the deployment unit monitors cache data of cache units in one or more proxy modules assigned by the load balancing module.

[0084] In some embodiments, the deployment unit monitors cached data in one proxy module. If the load of the monitored proxy module is too large, the deployment unit may monitor cached data in other proxy modules.

[0085] It can be understood that the cache data in different proxy modules is the full amount of data in the storage unit.

[0086] In some embodiments, cached data is isolated through different identifiers to ensure the security of cluster metadata.

[0087] In some embodiments, different VPCs correspond to different identifiers, and cluster metadata deployed on different VPCs are isolated by the identifiers of the corresponding VPCs.

[0088] In some embodiments, the VPC corresponds to a configuration module, that is, different configuration modules correspond to different identifiers.

[0089] In some embodiments, the configuration module monitors the cluster metadata corresponding to the identifier in the cache data.

[0090] S202: If the monitored cache data includes updated first cache cluster metadata, deploy a corresponding target cluster based on the first cache cluster metadata, where the identifier corresponding to the first cache cluster metadata is the target identifier corresponding to the first configuration module.

[0091] In some embodiments, the identifier corresponding to the first configuration module is a target identifier, and the deployment unit in the first configuration module monitors the cache cluster metadata of the target identifier. If the cache cluster metadata of the target identifier changes, such as the cache cluster metadata of the target identifier is added or modified, the cache cluster metadata of the target identifier is determined as the first cache cluster metadata.

[0092] The deployment unit corresponding to the first configuration module runs the cached first cache cluster metadata and builds the corresponding target machine on the VM deployed by the corresponding VPC.

[0093] In an embodiment of the present application, the storage unit is arranged in the management layer, and the cache unit in the proxy module loads the data in the storage unit. If the load in one proxy module is too high or the network communication is abnormal, it will affect the cache unit in the proxy module. The configuration module can monitor the data in the cache units of other proxy modules to deploy the cluster, thereby improving the security and reliability of cluster data processing.

[0094] In some implementations, indicator pulling information sent by an indicator acquisition unit of at least one proxy module is obtained, and the indicator pulling information is used to indicate a target indicator, and the target indicator is obtained by an indicator receiving module located in the management layer; based on the indicator pulling information, indicator data corresponding to the target indicator is pulled from the business data corresponding to the target cluster; and the indicator data is sent to the indicator acquisition unit.

[0095] In some embodiments, a method of pulling indicators is also proposed. The indicator receiving module at the management level can receive the target indicator, or receive indicator pulling information. The indicator pulling information can be information of the target indicator, or information indirectly indicating the target indicator. No specific restrictions are made here.

[0096] In some embodiments, the indicator acquisition unit serves as a communication channel to transmit the indicator pulling information to the first configuration module, and the first configuration module can pull data based on the target indicator.

[0097] In some embodiments, the indicator pulling unit in the first configuration module pulls indicator data of the target indicator. In some embodiments, the first configuration module creates multiple clusters on the corresponding VM, such as the target cluster created above. The indicator pulling information can also indicate the pulled cluster, and pull the target indicator for the business data in the indicated cluster, or directly pull the target indicator for the newly created target cluster, or pull the target indicator for all clusters configured by the first configuration module.

[0098] In some embodiments, other configuration modules may also receive the indicator pulling information and pull target indicators for the business data of the cluster created by the other configuration modules.

[0099] In an embodiment of the present application, indicator pulling information used to indicate the target indicator is transmitted through the indicator receiving module and the indicator acquisition unit, thereby breaking the firewall between the indicator receiving module and the first configuration module, and pulling the indicator data of each VM can be achieved. The obtained indicator data can be used to detect the operating status of the cluster.

[0100] Figure 3 is a flowchart of a cluster data processing method shown in another exemplary embodiment, which is applied to Figure 1 The proxy module in the method includes step S301, which is described in detail as follows:

[0101] S301. Load the data in the storage unit located in the management layer based on the cache unit to obtain cache data, where the cache data includes cache cluster metadata with different identifiers. Different configuration modules correspond to cache cluster metadata with different identifiers. The updated cache cluster metadata in the cache data is used to instruct the corresponding configuration module to deploy the cluster based on the updated cache cluster metadata.

[0102] In some embodiments, the first agent module is Figure 1 Any proxy module in .

[0103] The cache unit will load the data in the storage unit to obtain cache data, which is the full data in the storage unit. The subsequent configuration module can deploy the cluster based on the data in the storage unit.

[0104] In some embodiments, the deployment unit in the configuration module monitors the cache data in the cache unit. When there is updated cache cluster metadata in the cache data, the configuration module corresponding to the identifier of the updated cache cluster metadata can create a cluster based on the updated cache cluster metadata.

[0105] In an embodiment of the present application, a storage unit is arranged in the management layer, a cache unit in the proxy module loads data in the storage unit, and a configuration module can monitor data in the cache units of other proxy modules to deploy the cluster, thereby improving the security and reliability of cluster data processing.

[0106] In some implementations, indicator pulling information sent by an indicator receiving module located in the management layer is received, and the indicator pulling information is used to indicate a target indicator, which is obtained by an indicator acquisition receiving module located in the management layer; the indicator pulling information is sent to a configuration module, and the indicator pulling information is also used to instruct the configuration module to pull indicator data corresponding to the target indicator from the business data of the cluster deployed by the configuration module based on the indicator pulling information; the indicator data returned by the configuration module is received; and the indicator data is sent to the indicator receiving module.

[0107] In some embodiments, the indicator receiving module located at the management level transmits the indicator pulling information to the configuration module through the indicator acquisition unit. It can be transmitted to all configuration modules, or it can be at least one configuration module indicated by the load balancing section. The configuration module that receives the indicator pulling information pulls the target indicator.

[0108] In the embodiment of the present application, the indicator pulling information used to indicate the target indicator is transmitted through the indicator receiving module and the indicator acquisition unit, so that the indicator data of each VM can be pulled, so that the status of the created cluster can be monitored.

[0109] Figure 4 4 is a structural diagram of a cluster data processing device shown in an exemplary embodiment, which is applied to the first configuration module. The cluster data processing device 400 may include:

[0110] A monitoring module 410 is used to monitor cache data of a cache unit in at least one proxy module, where the cache data is obtained by loading data from a storage unit in a management layer by the cache unit, and the cache data includes cache cluster metadata with different identifiers, where different identifiers correspond to different configuration modules;

[0111] The deployment module 430 is used to deploy the corresponding target cluster based on the first cache cluster metadata if the cache data monitored includes the updated first cache cluster metadata, and the identifier corresponding to the first cache cluster metadata is the target identifier correspondingly monitored by the first configuration module.

[0112] In one achievable manner, the cluster data processing device further includes:

[0113] An indicator acquisition module, used to acquire indicator pulling information sent by an indicator acquisition unit of at least one proxy module, the indicator pulling information is used to indicate a target indicator, and the target indicator is obtained by an indicator receiving module located in the management layer;

[0114] The indicator pulling module is used to pull the indicator data corresponding to the target indicator from the business data corresponding to the target cluster based on the indicator pulling information;

[0115] The indicator data sending module is used to send the indicator data to the indicator acquisition unit.

[0116] Figure 5 1 is a structural diagram of a cluster data processing device shown in an exemplary embodiment, which is applied to a first agent module. The cluster data processing device 500 may include:

[0117] The cache module 510 is used to load the data in the storage unit located in the management layer based on the cache unit to obtain cache data. The cache data includes cache cluster metadata with different identifiers. Different configuration modules correspond to cache cluster metadata with different identifiers. The updated cache cluster metadata in the cache data is used to instruct the corresponding configuration module to deploy the cluster based on the updated cache cluster metadata.

[0118] In an implementable manner, the cluster data processing device 500 may further include:

[0119] An indicator receiving module, used to receive indicator pulling information sent by the indicator receiving module in the management layer, the indicator pulling information is used to indicate the target indicator, and the target indicator is obtained by the indicator obtaining receiving module in the management layer;

[0120] An indicator sending module is used to send the indicator pulling information to the configuration module, and the indicator pulling information is also used to instruct the configuration module to pull the indicator data corresponding to the target indicator from the business data of the cluster deployed by the configuration module based on the indicator pulling information;

[0121] The indicator data receiving module is used to receive the indicator data returned by the configuration module;

[0122] The indicator data transmission module is used to send the indicator data to the indicator receiving module.

[0123] The cluster data processing device provided in this embodiment can be used to execute the above cluster data processing method. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0124] Figure 6 is a block diagram of an electronic device shown in an exemplary embodiment, see Figure 6 The electronic device 600 may include: a processor 61 and a memory 62, wherein the processor 61 and the memory 62 can communicate; illustratively, the processor 61 and the memory 62 communicate via a communication bus 63, the memory 62 is used to store computer execution instructions, and the processor 61 is used to call the computer execution instructions in the memory to execute the cluster data processing method shown in any of the above method embodiments.

[0125] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the present application may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0126] The present application provides a computer-readable storage medium, on which computer-executable instructions are stored; when the computer-executable instructions are executed by a processor, they are used to implement a cluster data processing method as described in any of the above embodiments.

[0127] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the cluster data processing method of the distributed system is implemented.

[0128] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0129] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0130] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0131] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0132] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0133] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0134] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0136] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A cluster data processing method, characterized in that: Applied to the first configuration module, the method comprises: Monitor cache data of a cache unit in at least one proxy module, where the cache data is obtained by the cache unit loading data in a storage unit in a management layer, and the cache data includes cache cluster metadata with different identifiers, where different identifiers correspond to different configuration modules; If it is monitored that the cache data includes updated first cache cluster metadata, a corresponding target cluster is deployed based on the first cache cluster metadata, and an identifier corresponding to the first cache cluster metadata is a target identifier corresponding to the first configuration module.

2. The method according to claim 1, characterized in that The method further comprises: Acquire indicator pulling information sent by an indicator acquisition unit of at least one proxy module, wherein the indicator pulling information is used to indicate a target indicator, and the target indicator is obtained by an indicator receiving module located in the management layer; Based on the indicator pulling information, pull indicator data corresponding to the target indicator from the business data corresponding to the target cluster; The indicator data is sent to the indicator acquisition unit.

3. A cluster data processing method, characterized in that: Applied to the first agent module, the method comprises: Based on the cache unit, data in the storage unit located in the management layer is loaded to obtain cache data, wherein the cache data includes cache cluster metadata with different identifiers, and different configuration modules correspond to cache cluster metadata with different identifiers. The updated cache cluster metadata in the cache data is used to instruct the corresponding configuration module to deploy the cluster based on the updated cache cluster metadata.

4. The method according to claim 3, characterized in that The method further comprises: Receiving indicator pulling information sent by an indicator receiving module located in the management layer, wherein the indicator pulling information is used to indicate a target indicator, and the target indicator is obtained by the indicator obtaining receiving module located in the management layer; Sending the indicator pulling information to the configuration module, the indicator pulling information is also used to instruct the configuration module to pull the indicator data corresponding to the target indicator from the business data of the cluster deployed by the configuration module based on the indicator pulling information; Receiving the indicator data returned by the configuration module; The indicator data is sent to the indicator receiving module.

5. A cluster data processing device, characterized in that: Applied to the first configuration module, the device comprises: A monitoring module, used to monitor cache data of a cache unit in at least one proxy module, wherein the cache data is obtained by the cache unit loading data in a storage unit located in a management layer, and the cache data includes cache cluster metadata with different identifiers, and different identifiers correspond to different configuration modules; The deployment module is configured to deploy a corresponding target cluster based on the first cache cluster metadata if the cache data is monitored to include updated first cache cluster metadata, wherein the identifier corresponding to the first cache cluster metadata is the target identifier correspondingly monitored by the first configuration module.

6. A cluster data processing device, characterized in that: Applied to the first proxy module, the device comprises: A cache module is used to load data in a storage unit located in a management layer based on a cache unit to obtain cache data, wherein the cache data includes cache cluster metadata with different identifiers, and different configuration modules correspond to cache cluster metadata with different identifiers. The updated cache cluster metadata in the cache data is used to instruct the corresponding configuration module to deploy the cluster based on the updated cache cluster metadata.

7. A cluster data processing architecture, characterized in that: It includes at least one proxy module, at least one configuration module, and a storage unit: wherein, The storage unit is arranged at the management layer, and the storage unit stores cluster metadata; Each proxy module includes a cache unit, in which cache data is stored. The cache data is obtained by loading data in the storage unit. The cache data includes cache cluster metadata with different identifiers, and different identifiers correspond to different configuration modules. The at least one proxy module and the at least one configuration module are both communicatively connected, the at least one configuration module is used to monitor cache data of a cache unit in the at least one proxy module, the at least one configuration module creates a cluster based on updated cache cluster metadata of a corresponding identifier monitored by the cache data, and different configuration modules monitor cache cluster metadata of different identifiers.

8. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 4 when executed by a processor.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 4 when being executed by a processor.

Images