An edge-side-based cluster node data processing method and system

Abstract

The present invention relates to the technical field of highly available clusters, and discloses a method and system for processing cluster node data based on the edge side. By obtaining the edge-side cluster node resources, querying the hardware performance parameters and idle periods of each dynamically accessed cluster node and constructing a node association information group, extracting the network condition prediction information of each dynamically accessed cluster node, considering the constraint set constructed from the node dynamic access distribution information and network condition prediction information corresponding to the node association information group of each dynamically accessed cluster node, planning the dynamic leader tenure in the target cluster data processing cycle, generating an election strategy and using this election strategy to perform interval elections during the tenure when executing the target operation service, realizing real-time dynamic adjustment of the master node of a highly available cluster considering the influence of multiple factors, helping the highly available cluster to perform tenure release and node switching of the master node before a failure occurs, and improving the operation stability and processing performance of the highly available cluster.

Description

Technical Field

[0001] The present invention relates to the technical field of high-availability clusters, and particularly to a method and system for processing cluster node data based on the edge side. Background Art

[0002] A high-availability cluster (HA Cluster) is a system composed of multiple nodes (physical machines or virtual machines) interconnected through a network. Its core goal is to ensure that services can still run continuously through redundant design and failover mechanisms when a single-point failure (such as node hardware failure, software crash, network interruption) occurs, achieving high service availability. Usually, when the master node (Leader or primary node) in a high-availability cluster fails, the system will achieve failover through the following steps to ensure business continuity. When the master node fails, the cluster completes the switch through the process of heartbeat detection → election of a new master → resource takeover → data synchronization. The core goal is to restore the service in the shortest time while ensuring that data is not lost and the business is not interrupted.

[0003] In actual application scenarios, the cluster node network is also affected by factors such as changes in the number of cluster nodes accessed due to dynamic access and exit of cluster nodes, network condition parameters caused by network environment fluctuations, and hardware differences of currently accessed cluster nodes. This challenges the stability of the high-availability cluster, and there is a need to adjust the master node of the high-availability cluster in real time in this scenario to ensure that the high-availability cluster always maintains high processing performance and running stability. The existing failover mechanism based on heartbeat detection does not have good applicability under the real-time adjustment requirements before the above-mentioned fault detection.

[0004] Therefore, how to achieve real-time dynamic adjustment of the master node of a high-availability cluster considering the influence of multiple factors, help the high-availability cluster to release the tenure of the master node and switch nodes before a fault occurs, and improve the running stability and processing performance of the high-availability cluster is a technical problem that urgently needs to be solved. Summary of the Invention

[0005] The main purpose of the present invention is to provide a method and system for processing cluster node data based on the edge side, aiming to solve at least one of the above technical problems.

[0006] To achieve the above purpose, the present invention provides a method for processing cluster node data based on the edge side, and the method includes the following steps:

[0007] Obtain the edge-side cluster node resources for executing the target operation service within the target cluster range; wherein, the edge-side cluster node resources include the node identifiers of several dynamically accessed cluster nodes;

[0008] Query the hardware performance parameters of each dynamically - accessed cluster node and the idle periods during the target cluster data - processing cycle according to the node identifier, and use the hardware performance parameters and the idle periods to construct a node - association information group for each dynamically - accessed cluster node;

[0009] Based on the node identifier, call the network - condition prediction information library, extract the network - condition prediction information of each dynamically - accessed cluster node during the target cluster data - processing cycle, consider the node - association information group and the network - condition prediction information of each dynamically - accessed cluster node, plan the dynamic leader tenure during the target cluster data - processing cycle, and generate an election strategy within the target cluster scope;

[0010] Deploy the election strategy to each dynamically - accessed cluster node within the target cluster scope, and drive each dynamically - accessed cluster node to perform interval elections during its tenure according to the election strategy when executing the target operation service.

[0011] Optionally, the steps for obtaining the edge - side cluster node resources within the target cluster scope specifically include:

[0012] Receive the target operation service transmitted by the service - running terminal, and extract the service - running type in the target operation service; where each service - running type is mapped to a corresponding service - running requirement;

[0013] Call the edge - side cluster node database, and perform a secondary screening of the edge - side cluster nodes according to the service - running requirements to obtain the edge - side cluster node resources that execute the target operation service within the target cluster scope.

[0014] Optionally, the steps for calling the edge - side cluster node database and performing a secondary screening of the edge - side cluster nodes according to the service - running requirements to obtain the edge - side cluster node resources that execute the target operation service within the target cluster scope specifically include:

[0015] Call the edge - side cluster node database, perform a first - stage screening of all edge - side cluster nodes within the entire cluster scope according to the hardware requirements for service execution in the service - running requirements, and perform a second - stage screening of all edge - side cluster nodes within the entire cluster scope according to the service - execution regional - scope requirements in the service - running requirements;

[0016] Use the edge - side cluster nodes after the two - stage screening as the edge - side cluster node resources that execute the target operation service within the target cluster scope.

[0017] Optionally, according to the node identifier, query the hardware performance parameters of each dynamically accessed cluster node and the idle period during the target cluster data processing cycle, and use the hardware performance parameters and the idle period to construct a node association information group for each dynamically accessed cluster node. The specific steps include:

[0018] Extract the comparison table of edge-side cluster nodes and hardware performance parameters and the business operation queue of each edge-side cluster node from the edge-side cluster node database.

[0019] According to the node identifier, query the hardware performance parameters of each dynamically accessed cluster node in the comparison table of edge-side cluster nodes and hardware performance parameters; according to the node identifier, extract the business operation queue of each dynamically accessed cluster node, and determine the idle period of each dynamically accessed cluster node during the target cluster data processing cycle.

[0020] Use the hardware performance parameters and the idle period to construct a node association information group for each dynamically accessed cluster node.

[0021] Optionally, based on the node identifier, call the network condition prediction information library to extract the network condition prediction information of each dynamically accessed cluster node during the target cluster data processing cycle. The specific steps include:

[0022] Call the network condition prediction information library; wherein, the network condition prediction information library records the network condition prediction information of each dynamically accessed cluster node.

[0023] Based on the node identifier, match the network condition prediction information of each dynamically accessed cluster node during the target cluster data processing cycle in the network condition prediction information library.

[0024] Optionally, the construction of the network condition prediction information library specifically includes:

[0025] Obtain the network condition parameters of each dynamically accessed cluster node in a number of historical cluster data processing cycles within the target cluster range; wherein, the network condition parameters include any one of data throughput, packet loss rate, or latency.

[0026] Calculate the average value of the network condition parameters of several dynamically accessed cluster nodes in each minimum data processing period during each historical cluster data processing cycle, and construct a training sample and a prediction sample for each historical cluster data processing cycle based on the average value of the network condition parameters and the timestamp in all minimum data processing periods during each historical cluster data processing cycle.

[0027] Using the training samples, train a pre-constructed convolutional neural network model, and use the prediction samples and the network condition prediction model obtained after training to predict the network condition prediction information of each dynamic access cluster node in the target cluster data processing cycle.

[0028] Optionally, considering the node association information group and the network condition prediction information of each dynamic access cluster node, plan the dynamic leader tenure in the target cluster data processing cycle, and generate the election strategy steps within the target cluster scope, specifically including:

[0029] Extract the idle periods in the node association information group of each dynamic access cluster node, and generate the node dynamic access distribution information of the target cluster data processing cycle according to the distribution of the idle periods in the target cluster data processing cycle; wherein, the node dynamic access distribution information includes the number of dynamic access cluster nodes in each minimum data processing period in the target cluster data processing cycle.

[0030] Extract the hardware performance parameters in the node association information group of each dynamic access cluster node, and divide each dynamic access cluster node into several performance levels according to the service operation type in the target operation service and the hardware performance parameters of each dynamic access cluster node.

[0031] Based on the performance levels of each dynamic access cluster node, calculate the average performance level of the dynamic access cluster nodes in each minimum data processing period of the target cluster data processing cycle in the node dynamic access distribution information, and write the average performance level into the node dynamic access distribution information to generate the node hardware performance distribution information including the number of dynamic access cluster nodes and the average performance level in each minimum data processing period in the target cluster data processing cycle.

[0032] According to the node hardware performance distribution information and the network condition prediction information, plan the dynamic leader tenure in the target cluster data processing cycle, and generate the election strategy within the target cluster scope.

[0033] Optionally, according to the node hardware performance distribution information and the network condition prediction information, plan the dynamic leader tenure in the target cluster data processing cycle, and generate the election strategy steps, specifically including:

[0034] Construct the number of dynamic access cluster nodes and the average performance level in the node hardware performance distribution information and the network condition parameters in the network condition prediction information into a constraint set.

[0035] Query the business operation failure rate of each constraint parameter group in the query constraint set during the business operation of historical cluster nodes with different leader tenures. With the business operation failure rate of each minimum data processing period in the target cluster data processing cycle being less than the failure rate allowable threshold after being assigned to a dynamic leader tenure as the constraint condition, and the sum of the number of times the dynamic leader tenures of every two adjacent minimum data processing periods are different being minimized as the optimization objective, optimize and solve the dynamic leader tenure set assigned to each minimum data processing period in the target cluster data processing cycle;

[0036] Use the dynamic leader tenure set as the election tenure interval in the target cluster data processing cycle to generate an election strategy within the target cluster scope.

[0037] Optionally, deploy the election strategy to each dynamically accessible cluster node within the target cluster scope, and drive each dynamically accessible cluster node to execute the tenure interval election step according to the election strategy during the target cluster data processing cycle when performing the target operation service, specifically including:

[0038] Merge the minimum data processing periods with the same dynamic leader tenure in two adjacent minimum data processing periods within the target cluster data processing cycle in the election strategy to generate a deployment sequence including several processing periods and the dynamic leader tenure of each processing period;

[0039] Distribute the dynamic leader tenure corresponding to each processing period to each dynamically accessible cluster node before each processing period, and drive each dynamically accessible cluster node to execute the tenure interval election according to the election strategy during the target cluster data processing cycle when performing the target operation service.

[0040] In addition, to achieve the above object, the present invention also provides a cluster node data processing system based on the edge side, including:

[0041] An acquisition module, configured to acquire edge side cluster node resources for performing target operation services within the target cluster scope; wherein, the edge side cluster node resources include the node identifiers of several dynamically accessible cluster nodes;

[0042] A construction module, configured to query the hardware performance parameters and idle periods of each dynamically accessible cluster node in the target cluster data processing cycle according to the node identifiers, and use the hardware performance parameters and the idle periods to construct a node association information group for each dynamically accessible cluster node;

[0043] A generation module, configured to call a network condition prediction information library based on the node identifier, extract network condition prediction information of each dynamically accessed cluster node in a target cluster data processing cycle, consider the node association information group and the network condition prediction information of each dynamically accessed cluster node, plan the dynamic leader tenure in the target cluster data processing cycle, and generate an election strategy within the target cluster scope;

[0044] An execution module, configured to deploy the election strategy to each dynamically accessed cluster node within the target cluster scope, and drive each dynamically accessed cluster node to perform an interval election during its tenure according to the election strategy when executing the target operation service.

[0045] The beneficial effects of the present invention are as follows: A method and system for processing cluster node data based on the edge side are proposed. By obtaining the resources of the edge side cluster nodes, querying the hardware performance parameters and idle periods of each dynamically accessed cluster node and constructing a node association information group, extracting the network condition prediction information of each dynamically accessed cluster node, considering the constraint set constructed by the node dynamic access distribution information and the network condition prediction information corresponding to the node association information group of each dynamically accessed cluster node, using the service operation failure rate of the leader tenure during the historical cluster node service operation of each constraint parameter group in the constraint set as a constraint condition, and taking the minimum number of dynamic adjustments of the leader task in the target cluster data processing cycle as an optimization goal, plan the dynamic leader tenure in the target cluster data processing cycle, generate an election strategy within the target cluster scope, and use this election strategy to perform an interval election during the execution of the target operation service, realizing the real-time dynamic adjustment of the master node of a highly available cluster considering multiple factors, helping the highly available cluster to publish the tenure of the master node and switch nodes before a failure occurs, and improving the operation stability and processing performance of the highly available cluster. Description of the Drawings

[0046] Figure 1 It is a flowchart of the method for processing cluster node data based on the edge side of the present invention;

[0047] Figure 2 It is a structural diagram of the system for processing cluster node data based on the edge side of the present invention. Detailed Embodiments

[0048] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] The embodiment of the present invention provides a method for processing cluster node data based on the edge side, referring to Figure 1 ,Figure 1 This is a schematic flowchart of an embodiment of the method for processing cluster node data based on the edge side of the present invention.

[0050] In this embodiment, a method for processing cluster node data based on the edge side, the method includes the following steps:

[0051] S100: Obtain the edge-side cluster node resources for executing the target operation service within the target cluster range; wherein, the edge-side cluster node resources include the node identifiers of a number of dynamically accessed cluster nodes;

[0052] S200: According to the node identifiers, query the hardware performance parameters of each dynamically accessed cluster node and the idle periods during the target cluster data processing cycle, and use the hardware performance parameters and the idle periods to construct a node association information group for each dynamically accessed cluster node;

[0053] S300: Based on the node identifiers, call the network condition prediction information library, extract the network condition prediction information of each dynamically accessed cluster node during the target cluster data processing cycle, consider the node association information group and the network condition prediction information of each dynamically accessed cluster node, plan the dynamic leader tenure during the target cluster data processing cycle, and generate an election strategy for the target cluster range;

[0054] S400: Deploy the election strategy to each dynamically accessed cluster node within the target cluster range, and drive each dynamically accessed cluster node to perform interval elections during the tenure according to the election strategy when executing the target operation service during the target cluster data processing cycle.

[0055] It should be noted that in the actual application scenario, the cluster node network is also affected by factors such as changes in the number of cluster nodes accessed caused by the dynamic access and exit of cluster nodes, network condition parameters caused by network environment fluctuations, and hardware differences of currently accessed cluster nodes, which challenges the stability of the highly available cluster. There is a need to adjust the master node of the highly available cluster in real time in this scenario to ensure that the highly available cluster always maintains high processing performance and running stability. The existing failover mechanism based on heartbeat detection does not have good applicability under the real-time adjustment requirements before the above-mentioned fault detection.

[0056] To solve the above problems, in this embodiment, by obtaining the edge-side cluster node resources, querying the hardware performance parameters and idle periods of each dynamically accessed cluster node and constructing a node association information group, extracting the network condition prediction information of each dynamically accessed cluster node, considering the constraint set constructed from the node dynamic access distribution information and network condition prediction information corresponding to the node association information group of each dynamically accessed cluster node, planning the dynamic leader tenure in the target cluster data processing cycle, generating an election strategy and using this election strategy to perform interval elections during the tenure when executing the target operation service, the real-time dynamic adjustment of the master node of the highly available cluster considering multiple factors is realized, which helps the highly available cluster to perform the tenure release and node switching of the master node before a failure occurs, and improves the operation stability and processing performance of the highly available cluster.

[0057] In a preferred embodiment, the step of obtaining the edge-side cluster node resources within the target cluster range specifically includes:

[0058] S110: Receive the target operation service transmitted by the service operation terminal, and extract the service operation type in the target operation service; wherein, each service operation type is mapped to a corresponding service operation requirement;

[0059] S120: Invoke the edge-side cluster node database, and perform a secondary screening on the edge-side cluster nodes according to the service operation requirements to obtain the edge-side cluster node resources within the target cluster range that execute the target operation service.

[0060] On this basis, the step of invoking the edge-side cluster node database and performing a secondary screening on the edge-side cluster nodes according to the service operation requirements to obtain the edge-side cluster node resources within the target cluster range that execute the target operation service specifically includes:

[0061] S121: Invoke the edge-side cluster node database, perform a first screening on the edge-side cluster nodes within the entire cluster range according to the service execution hardware requirements in the service operation requirements, and perform a second screening on the edge-side cluster nodes within the entire cluster range according to the service execution area range requirements in the service operation requirements;

[0062] S122: Use the edge-side cluster nodes after the two screenings as the edge-side cluster node resources within the target cluster range that execute the target operation service.

[0063] In this embodiment, obtaining the edge-side cluster node resources within the target cluster range is achieved by receiving the target operation service, extracting the service operation type, then performing a first screening according to the service execution hardware requirements in the service operation requirements and a second screening according to the service execution area range, and finally obtaining all the edge-side cluster node resources within the target cluster range that meet the target operation service.

[0064] In a preferred embodiment, according to the node identifier, query the hardware performance parameters of each dynamic access cluster node and the idle period during the target cluster data processing cycle, and use the hardware performance parameters and the idle period to construct the node association information group for each dynamic access cluster node. The steps specifically include:

[0065] S210: Extract the comparison table of edge-side cluster nodes and hardware performance parameters and the service operation queue of each edge-side cluster node from the edge-side cluster node database;

[0066] S220: According to the node identifier, query the hardware performance parameters of each dynamic access cluster node in the comparison table of edge-side cluster nodes and hardware performance parameters; according to the node identifier, extract the service operation queue of each dynamic access cluster node, and determine the idle period of each dynamic access cluster node during the target cluster data processing cycle;

[0067] S230: Use the hardware performance parameters and the idle period to construct the node association information group for each dynamic access cluster node.

[0068] In this embodiment, after obtaining the node identifier in the previous step, the hardware performance parameters recorded in the comparison table of edge-side cluster nodes and hardware performance parameters can also be queried, and the idle period of each dynamic access cluster node during the target cluster data processing cycle can be analyzed by using the service operation queue of each dynamic access cluster node, so as to construct the node association information group.

[0069] In a preferred embodiment, based on the node identifier, the step of calling the network condition prediction information library and extracting the network condition prediction information of each dynamic access cluster node during the target cluster data processing cycle specifically includes:

[0070] S310: Call the network condition prediction information library; wherein, the network condition prediction information library records the network condition prediction information of each dynamic access cluster node;

[0071] S320: Based on the node identifier, match the network condition prediction information of each dynamic access cluster node during the target cluster data processing cycle in the network condition prediction information library.

[0072] In practical applications, the construction of the network condition prediction information library specifically includes:

[0073] S311: Obtain the network condition parameters of each dynamic access cluster node in a number of historical cluster data processing cycles within the target cluster range; wherein, the network condition parameters include any one of data throughput, packet loss rate or latency;

[0074] S312: Calculate the average value of network condition parameters of several dynamically accessed cluster nodes in each minimum data processing period of each historical cluster data processing cycle. Based on the average value of network condition parameters and timestamps in all minimum data processing periods of each historical cluster data processing cycle, construct training samples and prediction samples for each historical cluster data processing cycle.

[0075] S313: Use the training samples to train a pre-constructed convolutional neural network model, and use the prediction samples and the network condition prediction model obtained after training to predict the network condition prediction information of each dynamically accessed cluster node in the target cluster data processing cycle.

[0076] In this embodiment, after obtaining the node identifiers in the previous steps, a network condition prediction information library can also be constructed by using the network condition prediction information of the target cluster data processing cycle predicted by the network condition prediction model. Then, according to the node identifiers, query the network condition prediction information of the target cluster data processing cycle, providing data support for planning the dynamic leader tenure in the target cluster data processing cycle.

[0077] In a preferred embodiment, considering the node association information group and network condition prediction information of each dynamically accessed cluster node, plan the dynamic leader tenure in the target cluster data processing cycle, and generate the election strategy steps within the target cluster scope, specifically including:

[0078] S330: Extract the idle periods in the node association information group of each dynamically accessed cluster node, and generate the node dynamic access distribution information of the target cluster data processing cycle according to the distribution of the idle periods in the target cluster data processing cycle; wherein, the node dynamic access distribution information includes the number of dynamically accessed cluster nodes in each minimum data processing period of the target cluster data processing cycle.

[0079] S340: Extract the hardware performance parameters in the node association information group of each dynamically accessed cluster node, and divide each dynamically accessed cluster node into several performance levels according to the service operation type in the target running service and the hardware performance parameters of each dynamically accessed cluster node.

[0080] S350: Based on the performance levels of each dynamically accessed cluster node, calculate the average value of the performance levels of the dynamically accessed cluster nodes in each minimum data processing period of the target cluster data processing cycle in the node dynamic access distribution information, write the average value of the performance levels into the node dynamic access distribution information, and generate the node hardware performance distribution information including the number of dynamically accessed cluster nodes and the average value of the performance levels in each minimum data processing period of the target cluster data processing cycle.

[0081] S360: According to the node hardware performance distribution information and the network condition prediction information, plan the dynamic leader tenure in the target cluster data processing cycle, and generate an election strategy for the target cluster scope.

[0082] Furthermore, the steps of planning the dynamic leader tenure in the target cluster data processing cycle and generating an election strategy for the target cluster scope according to the node hardware performance distribution information and the network condition prediction information specifically include:

[0083] S361: Construct a constraint set with the number of dynamically accessed cluster nodes and the average performance level in the node hardware performance distribution information and the network condition parameters in the network condition prediction information;

[0084] S362: Query the business operation failure rates of each constraint parameter group in the constraint set with different leader tenures during the historical cluster node business operation. Taking the business operation failure rate of each minimum data processing period in the target cluster data processing cycle after being assigned a dynamic leader tenure to be less than the failure rate allowable threshold as a constraint condition, and the minimum total number of times the dynamic leader tenures of every two adjacent minimum data processing periods are different as the optimization goal, optimize and solve the dynamic leader tenure set assigned to each minimum data processing period in the target cluster data processing cycle;

[0085] S363: Use the dynamic leader tenure set as the election tenure interval in the target cluster data processing cycle to generate an election strategy for the target cluster scope.

[0086] In this embodiment, by obtaining the edge-side cluster node resources, querying the hardware performance parameters and idle periods of each dynamically accessed cluster node and constructing a node association information group, extracting the network condition prediction information of each dynamically accessed cluster node, considering the constraint set constructed with the node dynamic access distribution information corresponding to the node association information group of each dynamically accessed cluster node and the network condition prediction information, taking the business operation failure rate of the leader tenure during the historical cluster node business operation of each constraint parameter group in the constraint set as a constraint condition, and taking the minimum number of dynamic adjustments of the leader task in the target cluster data processing cycle as the optimization goal, plan the dynamic leader tenure in the target cluster data processing cycle, generate an election strategy for the target cluster scope and use this to perform tenure interval elections using this election strategy when executing the target operation service, realize the real-time dynamic adjustment of the master node of the highly available cluster considering the influence of multiple factors, help the highly available cluster to perform tenure release and node switching of the master node before a failure occurs, and improve the operation stability and processing performance of the highly available cluster.

[0087] In a preferred embodiment, the election strategy is deployed to each dynamic access cluster node within the scope of the target cluster, driving each dynamic access cluster node to perform an interval election step during the tenure according to the election strategy in the target cluster data processing cycle when executing the target operation service, specifically including:

[0088] Merge the minimum data processing periods with the same dynamic leader tenure in two adjacent minimum data processing periods within the target cluster data processing cycle in the election strategy to generate a deployment sequence including several processing periods and the dynamic leader tenure of each processing period;

[0089] Before each processing period, distribute the dynamic leader tenure corresponding to this processing period to each dynamic access cluster node, driving each dynamic access cluster node to perform an interval election during the tenure according to the election strategy in the target cluster data processing cycle when executing the target operation service.

[0090] In this embodiment, after obtaining the deployment sequence of the dynamic leader tenure, distribute the dynamic leader tenure corresponding to each processing period to each dynamic access cluster node before each processing period, thereby realizing the dynamic leader tenure management of the cluster nodes.

[0091] Refer to Figure 2 , Figure 2 which is the structural block diagram of the embodiment of the cluster node data processing system based on the edge side of the present invention.

[0092] As Figure 2 shown, the cluster node data processing system based on the edge side proposed in the embodiment of the present invention includes:

[0093] An acquisition module 10, configured to acquire edge side cluster node resources for executing a target operation service within the scope of the target cluster; wherein, the edge side cluster node resources include node identifiers of several dynamic access cluster nodes;

[0094] A construction module 20, configured to query the hardware performance parameters of each dynamic access cluster node and the idle periods in the target cluster data processing cycle according to the node identifiers, and construct a node association information group for each dynamic access cluster node by using the hardware performance parameters and the idle periods;

[0095] A generation module 30, configured to call a network condition prediction information library based on the node identifiers, extract the network condition prediction information of each dynamic access cluster node in the target cluster data processing cycle, plan the dynamic leader tenure in the target cluster data processing cycle in consideration of the node association information group and the network condition prediction information of each dynamic access cluster node, and generate an election strategy for the target cluster scope;

[0096] An execution module 40 is configured to deploy the election policy to each dynamic access cluster node within the target cluster range, and drive each dynamic access cluster node to perform an interval election during its tenure in the target cluster data processing cycle according to the election policy when executing the target operation service.

[0097] For other embodiments or specific implementation manners of the cluster node data processing system based on the edge side of the present invention, reference may be made to the above method embodiments, which will not be elaborated herein.

[0098] It can be understood that in the description of this specification, the descriptions referring to terms such as "one embodiment", "another embodiment", "other embodiments", or "the first embodiment to the Nth embodiment" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0099] It should be noted that in this article, the term "comprising", "including" or any other variation thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or system comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or system comprising the element.

[0100] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for processing cluster node data based on the edge side, characterized in that The method includes the following steps: Obtain the edge-side cluster node resources for executing the target operation service within the target cluster range; wherein, the edge-side cluster node resources include the node identifiers of a number of dynamically accessed cluster nodes; According to the node identifiers, query the hardware performance parameters of each dynamically accessed cluster node and the idle periods during the target cluster data processing cycle, and use the hardware performance parameters and the idle periods to construct a node association information group for each dynamically accessed cluster node; Based on the node identifiers, call the network condition prediction information library, extract the network condition prediction information of each dynamically accessed cluster node during the target cluster data processing cycle, consider the node association information group and the network condition prediction information of each dynamically accessed cluster node, plan the dynamic leader tenure during the target cluster data processing cycle, and generate an election strategy for the target cluster range; Among them, the step of considering the node association information group and the network condition prediction information of each dynamically accessed cluster node, planning the dynamic leader tenure during the target cluster data processing cycle, and generating an election strategy for the target cluster range specifically includes: Extract the idle periods in the node association information group of each dynamically accessed cluster node, and generate node dynamic access distribution information for the target cluster data processing cycle according to the distribution of the idle periods in the target cluster data processing cycle; wherein, the node dynamic access distribution information includes the number of dynamically accessed cluster nodes in each minimum data processing period during the target cluster data processing cycle; Extract the hardware performance parameters in the node association information group of each dynamically accessed cluster node, and divide each dynamically accessed cluster node into several performance levels according to the service operation type in the target operation service and the hardware performance parameters of each dynamically accessed cluster node; Based on the performance levels of each dynamically accessed cluster node, calculate the average performance level of the dynamically accessed cluster nodes in each minimum data processing period of the target cluster data processing cycle in the node dynamic access distribution information, and write the average performance level into the node dynamic access distribution information to generate node hardware performance distribution information including the number of dynamically accessed cluster nodes and the average performance level in each minimum data processing period during the target cluster data processing cycle; According to the node hardware performance distribution information and the network condition prediction information, plan the dynamic leader tenure during the target cluster data processing cycle, and generate an election strategy for the target cluster range; Among them, the step of planning the dynamic leader tenure during the target cluster data processing cycle and generating an election strategy for the target cluster range according to the node hardware performance distribution information and the network condition prediction information specifically includes: Construct a constraint set by combining the number of dynamically accessed cluster nodes and the average performance level in the node hardware performance distribution information with the network condition parameters in the network condition prediction information; Query the business operation failure rate of each constraint parameter group in the query constraint set during the business operation of historical cluster nodes with different leader tenures. Taking the business operation failure rate of each minimum data processing period in the target cluster data processing cycle after being assigned to a dynamic leader tenure to be less than the failure rate allowable threshold as the constraint condition, and the sum of the number of times the dynamic leader tenures of every two adjacent minimum data processing periods are different to be minimized as the optimization objective, optimize and solve the dynamic leader tenure set assigned to each minimum data processing period in the target cluster data processing cycle; Use the dynamic leader tenure set as the election tenure interval in the target cluster data processing cycle to generate an election strategy within the target cluster scope; Deploy the election strategy to each dynamically accessed cluster node within the target cluster scope, driving each dynamically accessed cluster node to perform tenure interval elections according to the election strategy during the target cluster data processing cycle when executing the target operation business.

2. The method for processing cluster node data based on the edge side according to claim 1, wherein Steps for obtaining edge-side cluster node resources within the target cluster scope, specifically including: Receive the target operation business transmitted by the business operation terminal, and extract the business operation type in the target operation business; wherein, each business operation type is mapped to a corresponding business operation requirement; Call the edge-side cluster node database, and perform secondary screening on the edge-side cluster nodes according to the business operation requirements to obtain the edge-side cluster node resources within the target cluster scope that execute the target operation business.

3. The method for processing cluster node data based on the edge side according to claim 2, wherein Steps for calling the edge-side cluster node database and performing secondary screening on the edge-side cluster nodes according to the business operation requirements to obtain the edge-side cluster node resources within the target cluster scope that execute the target operation business, specifically including: Call the edge-side cluster node database, perform the first screening on the edge-side cluster nodes within the entire cluster scope according to the business execution hardware requirements in the business operation requirements, and perform the second screening on the edge-side cluster nodes within the entire cluster scope according to the business execution area scope requirements in the business operation requirements; Take the edge-side cluster nodes after the two screenings as the edge-side cluster node resources within the target cluster scope that execute the target operation business.

4. The method for processing cluster node data based on the edge side according to claim 1, wherein Steps for querying the hardware performance parameters of each dynamically accessed cluster node and the idle periods during the target cluster data processing cycle according to the node identifier, and using the hardware performance parameters and the idle periods to construct a node association information group for each dynamically accessed cluster node, specifically including: Extract the comparison table of edge-side cluster nodes and hardware performance parameters and the business operation queue of each edge-side cluster node in the edge-side cluster node database; Query the hardware performance parameters of each dynamically accessed cluster node in the comparison table of edge-side cluster nodes and hardware performance parameters according to the node identifier; extract the business operation queue of each dynamically accessed cluster node according to the node identifier, and determine the idle periods of each dynamically accessed cluster node during the target cluster data processing cycle; Construct a node association information group for each dynamic access cluster node by using the hardware performance parameters and the idle period.

5. The method for processing cluster node data based on the edge side according to claim 1, wherein Based on the node identifier, the step of invoking the network condition prediction information database to extract the network condition prediction information of each dynamic access cluster node in the target cluster data processing cycle specifically includes: Invoke the network condition prediction information database; wherein, the network condition prediction information database records the network condition prediction information of each dynamic access cluster node. Based on the node identifier, match the network condition prediction information of each dynamic access cluster node in the target cluster data processing cycle in the network condition prediction information database.

6. The method for processing cluster node data based on the edge side according to claim 5, wherein The construction of the network condition prediction information database specifically includes: Obtain the network condition parameters of each dynamic access cluster node in a number of historical cluster data processing cycles within the target cluster range; wherein, the network condition parameters include any one of data throughput, packet loss rate, or latency. Calculate the average value of the network condition parameters of a number of dynamic access cluster nodes in each minimum data processing period in each historical cluster data processing cycle, and construct the training samples and prediction samples of each historical cluster data processing cycle based on the average value of the network condition parameters and the timestamp in all minimum data processing periods in each historical cluster data processing cycle. Use the training samples to train a pre-constructed convolutional neural network model, and use the prediction samples and the network condition prediction model obtained after training to predict the network condition prediction information of each dynamic access cluster node in the target cluster data processing cycle.

7. The method for processing cluster node data based on the edge side according to claim 1, wherein Deploy the election policy to each dynamic access cluster node within the target cluster range, and drive each dynamic access cluster node to execute the tenure interval election step according to the election policy during the execution of the target operation service in the target cluster data processing cycle, specifically including: Merge the minimum data processing periods with the same dynamic leader tenure in two adjacent minimum data processing periods within the target cluster data processing cycle in the election policy to generate a deployment sequence including a number of processing periods and the dynamic leader tenure of each processing period. Distribute the dynamic leader tenure corresponding to each processing period to each dynamic access cluster node before each processing period, and drive each dynamic access cluster node to execute the tenure interval election according to the election policy during the execution of the target operation service in the target cluster data processing cycle.

8. An edge-side-based cluster node data processing system, characterized in that, Include: An acquisition module for acquiring the edge-side cluster node resources for executing the target operation service within the target cluster range; wherein, the edge-side cluster node resources include the node identifiers of a number of dynamic access cluster nodes. A construction module for querying the hardware performance parameters and the idle period of each dynamic access cluster node in the target cluster data processing cycle according to the node identifier, and constructing a node association information group for each dynamic access cluster node by using the hardware performance parameters and the idle period. A generation module, configured to call a network status prediction information library based on the node identifier, extract the network status prediction information of each dynamically accessed cluster node in the target cluster data processing cycle, consider the node association information group and the network status prediction information of each dynamically accessed cluster node, plan the dynamic leader tenure in the target cluster data processing cycle, and generate an election strategy for the target cluster scope; Among them, considering the node association information group and the network status prediction information of each dynamically accessed cluster node, planning the dynamic leader tenure in the target cluster data processing cycle, and generating an election strategy for the target cluster scope, specifically includes: Extract the idle periods in the node association information group of each dynamically accessed cluster node, and generate node dynamic access distribution information for the target cluster data processing cycle according to the distribution of the idle periods in the target cluster data processing cycle; wherein, the node dynamic access distribution information includes the number of dynamically accessed cluster nodes in each minimum data processing period in the target cluster data processing cycle; Extract the hardware performance parameters in the node association information group of each dynamically accessed cluster node, and divide each dynamically accessed cluster node into several performance levels according to the service operation type in the target operation service and the hardware performance parameters of each dynamically accessed cluster node; Based on the performance levels of each dynamically accessed cluster node, calculate the average performance level of the dynamically accessed cluster nodes in each minimum data processing period in the target cluster data processing cycle in the node dynamic access distribution information, write the average performance level into the node dynamic access distribution information, and generate node hardware performance distribution information including the number of dynamically accessed cluster nodes and the average performance level in each minimum data processing period in the target cluster data processing cycle; According to the node hardware performance distribution information and the network status prediction information, plan the dynamic leader tenure in the target cluster data processing cycle, and generate an election strategy for the target cluster scope; Among them, according to the node hardware performance distribution information and the network status prediction information, planning the dynamic leader tenure in the target cluster data processing cycle, and generating an election strategy for the target cluster scope, specifically includes: Construct a constraint set with the number of dynamically accessed cluster nodes and the average performance level in the node hardware performance distribution information and the network status parameters in the network status prediction information; Query the service operation failure rates of different leader tenures in the historical cluster node service operation for each constraint parameter group in the constraint set. With the constraint that the service operation failure rate of each minimum data processing period in the target cluster data processing cycle after being assigned to the dynamic leader tenure is less than the failure rate allowable threshold, and the sum of the number of times the dynamic leader tenures of every two adjacent minimum data processing periods are different is minimized as the optimization objective, optimize and solve the dynamic leader tenure set assigned to each minimum data processing period in the target cluster data processing cycle; Generate an election strategy for the target cluster scope by using the dynamic leader tenure set as the election tenure interval in the target cluster data processing cycle; An execution module is used to deploy the election strategy to each dynamically accessed cluster node within the target cluster scope, and drive each dynamically accessed cluster node to perform tenure interval elections according to the election strategy during the target cluster data processing cycle when executing the target operation service.

Images