Multi-node intelligent scheduling service segmentation cooperative control method and system

Abstract

The invention discloses a multi-node intelligent scheduling service segmentation cooperative control method and system, and relates to the technical field of the Internet. By deploying a heartbeat detection mechanism, heartbeat signals are sent and monitored regularly, and abnormal signals are identified; marking the node agent of the abnormal signal as a fault node, obtaining a resource index of the fault node, and identifying a fault type; sending a fault notification to other nodes according to the fault type, and judging whether to elect a new master control node agent or not according to the fault notification; the master control node reads the service execution information of the fault node, splits the uncompleted service into sub-services, obtains the remaining available resources of the available nodes and the resource requirements of the sub-services, and allocates the remaining available resources to the available nodes according to the resource requirements of the sub-services; the fault node is repaired, and if the repair is successful, the master control node returns the service; if the fault node cannot be successfully repaired, performing service redistribution and replacement on the fault node; and processing and load balancing of the fault node are realized.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method and system for collaborative control of service segments for multi-node intelligent scheduling. Background Art

[0002] In modern large-scale distributed systems, such as cloud computing, the Internet of Things, and large-scale industrial control systems, there are problems involving intelligent scheduling and control of multiple nodes. These nodes may be distributed in different geographical locations or in different subsystems, each handling different business tasks. In order to improve the overall efficiency, reliability, and response speed of the system, these nodes need to be intelligently scheduled and coordinated at the business level.

[0003] A Chinese patent with publication number CN110430071A discloses a service node fault self-healing method, device, computer equipment and storage medium. The method includes obtaining monitoring data of service nodes, load balancing equipment and service distribution to form data to be analyzed; performing intelligent analysis based on the data to be analyzed to determine whether there is a fault problem; if so, obtaining the fault point corresponding to the fault problem and sending corresponding alarm information; performing self-healing repair on the fault point to obtain the repair result; and confirming the repair result.

[0004] Traditional methods for handling faulty nodes usually adopt a simple business transfer strategy when a node failure occurs. They are unable to make flexible decisions based on the characteristics of the failure and lack a mechanism to dynamically decide whether the master control node needs to be replaced based on the type of failure. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems in the related art to a certain extent. To this end, one purpose of the present application is to propose a service segmentation collaborative control method and system for multi-node intelligent scheduling, which realizes the processing of faulty nodes and the allocation of service segments.

[0006] One aspect of the present application provides a service segmentation collaborative control method for multi-node intelligent scheduling, including:

[0007] Step S100: deploy a heartbeat detection mechanism in each node agent, regularly send heartbeat signals to other node agents and monitor the heartbeat signals of other node agents to identify abnormal signals;

[0008] Step S200: marking the node agent that sends the abnormal signal as a faulty node agent, obtaining the resource index of the faulty node agent, and identifying the fault type of the faulty node agent based on the resource index classification model;

[0009] Step S300: The faulty node agent sends a fault notification to other node agents according to the fault type, and all node agents determine whether a new master node agent needs to be elected based on the fault notification. If necessary, a new master node agent is elected.

[0010] Step S400: The master control node agent reads the service execution information of the faulty node agent, splits the unfinished service into m sub-services, obtains the remaining available resources of the available node agent and the resource requirements of the sub-services, and allocates them to the available node agents according to the resource requirements of the sub-services;

[0011] Step S500: Repair the faulty node agent. If the repair is successful, the master node agent will return the service. If the repair is unsuccessful, the service of the faulty node agent will be redistributed and replaced.

[0012] Specifically, the heartbeat detection mechanism is deployed in each node agent, heartbeat signals are regularly sent to other node agents and heartbeat signals of other node agents are monitored, and the specific method for identifying abnormal signals is:

[0013] Step S110: define the node information of the node agent, including node ID, node role, resource information, node status, and service execution information. The node role includes a master node agent and a non-master node agent. The node status includes normal, faulty, and unavailable. Register the node information of the node agent into the global node status table.

[0014] Step S120: deploy a heartbeat detection mechanism in each node agent, the heartbeat detection mechanism includes a heartbeat transmitter and a heartbeat listener, the heartbeat transmitter sends a heartbeat signal to all other node agents every a1 unit time, the heartbeat listener continuously listens to the heartbeat signals sent by other node agents, and records the timestamp of receiving the most recent heartbeat signal of each node agent, the heartbeat signal includes the node ID and timestamp of the sender;

[0015] Step S130: The heartbeat listener checks the timestamp of each heartbeat signal every a2 unit time. When the time difference between the timestamp of the heartbeat signal of the node agent and the current timestamp is greater than the time difference threshold, the heartbeat signal is marked as an abnormal signal.

[0016] Specifically, the specific method of marking the node agent that sends the abnormal signal as a faulty node agent, obtaining the resource index of the faulty node agent, and identifying the fault type of the faulty node agent based on the resource index classification model is as follows:

[0017] Step S210: When the heartbeat listener detects that the heartbeat signal of the node agent is an abnormal signal for a3 consecutive times, a fault alarm is triggered, the node state of the node agent that sends the abnormal signal is obtained, and its node state is changed to fault in the node state table, and the node agent is marked as a faulty node agent;

[0018] Step S220: Collect abnormal resource indicators and fault features of historical fault node agents, extract feature vectors, construct n1 training samples, and train to obtain a resource indicator classification model containing K categories;

[0019] The specific training process of the resource indicator classification model is as follows:

[0020] Step S221: Collect abnormal resource indicators of historical fault node agents to form an abnormal resource indicator time series;

[0021] Step S222: define the length of the time window, extract features of the abnormal resource indicators in each time window to obtain feature vectors, and use the sliding window technology to divide the abnormal resource indicator time series into n1 training samples;

[0022] Step S223: setting the number of target cluster categories K, and randomly selecting K training samples as initial cluster centers;

[0023] Step S224: for each feature vector of a training sample, calculate the Euclidean distance between it and the feature vector of each cluster center, and assign the training sample to the category to which the cluster center with the closest distance belongs;

[0024] Step S225: recalculate the mean vector of the feature vectors of all training samples in each category, and use the mean vector as a new cluster center;

[0025] Step S226: repeating steps S224 to S225 until the training samples in each category no longer change, thereby obtaining a trained resource indicator classification model;

[0026] Step S230: Count the mean vector μ of the training samples in each category k and the covariance matrix C k ;

[0027] Step S240: manually marking corresponding fault labels according to the fault features of the cluster centers in each category, wherein the fault labels include but are not limited to node downtime, resource exhaustion, software failure, load fluctuation, network jitter and security incidents;

[0028] Step S250: obtaining resource indicators and feature vectors of the faulty node agent in real time, calculating the Mahalanobis distance between the resource indicators of the faulty node agent and each category, and obtaining a similarity measure between the resource indicators of the faulty node agent and each category based on the Mahalanobis distance;

[0029] Step S260: Based on the Mahalanobis distance and covariance matrix C between the resource index of the faulty node agent and each category k , calculate the abnormal probability that the resource indicators of the faulty node agent belong to each category;

[0030] Step S270: performing weighted summation of the similarity measure and the abnormal probability to obtain the abnormal diagnosis score between the resource index of the faulty node agent and each category, and obtaining the fault label of the category with the smallest abnormal diagnosis score between the resource index of the faulty node agent, wherein the fault label is the fault type of the faulty node agent;

[0031] The calculation formula of the abnormal diagnosis score is: k =α1×q k +β1×p k , where q k represents the similarity measure between the resource index of the faulty node agent and the kth category, p k represents the abnormal probability that the resource indicator of the faulty node agent belongs to the kth category;

[0032] The calculation formula for the similarity measure between the resource index of the faulty node agent and the kth category is: Among them, d k represents the Mahalanobis distance between the resource index of the faulty node agent and the cluster center of the kth category;

[0033] The calculation formula of the Mahalanobis distance between the resource index of the faulty node agent and the cluster center of the kth category is: Where x represents the characteristic vector of resource indicators of the faulty node agent, μ k represents the mean vector of the training samples in the kth category, T is the transposed symbol, represents the inverse matrix of the covariance matrix of the training samples in the kth category;

[0034] The calculation formula of the abnormal probability of the resource index of the faulty node agent in the kth category is: Among them, n represents the characteristic dimension of the feature vector, |C k | represents the covariance matrix C k The determinant of , exp(·) represents the exponential function.

[0035] Specifically, the fault node agent sends a fault notification to other node agents according to the fault type, and all node agents determine whether a new master node agent needs to be elected according to the fault notification. If necessary, the specific method for electing a new master node agent is as follows:

[0036] Step S310: The faulty node agent constructs a fault notification according to its fault type, node role and timestamp, and sends the fault notification to other node agents;

[0037] Step S320: After receiving the fault notification, other node agents mark the node status of the faulty node agent as unavailable in the node status table;

[0038] Step S330: other node agents determine whether it is necessary to elect a new master node agent according to the node role in the fault notification. If the faulty node agent is a non-master node agent, it is not necessary to elect a new master node agent. If the faulty node agent is a master node agent and the fault type is any one of node downtime, resource exhaustion, software failure and security incident, a new master node agent is elected. Otherwise, it is not necessary to elect a new master node agent.

[0039] Step S340: For each node agent whose node status is normal, count the stability index S of the heartbeat signal sent by it in the past time T1. success , get the number of nodes n that each node agent is directly connected to in the network topology neib and the resource index y in the past time T1, and calculate the average resource index based on the resource index in the past time T1 Calculate the comprehensive score of each node agent based on stability index, number of nodes and average resource index;

[0040] Step S350: Each node agent packages its own comprehensive score and node ID into a voting message and sends it to all other non-faulty node agents. Each node agent is sorted in descending order based on the comprehensive score, and the node agent with the highest comprehensive score is used as the master node agent.

[0041] Specifically, the master control node agent reads the service execution information of the faulty node agent, splits the unfinished service into m sub-services, obtains the remaining available resources of the available node agent and the resource requirements of the sub-services, and allocates the resources to the available node agents according to the resource requirements of the sub-services. The specific method is as follows:

[0042] Step S410: the master control node agent reads the service execution information of the faulty node agent, wherein the service execution information includes: service logic, service priority and service execution progress;

[0043] Step S420: the master control node agent sorts the unfinished business in descending order of business priority, and divides the unfinished business into m sub-businesses based on the priority order and business logic;

[0044] Step S430: Obtain the resource requirements of each sub-service, and the master node agent forms a list of available node agents based on all node agents in normal state in the current node state table;

[0045] Step S440: The master node agent traverses the unfinished sub-services, and for each sub-service, selects the available node agent with the highest remaining available resources as the node agent to execute the sub-service;

[0046] Step S450: If the remaining available resources of all available node agents are insufficient to process the sub-service, the sub-service is further split until an available node agent that meets the resource requirements of the sub-service is found;

[0047] Step S460: The master control node agent allocates the sub-service to the selected available node agent.

[0048] Specifically, the faulty node agent is repaired. If the repair is successful, the master node agent returns the service. If the repair is unsuccessful, the specific method for reallocating and replacing the service of the faulty node agent is as follows:

[0049] Step S510: the master control node agent repairs the faulty node agent. During the repair process, the master control node agent monitors the node status of the faulty node agent in real time and sets a first time length t1 as a time limit for the repair;

[0050] Step S520: If the repair is successful within the first time duration t1, the master control node agent returns the transferred service to the repaired node agent and updates the node status table;

[0051] Step S530: If the fault cannot be repaired successfully within the first time period t1, the master control node agent distributes the remaining services of the faulty node agent to other available node agents;

[0052] Step S540: Set a second time duration t2 as the time limit for node replacement. If the faulty node agent cannot be repaired successfully within the second time duration t2, the master node agent sends a replacement notification to the administrator, informing the administrator to replace the faulty node agent with a new node agent.

[0053] One aspect of the present application provides a business segmentation collaborative control system for multi-node intelligent scheduling, including:

[0054] The abnormal signal identification module is used to deploy a heartbeat detection mechanism in each node agent, regularly send heartbeat signals to other node agents and monitor the heartbeat signals of other node agents to identify abnormal signals;

[0055] A fault type identification module is used to mark the node agent that sends abnormal signals as a faulty node agent, obtain resource indicators of the faulty node agent, and identify the fault type of the faulty node agent based on a resource indicator classification model;

[0056] The master node election module is used for the fault node agent to send fault notifications to other node agents according to the fault type. All node agents determine whether a new master node agent needs to be elected based on the fault notification. If necessary, a new master node agent is elected.

[0057] The node service allocation module is used for the master node agent to read the service execution information of the faulty node agent, split the unfinished service into m sub-services, obtain the remaining available resources of the available node agent and the resource requirements of the sub-services, and allocate them to the available node agents according to the resource requirements of the sub-services;

[0058] The fault node processing module is used to repair the fault node agent. If the repair is successful, the master node agent will return the service; if the repair is unsuccessful, the service of the fault node agent will be redistributed and replaced.

[0059] One aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the service segment collaborative control method for multi-node intelligent scheduling are implemented.

[0060] One aspect of the present application provides a readable storage medium storing a computer program, wherein the computer program is suitable for loading by a processor to execute the steps in a service segment collaborative control method for multi-node intelligent scheduling.

[0061] Compared with the prior art, the service segmentation collaborative control method and system for multi-node intelligent scheduling proposed in this application have the following advantages:

[0062] This application proposes a fine-grained business splitting and resource matching method, which can flexibly split and allocate the unfinished business of the faulty node to other available nodes, thereby achieving continuous execution of the business and minimizing the business loss caused by the failure.

[0063] This application proposes a mechanism for node switching based on the fault type. For the failure of the master node, switching is triggered only in the case of a serious fault, and no switching is performed for relatively minor faults. This strategy of treating the fault differently based on its severity helps to strike a balance between system availability and switching costs.

[0064] The dynamic decision-making mechanism proposed in this application avoids unnecessary and frequent master node switching, reduces system jitter and service interruption caused by switching, and makes switching decisions more intelligent, adaptively choosing whether to switch and which node to switch to based on the fault scenario, rather than simply switching to the backup node according to a fixed process.

[0065] This application transforms from passive fault response to active risk control, timely evaluates and adjusts according to the severity and scope of the fault, and improves the system's ability to cope with complex faults. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A method flow chart of the service segmentation collaborative control method for multi-node intelligent scheduling provided by this application;

[0067] Figure 2 A flow chart of a method for identifying the fault type of a faulty node agent provided in this application;

[0068] Figure 3 A flow chart of the dynamic decision-making method for master node switching provided in this application;

[0069] Figure 4 Functional module diagram of the business segmentation collaborative control system for multi-node intelligent scheduling provided by this application;

[0070] Figure 5 A schematic diagram of the structure of an electronic device provided in this application;

[0071] Figure 6 It is a schematic diagram of the structure of a readable storage medium provided by this application. DETAILED DESCRIPTION

[0072] In order to better understand the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application and do not limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0073] In the accompanying drawings, the size, dimensions and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are for illustration only and are not strictly drawn to scale. As used herein, the terms "substantially", "approximately" and similar terms are used as terms of approximation, not as terms of degree, and are intended to illustrate the inherent deviations in measurements or calculations that will be recognized by those of ordinary skill in the art. In addition, in this application, the order in which the steps are described does not necessarily represent the order in which these processes occur in actual operation, unless otherwise specified or can be derived from the context.

[0074] It should also be understood that expressions such as "including", "comprising", "having", "containing" and / or "comprising" are open rather than closed expressions in this specification, which indicate the presence of the stated features, elements and / or components, but do not exclude the presence of one or more other features, elements, components and / or combinations thereof. In addition, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features rather than just the individual elements in the list. In addition, when describing embodiments of the present application, "may" is used to mean "one or more embodiments of the present application". And, the term "exemplary" is intended to refer to an example or illustration.

[0075] Unless otherwise specified, all words (including engineering terms and scientific and technological terms) used in this article have the same meaning as those commonly understood by ordinary technicians in the field to which this application belongs. It should also be understood that, unless clearly stated in this application, words defined in common dictionaries should be interpreted as having the same meaning as their meaning in the context of the relevant technology, and should not be interpreted in an idealized or overly formal sense.

[0076] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0077] Example 1

[0078] like Figure 1 As shown, the service segmentation collaborative control method for multi-node intelligent scheduling provided by this application includes:

[0079] Step S100: deploy a heartbeat detection mechanism in each node agent, regularly send heartbeat signals to other node agents and monitor the heartbeat signals of other node agents to identify abnormal signals.

[0080] The heartbeat detection mechanism is deployed in each node agent, and heartbeat signals are sent to other node agents regularly and the heartbeat signals of other node agents are monitored. The specific method for identifying abnormal signals is as follows:

[0081] Step S110: define the node information of the node agent, including node ID, node role, resource information, node status, and service execution information. The node role includes a master node agent and a non-master node agent. The node status includes normal, faulty, and unavailable. Register the node information of the node agent into the global node status table.

[0082] The master node agent is a special node in the system responsible for coordinating and managing other node agents. It has higher authority and a more comprehensive system view.

[0083] The relationship between the master node agent and other node agents is a coordinator-coordinator relationship. The master node agent is responsible for managing and scheduling cluster resources and services, while other node agents register node information into the global node status table.

[0084] Centralized state management provides a global perspective for the master node, making it easier to understand the operating status of the entire cluster and provide a basis for making global decisions. The non-master node agents execute specific business logic according to the instructions of the master node agent.

[0085] Step S120: deploy a heartbeat detection mechanism in each node agent, the heartbeat detection mechanism includes a heartbeat transmitter and a heartbeat listener, the heartbeat transmitter sends a heartbeat signal to all other node agents every a1 unit time, the heartbeat listener continuously listens to the heartbeat signals sent by other node agents, and records the timestamp of receiving the most recent heartbeat signal of each node agent, the heartbeat signal includes the node ID and timestamp of the sender;

[0086] The timestamp refers to the timestamp when the sender's node agent sends the heartbeat signal;

[0087] Step S130: The heartbeat listener checks the timestamp of each heartbeat signal every a2 unit time. When the time difference between the timestamp of the heartbeat signal of the node agent and the current timestamp is greater than the time difference threshold, the heartbeat signal is marked as an abnormal signal.

[0088] The unit duration is 1 second;

[0089] The values ​​of a1, a2 and the time difference threshold are set by those skilled in the art based on experience and actual needs;

[0090] Preferably, a1 is equal to 5, a2 is equal to 10, and the time difference threshold is equal to 30 seconds.

[0091] The heartbeat detection mechanism proposed in step S100 is the basis and prerequisite for solving the subsequent dynamic decision-making of the master node switching. By deploying a heartbeat transmitter and a heartbeat listener in each node agent and regularly exchanging heartbeat signals, mutual perception and synchronization of the status between nodes are achieved.

[0092] Step S200: Mark the node agent that sends the abnormal signal as a faulty node agent, obtain the resource index of the faulty node agent, and identify the fault type of the faulty node agent based on the resource index classification model.

[0093] like Figure 2 As shown, it is a flow chart of the method for identifying the fault type of a faulty node intelligent agent provided by the present application.

[0094] The specific method of marking the node agent that sends the abnormal signal as a faulty node agent, obtaining the resource index of the faulty node agent, and identifying the fault type of the faulty node agent based on the resource index classification model is as follows:

[0095] Step S210: When the heartbeat listener detects that the heartbeat signal of the node agent is an abnormal signal for a3 consecutive times, a fault alarm is triggered, the node state of the node agent that sends the abnormal signal is obtained, and its node state is changed to fault in the node state table, and the node agent is marked as a faulty node agent;

[0096] The value of a3 is set by those skilled in the art based on experience and actual needs; preferably, a3 is equal to 3;

[0097] Step S220: Collect abnormal resource indicators and fault features of historical fault node agents, extract feature vectors, construct n1 training samples, and train to obtain a resource indicator classification model containing K categories;

[0098] The resource indicators are the resources used by the node agent;

[0099] The specific training process of the resource indicator classification model is as follows:

[0100] Step S221: Collect abnormal resource indicators of historical fault node agents to form an abnormal resource indicator time series;

[0101] The fault types of abnormal resource indicators of the historical fault node agent include but are not limited to six fault types, namely node downtime, resource exhaustion, software failure, load fluctuation, network jitter and security incident;

[0102] Step S222: define the length of the time window, extract features of the abnormal resource indicators in each time window to obtain feature vectors, and use the sliding window technology to divide the abnormal resource indicator time series into n1 training samples;

[0103] The feature vector is an n-dimensional feature vector;

[0104] Step S223: setting the number of target cluster categories K, and randomly selecting K training samples as initial cluster centers;

[0105] Step S224: for each feature vector of a training sample, calculate the Euclidean distance between it and the feature vector of each cluster center, and assign the training sample to the category to which the cluster center with the closest distance belongs;

[0106] Step S225: recalculate the mean vector of the feature vectors of all training samples in each category, and use the mean vector as a new cluster center;

[0107] Step S226: Repeat steps S224 to S225 until the training samples in each category no longer change, thereby obtaining a trained resource indicator classification model.

[0108] Step S230: Count the mean vector μ of the training samples in each category k and the covariance matrix C k ;

[0109] Step S240: manually marking corresponding fault labels according to the fault features of the cluster centers in each category, wherein the fault labels include but are not limited to node downtime, resource exhaustion, software failure, load fluctuation, network jitter and security incidents;

[0110] Fault labels include but are not limited to the above six fault labels, which are common fault types. When the present application is applied to actual situations, the fault labels of each clustering category are annotated by technicians in this field according to the characteristics of the samples in that category.

[0111] For example, suppose two categories are obtained after clustering, and the fault characteristics of the cluster center of category 1 are CPU utilization 90% and memory utilization 30%, and the fault characteristics of the cluster center of category 2 are CPU utilization 30% and memory utilization 90%. Then, according to the fault characteristics of category 1, the fault label of category 1 is marked as CPU resource exhaustion, and according to the fault characteristics of category 2, the fault label of category 2 is marked as memory resource exhaustion.

[0112] Step S250: obtaining resource indicators and feature vectors of the faulty node agent in real time, calculating the Mahalanobis distance between the resource indicators of the faulty node agent and each category, and obtaining a similarity measure between the resource indicators of the faulty node agent and each category based on the Mahalanobis distance;

[0113] Step S260: Based on the Mahalanobis distance and covariance matrix C between the resource index of the faulty node agent and each categoryk , calculate the abnormal probability that the resource indicators of the faulty node agent belong to each category;

[0114] Step S270: Perform a weighted summation of the similarity measure and the abnormality probability to obtain the abnormality diagnosis score between the resource index of the faulty node intelligent body and each category, and obtain the fault label of the category with the smallest abnormality diagnosis score between the resource index of the faulty node intelligent body, and the fault label is the fault type of the faulty node intelligent body.

[0115] The calculation formula of the abnormal diagnosis score is: k =α1×q k +β1×p k , where q k represents the similarity measure between the resource index of the faulty node agent and the kth category, p k represents the abnormal probability that the resource indicator of the faulty node agent belongs to the kth category, α1 and β1 are the weight coefficients of the similarity metric and the abnormal probability, respectively, which are set by technicians in this field based on experience;

[0116] The calculation formula for the similarity measure between the resource index of the faulty node agent and the kth category is: Among them, d k represents the Mahalanobis distance between the resource index of the faulty node agent and the cluster center of the kth category;

[0117] The calculation formula of the Mahalanobis distance between the resource index of the faulty node agent and the cluster center of the kth category is: Where x represents the characteristic vector of resource indicators of the faulty node agent, μ k represents the mean vector of the training samples in the kth category, T is the transposed symbol, represents the inverse matrix of the covariance matrix of the training samples in the kth category;

[0118] The calculation formula of the abnormal probability of the resource index of the faulty node agent in the kth category is: Among them, n represents the characteristic dimension of the feature vector, C k Denotes the covariance matrix C k The determinant of , exp(·) represents the exponential function;

[0119] The smaller the Mahalanobis distance, the larger the similarity measure, indicating that the resource indicators of the faulty node agent are more similar to the kth category;

[0120] When the abnormal probability that the resource indicator of the faulty node agent belongs to the kth category is greater, it means that the resource indicator of the faulty node agent is more similar to the kth category.

[0121] For the establishment of a dynamic decision-making mechanism for the switching of the master control node, the resource indicator classification model established in the above steps is the basis for the establishment of the decision-making mechanism. One of the decision-making influencing factors of the decision-making mechanism is the fault type, and the fault type is obtained based on the resource indicator classification model; by applying the resource indicators of the node with actual fault to the trained resource indicator classification model, the category to which the faulty node belongs is determined according to the abnormal diagnosis score between the resource indicators of the faulty node and each category, thereby determining its fault type; the above steps introduce intelligent diagnosis technology based on clustering algorithms, and by automatically generating fault type diagnosis rules, rapid diagnosis and accurate positioning of faults are achieved, greatly improving the efficiency and accuracy of diagnosis.

[0122] Step S300: The faulty node agent sends a fault notification to other node agents according to the fault type. All node agents determine whether a new master node agent needs to be elected based on the fault notification. If necessary, a new master node agent is elected.

[0123] The fault node agent sends a fault notification to other node agents according to the fault type, and all node agents determine whether a new master node agent needs to be elected based on the fault notification. If necessary, the specific method for electing a new master node agent is as follows:

[0124] Step S310: The faulty node agent constructs a fault notification according to its fault type, node role and timestamp, and sends the fault notification to other node agents;

[0125] Step S320: After receiving the fault notification, other node agents mark the node status of the faulty node agent as unavailable in the node status table;

[0126] Step S330: other node agents determine whether it is necessary to elect a new master node agent according to the node role in the fault notification. If the faulty node agent is a non-master node agent, it is not necessary to elect a new master node agent. If the faulty node agent is a master node agent and the fault type is any one of node downtime, resource exhaustion, software failure and security incident, a new master node agent is elected. Otherwise, it is not necessary to elect a new master node agent.

[0127] Step S340: For each node agent whose node status is normal, count the stability index S of the heartbeat signal sent by it in the past time T1. success , get the number of nodes n that each node agent is directly connected to in the network topology neib and the resource index y in the past time T1, and calculate the average resource index based on the resource index in the past time T1 Calculate the comprehensive score of each node agent based on stability index, number of nodes and average resource index;

[0128] The calculation formula of the stability index is: Among them, count represents the number of times the heartbeat signal is successfully sent in the past time T1. Indicates the number of heartbeat signals theoretically sent successfully within the past time T1;

[0129] The calculation formula of the average resource index is: Among them, Δt is the sampling interval of resource indicators;

[0130] Preferably, Δt is equal to 5 seconds;

[0131] The calculation formula of the comprehensive score is: Among them, α S , α y are the weight coefficients of stability index, number of nodes and average resource index respectively;

[0132] The weight coefficients of the stability index, the number of nodes and the average resource index are set by those skilled in the art according to actual needs and experience.

[0133] Step S350: Each node agent packages its own comprehensive score and node ID into a voting message and sends it to all other non-faulty node agents. Each node agent is sorted in descending order based on the comprehensive score, and the node agent with the highest comprehensive score is used as the master node agent.

[0134] Figure 3 A flow chart of the dynamic decision-making method for master node switching provided in this application.

[0135] The above steps dynamically decide whether a new master node needs to be elected based on the role of the faulty node (master / non-master) and the type of fault (node ​​downtime, resource exhaustion, software failure, load fluctuation, network jitter, and security incidents, etc.), which can achieve more refined fault handling. For non-master node failures, there is no need to switch the master node, reducing unnecessary election overhead; the master node switch is triggered only when a serious fault occurs in the master node, avoiding frequent master node switching; when a new master node needs to be elected, the comprehensive score is calculated by comprehensively considering the stability, connectivity, and resource indicators of the node, and a better new master node is selected to ensure that the new master node has sufficient capabilities to take over the work of the original master node.

[0136] Step S400: The master control node agent reads the service execution information of the faulty node agent, splits the unfinished service into m sub-services, obtains the remaining available resources of the available node agents and the resource requirements of the sub-services, and allocates them to the available node agents according to the resource requirements of the sub-services.

[0137] The master control node agent reads the service execution information of the faulty node agent, splits the unfinished service into m sub-services, obtains the remaining available resources of the available node agent and the resource requirements of the sub-services, and allocates the resources to the available node agents according to the resource requirements of the sub-services. The specific method is as follows:

[0138] Step S410: the master control node agent reads the service execution information of the faulty node agent, wherein the service execution information includes: service logic, service priority and service execution progress;

[0139] Step S420: the master control node agent sorts the unfinished business in descending order of business priority, and divides the unfinished business into m sub-businesses based on the priority order and business logic;

[0140] For a data processing business, it can be divided into m sub-businesses according to the data partition and processing stage; for a mathematical calculation business, it can be divided into m sub-businesses according to the calculation sub-problems;

[0141] Step S430: Obtain the resource requirements of each sub-service, and the master node agent forms a list of available node agents based on all node agents in normal state in the current node state table;

[0142] Step S440: The master node agent traverses the unfinished sub-services, and for each sub-service, selects the available node agent with the highest remaining available resources as the node agent to execute the sub-service;

[0143] Step S450: If the remaining available resources of all available node agents are insufficient to process the sub-service, the sub-service is further split until an available node agent that meets the resource requirements of the sub-service is found;

[0144] Step S460: The master control node agent allocates the sub-service to the selected available node agent.

[0145] Splitting unfinished business into multiple sub-businesses according to business logic and priority can achieve more fine-grained task allocation. This split can better match the resource capabilities of available nodes, improve resource utilization efficiency, and speed up business recovery progress.

[0146] If the existing available node resources are insufficient, the sub-businesses can be further split until a suitable node is matched. This recursive splitting method can cope with complex resource constraint scenarios.

[0147] When the original master control node fails, the new master control node can seamlessly take over the business orchestration responsibilities of the original master control node to ensure the continuity of business orchestration work.

[0148] Step S500: Repair the faulty node agent. If the repair is successful, the master node agent will return the service. If the repair is unsuccessful, the service of the faulty node agent will be redistributed and replaced.

[0149] The faulty node agent is repaired. If the repair is successful, the master node agent returns the service. If the repair is unsuccessful, the specific method for reallocating and replacing the service of the faulty node agent is as follows:

[0150] Step S510: the master control node agent repairs the faulty node agent. During the repair process, the master control node agent monitors the node status of the faulty node agent in real time and sets a first time length t1 as a time limit for the repair;

[0151] Step S520: If the repair is successful within the first time duration t1, the master control node agent returns the transferred service to the repaired node agent and updates the node status table;

[0152] Step S530: If the fault cannot be repaired successfully within the first time period t1, the master control node agent distributes the remaining services of the faulty node agent to other available node agents;

[0153] When the master node agent distributes the remaining services of the faulty node agent to other available node agents, it comprehensively considers the service priority and node resource indicators, and preferentially distributes the services to the node agents with normal resource indicators and closer network distance;

[0154] Step S540: Set a second time duration t2 as the time limit for node replacement. If the faulty node agent cannot be repaired successfully within the second time duration t2, the master node agent sends a replacement notification to the administrator, informing the administrator to replace the faulty node agent with a new node agent.

[0155] The first duration t1 and the second duration t2 are set by those skilled in the art according to the fault type and actual needs; illustratively, t1 can be set to 30 minutes to 2 hours, and t2 can be set to 2 hours to 24 hours.

[0156] In this step, by returning the services of the repairable nodes, the service division of each node can be quickly restored after the master control node is switched, thereby reducing the impact of the master control node switching on the services.

[0157] Timely elimination of irreparable faulty nodes and introduction of new nodes as replacements ensures that the system always has sufficient computing resources, providing resource guarantees for master node switching and enabling new master nodes to exert their scheduling potential.

[0158] Example 2

[0159] like Figure 4 As shown, the business segmentation collaborative control method system for multi-node intelligent scheduling provided by this application includes:

[0160] The abnormal signal identification module is used to deploy a heartbeat detection mechanism in each node agent, regularly send heartbeat signals to other node agents and monitor the heartbeat signals of other node agents to identify abnormal signals;

[0161] A fault type identification module is used to mark the node agent that sends abnormal signals as a faulty node agent, obtain resource indicators of the faulty node agent, and identify the fault type of the faulty node agent based on a resource indicator classification model;

[0162] The master node election module is used for the fault node agent to send fault notifications to other node agents according to the fault type. All node agents determine whether a new master node agent needs to be elected based on the fault notification. If necessary, a new master node agent is elected.

[0163] The node service allocation module is used for the master node agent to read the service execution information of the faulty node agent, split the unfinished service into m sub-services, obtain the remaining available resources of the available node agent and the resource requirements of the sub-services, and allocate them to the available node agents according to the resource requirements of the sub-services;

[0164] The fault node processing module is used to repair the fault node agent. If the repair is successful, the master node agent will return the service; if the repair is unsuccessful, the service of the fault node agent will be redistributed and replaced.

[0165] Example 3

[0166] Figure 5 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 5As shown, according to another aspect of the present application, an electronic device is also provided. The electronic device may include one or more processors and one or more memories. The memory stores a computer readable code, and when the computer readable code is executed by one or more processors, the service segmentation collaborative control method for multi-node intelligent scheduling as described above can be executed.

[0167] The method or system according to the embodiment of the present application can also be used by Figure 5 The electronic device architecture shown in FIG. Figure 5 As shown, the electronic device may include a bus, one or more CPUs, a read-only memory (ROM), a random access memory (RAM), a communication port connected to a network, an input / output component, a hard disk, etc. A storage device in the electronic device, such as a ROM or a hard disk, may store the service segmentation collaborative control method for multi-node intelligent scheduling provided in the present application. The business segmentation collaborative control method of multi-node intelligent scheduling may, for example, include: deploying a heartbeat detection mechanism in each node intelligent agent, regularly sending heartbeat signals to other node intelligent agents and monitoring the heartbeat signals of other node intelligent agents, and identifying abnormal signals; marking the node intelligent agent that sends abnormal signals as a faulty node intelligent agent, obtaining the resource indicators of the faulty node intelligent agent, and identifying the fault type of the faulty node intelligent agent based on the resource indicator classification model; the faulty node intelligent agent sends a fault notification to other node intelligent agents according to the fault type, and all node intelligent agents determine whether it is necessary to elect a new master node intelligent agent based on the fault notification, and if necessary, elect a new master node intelligent agent; the master node intelligent agent reads the business execution information of the faulty node intelligent agent, splits the unfinished business into m sub-businesses, obtains the remaining available resources of the available node intelligent agent and the resource requirements of the sub-businesses, and allocates them to the available node intelligent agents according to the resource requirements of the sub-businesses; repairs the faulty node intelligent agent, and if the repair is successful, the master node intelligent agent returns the business; if the repair is unsuccessful, the business of the faulty node intelligent agent is redistributed and replaced. Furthermore, the electronic device may also include a user interface. Of course, Figure 5 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 5 One or more components of an electronic device are shown.

[0168] Example 4

[0169] Figure 6 Schematic diagram of a readable storage medium structure provided by an embodiment of the present application. Figure 6As shown, it is a readable storage medium according to an embodiment of the present application. Computer-readable instructions are stored on the computer-readable storage medium. When the computer-readable instructions are executed by the processor, the service segmentation collaborative control method for multi-node intelligent scheduling according to the embodiment of the present application described with reference to the above figures can be executed. The storage medium includes, but is not limited to, for example, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0170] In addition, according to the implementation mode of the present application, the process described in the above reference flowchart can be implemented as a computer software program. For example, the present application provides a non-transitory machine-readable storage medium, which stores machine-readable instructions, and the machine-readable instructions can be executed by a processor to execute instructions corresponding to the method steps provided by the present application, such as: deploying a heartbeat detection mechanism in each node intelligent agent, regularly sending heartbeat signals to other node intelligent agents and listening to the heartbeat signals of other node intelligent agents, and identifying abnormal signals; marking the node intelligent agent that sends abnormal signals as a faulty node intelligent agent, obtaining the resource indicators of the faulty node intelligent agent, and identifying the fault type of the faulty node intelligent agent based on the resource indicator classification model; the faulty node intelligent agent is classified according to the resource indicator classification model; The fault type sends a fault notification to other node agents, and all node agents determine whether it is necessary to elect a new master node agent based on the fault notification. If necessary, a new master node agent is elected; the master node agent reads the service execution information of the faulty node agent, splits the unfinished service into m sub-services, obtains the remaining available resources of the available node agents and the resource requirements of the sub-services, and allocates them to the available node agents according to the resource requirements of the sub-services; the faulty node agent is repaired, and if the repair is successful, the master node agent returns the service; if the repair is unsuccessful, the service of the faulty node agent is reallocated and replaced. When the computer program is executed by the central processing unit (CPU), the above functions defined in the method of the present application are executed.

[0171] The methods, apparatuses, and devices of the present application may be implemented in many ways. For example, the methods, apparatuses, and devices of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present application may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers recording media storing programs for executing the method according to the present application.

[0172] In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive redundancy.

[0173] The specific implementation modes as described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation mode of the present invention and is not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A business segmentation collaborative control method for multi-node intelligent scheduling, characterized in that: include: A heartbeat detection mechanism is deployed in each node agent to regularly send heartbeat signals to other node agents and monitor the heartbeat signals of other node agents to identify abnormal signals; Mark the node agent that sends the abnormal signal as a faulty node agent, obtain the resource index of the faulty node agent, and identify the fault type of the faulty node agent based on the resource index classification model; The faulty node agent sends a fault notification to other node agents according to the fault type. All node agents determine whether a new master node agent needs to be elected based on the fault notification. If necessary, a new master node agent is elected. The master node agent reads the service execution information of the faulty node agent, splits the unfinished service into m sub-services, obtains the remaining available resources of the available node agent and the resource requirements of the sub-services, and allocates them to the available node agents according to the resource requirements of the sub-services; The faulty node agent is repaired. If the repair is successful, the master node agent will return the service. If the repair is unsuccessful, the service of the faulty node agent will be redistributed and replaced.

2. The service segmentation collaborative control method for multi-node intelligent scheduling according to claim 1 is characterized in that: The heartbeat detection mechanism is deployed in each node agent, heartbeat signals are sent to other node agents regularly and heartbeat signals of other node agents are monitored. The specific method for identifying abnormal signals is as follows: Define the node information of the node agent, including node ID, node role, resource information, node status, and service execution information. The node role includes a master node agent and a non-master node agent. The node status includes normal, faulty, and unavailable. Register the node information of the node agent into the global node status table. A heartbeat detection mechanism is deployed in each node agent. The heartbeat detection mechanism includes a heartbeat transmitter and a heartbeat listener. The heartbeat transmitter sends a heartbeat signal to all other node agents every a1 unit time. The heartbeat listener continuously listens to the heartbeat signals sent by other node agents and records the timestamp of the last heartbeat signal received from each node agent. The heartbeat signal includes the node ID and timestamp of the sender. The heartbeat listener checks the timestamp of each heartbeat signal every a2 unit time. When the time difference between the timestamp of the heartbeat signal of the node agent and the current timestamp is greater than the time difference threshold, the heartbeat signal is marked as an abnormal signal.

3. The service segmentation collaborative control method for multi-node intelligent scheduling according to claim 2 is characterized in that: The specific method of marking the node agent that sends the abnormal signal as a faulty node agent, obtaining the resource index of the faulty node agent, and identifying the fault type of the faulty node agent based on the resource index classification model is as follows: When the heartbeat listener detects that the heartbeat signal of the node agent is an abnormal signal for a3 consecutive times, a fault alarm is triggered, the node status of the node agent that sends the abnormal signal is obtained, and its node status is changed to fault in the node status table, and the node agent is marked as a faulty node agent; Collect abnormal resource indicators and fault characteristics of historical fault node agents, extract feature vectors, construct n1 training samples, and train a resource indicator classification model containing K categories; Count the mean vector μ of the training samples in each category k and the covariance matrix C k ; The corresponding fault labels are manually marked according to the fault features of the cluster center in each category, and the fault labels include but are not limited to node downtime, resource exhaustion, software failure, load fluctuation, network jitter and security incident; Obtain resource indicators and feature vectors of the faulty node agent in real time, calculate the Mahalanobis distance between the resource indicators of the faulty node agent and each category, and obtain the similarity measure between the resource indicators of the faulty node agent and each category based on the Mahalanobis distance; The Mahalanobis distance and covariance matrix C between the resource index of the faulty node agent and each category k , calculate the abnormal probability that the resource indicators of the faulty node agent belong to each category; The similarity measure and the anomaly probability are weightedly summed to obtain the anomaly diagnosis score between the resource index of the faulty node intelligent agent and each category, and the fault label of the category with the smallest anomaly diagnosis score between the resource index of the faulty node intelligent agent is obtained. The fault label is the fault type of the faulty node intelligent agent.

4. The service segmentation collaborative control method for multi-node intelligent scheduling according to claim 3 is characterized in that: The specific training process of the resource indicator classification model is as follows: Step S221: Collect abnormal resource indicators of historical fault node agents to form an abnormal resource indicator time series; Step S222: define the length of the time window, extract features of the abnormal resource indicators in each time window to obtain feature vectors, and use the sliding window technology to divide the abnormal resource indicator time series into n1 training samples; Step S223: setting the number of target cluster categories K, and randomly selecting K training samples as initial cluster centers; Step S224: for each feature vector of a training sample, calculate the Euclidean distance between it and the feature vector of each cluster center, and assign the training sample to the category to which the cluster center with the closest distance belongs; Step S225: recalculate the mean vector of the feature vectors of all training samples in each category, and use the mean vector as a new cluster center; Step S226: Repeat steps S224 to S225 until the training samples in each category no longer change, thereby obtaining a trained resource indicator classification model.

5. The service segmentation collaborative control method for multi-node intelligent scheduling according to claim 4 is characterized in that: The fault node agent sends a fault notification to other node agents according to the fault type, and all node agents determine whether a new master node agent needs to be elected based on the fault notification. If necessary, the specific method for electing a new master node agent is as follows: The faulty node agent constructs a fault notification based on its fault type, node role, and timestamp, and sends the fault notification to other node agents; After receiving the fault notification, other node agents mark the node status of the faulty node agent as unavailable in the node status table; Other node agents determine whether it is necessary to elect a new master node agent based on the node role in the fault notification. If the faulty node agent is a non-master node agent, there is no need to elect a new master node agent. If the faulty node agent is the master node agent, and the fault type is any one of node downtime, resource exhaustion, software failure, and security incident, a new master node agent is elected; otherwise, there is no need to elect a new master node agent; For each node agent with a normal node status, the stability index S of its heartbeat signal sent in the past time T1 is counted success , get the number of nodes n that each node agent is directly connected to in the network topology neib and the resource index y in the past time T1, and calculate the average resource index based on the resource index in the past time T1 Calculate the comprehensive score of each node agent based on stability index, number of nodes and average resource index; Each node agent packages its comprehensive score and node ID into a voting message and sends it to all other non-faulty node agents. Each node agent is sorted in descending order based on the comprehensive score, and the node agent with the highest comprehensive score is used as the master node agent.

6. The service segmentation collaborative control method for multi-node intelligent scheduling according to claim 5 is characterized in that: The master control node agent reads the service execution information of the faulty node agent, splits the unfinished service into m sub-services, obtains the remaining available resources of the available node agent and the resource requirements of the sub-services, and allocates the resources to the available node agents according to the resource requirements of the sub-services. The specific method is as follows: The master control node agent reads the service execution information of the fault node agent, wherein the service execution information includes: service logic, service priority and service execution progress; The master node agent sorts the unfinished business in descending order of business priority, and divides the unfinished business into m sub-businesses based on the priority order and business logic; Obtain the resource requirements of each sub-business, and the master node agent forms a list of available node agents based on all node agents with normal node status in the current node status table; The master node agent traverses the unfinished sub-businesses, and for each sub-business, selects the available node agent with the highest remaining available resources as the node agent to execute the sub-business; If the remaining available resources of all available node agents are insufficient to process the sub-service, the sub-service will be further split until an available node agent that meets the resource requirements of the sub-service is found; The master node agent distributes the sub-service to the selected available node agent.

7. The service segmentation collaborative control method for multi-node intelligent scheduling according to claim 6 is characterized in that: The faulty node agent is repaired. If the repair is successful, the master node agent returns the service. If the repair is unsuccessful, the specific method for reallocating and replacing the service of the faulty node agent is as follows: The master control node agent repairs the faulty node agent. During the repair process, the master control node agent monitors the node status of the faulty node agent in real time and sets a first time length t1 as a time limit for the repair; If the repair is successful within the first time period t1, the master node agent returns the transferred business to the repaired node agent and updates the node status table; If the repair cannot be successfully completed within the first time period t1, the master node agent will distribute the remaining services of the failed node agent to other available node agents; A second time period t2 is set as the time limit for node replacement. If the faulty node agent still cannot be repaired successfully within the second time period t2, the master node agent sends a replacement notification to the administrator, informing the administrator to replace the faulty node agent with a new node agent.

8. A service segmentation collaborative control system for multi-node intelligent scheduling, which is used to implement the service segmentation collaborative control method for multi-node intelligent scheduling according to any one of claims 1 to 7, characterized in that: include: The abnormal signal identification module is used to deploy a heartbeat detection mechanism in each node agent, regularly send heartbeat signals to other node agents and monitor the heartbeat signals of other node agents to identify abnormal signals; A fault type identification module is used to mark the node agent that sends abnormal signals as a faulty node agent, obtain resource indicators of the faulty node agent, and identify the fault type of the faulty node agent based on a resource indicator classification model; The master node election module is used for the fault node agent to send fault notifications to other node agents according to the fault type. All node agents determine whether a new master node agent needs to be elected based on the fault notification. If necessary, a new master node agent is elected. The node service allocation module is used for the master node agent to read the service execution information of the faulty node agent, split the unfinished service into m sub-services, obtain the remaining available resources of the available node agent and the resource requirements of the sub-services, and allocate them to the available node agents according to the resource requirements of the sub-services; The fault node processing module is used to repair the fault node agent. If the repair is successful, the master node agent will return the service. If the repair is unsuccessful, the faulty node agent will be redistributed and replaced.

9. An electronic device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the service segment collaborative control method for multi-node intelligent scheduling as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and the computer program is suitable for being loaded by a processor to execute the steps in the service segment collaborative control method for multi-node intelligent scheduling as described in any one of claims 1-7.

Images