On-cloud scheduler conflict detection method and system based on formalization method and application

Abstract

The invention discloses an on-cloud scheduler conflict detection method based on a formalization method, and the method comprises the following steps: 1, inputting and / or importing an affinity and / or anti-affinity rule, converting the affinity and / or anti-affinity rule into a unified intermediate representation form constraint, shielding grammatical and semantic differences for affinity and / or anti-affinity expression among different schedulers; step 2, converting the intermediate representation form obtained by conversion in the step 1 into a format which can be identified and processed by a solver again, and converting an affinity and / or anti-affinity management problem into a formalized satisfiability problem; and step 3, checking the satisfiability of the constraint set through a solver, and for the unsatisfied rule, positioning the source file of the conflict rule and the position of the unsatisfied rule in the source file according to the metadata. The invention also discloses a conflict detection system for realizing the conflict detection method, and application of the method or the system.

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud computing and relates to a cloud scheduling conflict detection method, system and application based on a formal method. Background Art

[0002] In today's cloud data center environment, it has become common practice to deploy different applications on the same physical machine. This strategy improves the operational efficiency of the data center and reduces operating costs by fully utilizing computing resources. However, the mixed deployment of applications leads to the "noisy neighbor" problem, that is, different applications may compete for computing resources such as CPU and memory. When an application shares the same computing resources with other applications, the application may experience performance anomalies, such as increased response time or decreased throughput, which will affect the application service quality (QoS) and damage the end-user experience.

[0003] In order to alleviate the resource competition problem and ensure the stability of application performance, a major goal of the operation and maintenance management team of the cloud data center is to carefully plan and manage resource sharing between applications and properly isolate applications that are sensitive to resource contention. Therefore, modern cloud schedulers, such as Kubernetes and Apache Yarn, have introduced affinity and anti-affinity deployment constraints. Affinity constraints ensure that applications with collaborative relationships can be deployed in the same scheduling domain (such as the same physical server or rack), optimizing the communication efficiency between applications. In contrast, anti-affinity constraints are used to isolate applications with high demand for the same resources to prevent them from competing for resources in the same scheduling domain. Through these rules, the scheduler can comprehensively consider the dependencies and competition between applications when deploying applications, so as to more accurately formulate application deployment strategies and ensure the stability and performance of applications.

[0004] However, cloud data centers usually need to manage a large number of applications of various types. There are differences in the expression of rules between different schedulers. The continuous growth of the number of applications and the continuous evolution of constraints between applications make it difficult for operation and maintenance personnel to manually ensure the satisfiability of all constraints. Unsatisfiable scheduling constraints may cause the application to fail to deploy normally, and in serious cases may cause service interruptions and operation and maintenance accidents, bringing significant economic losses and reputation risks to cloud service providers. Therefore, there is an urgent need for a system that can automatically detect and solve constraint problems to support the efficient and stable deployment of applications in data centers. Such a system will simplify the cluster management process, reduce the risk of service quality degradation and service interruption caused by unsatisfied constraints, and help maintain high availability and service continuity of data centers. At present, the satisfiability of affinity and anti-affinity constraints in cloud data centers depends on manual verification by the operation and maintenance team, and the industry lacks an effective automated management and conflict detection system. Summary of the invention

[0005] The present invention aims to solve a series of constraint maintenance problems encountered in cloud data center management, especially affinity and / or anti-affinity constraints for application deployment. At present, with the continuous growth of the number of applications, application types and constraint complexity, it is becoming increasingly difficult to manually verify and manage these constraints in large-scale cloud data centers, which is prone to constraint conflicts, leading to application deployment failures and service interruptions.

[0006] Therefore, the purpose of the present invention is to provide a cloud scheduler conflict detection method and system based on formal methods, which can efficiently and accurately manage and verify such constraints in cloud data centers. The problems to be solved by the present invention are as follows:

[0007] 1) Different workloads in the production environment use different schedulers. For example, long-running services are scheduled by Kubernetes, and batch tasks are scheduled by Apache Yarn. The syntax and semantics of the rules described by different schedulers are different, making it difficult to manage them uniformly.

[0008] 2) There are many applications in cloud data centers, many scheduling rules, and complex constraints, which make it difficult to rely on manual analysis and verification. Manual analysis is difficult to ensure the accuracy of verification results, and the verification efficiency is low;

[0009] 3) The scheduling rules in cloud data centers are constantly updated due to application updates, and each update requires manual re-verification of the satisfiability of the constraints, which is labor-intensive and inefficient.

[0010] Based on the above problems, the present invention is committed to reducing the occurrence of operation and maintenance accidents caused by affinity and / or anti-affinity constraints, ensuring the continuity and stability of user services, and providing competitive advantages for cloud service providers in the fiercely competitive cloud computing market.

[0011] The specific technical solution for achieving the purpose of the present invention is:

[0012] The present invention provides a cloud scheduler conflict detection method based on a formal method, the method comprising the following steps:

[0013] Step 1: input and / or import affinity and / or anti-affinity rules, convert them into a unified intermediate representation, and shield the syntax and semantic differences of affinity and / or anti-affinity expressions between different schedulers;

[0014] Specifically, rule conversion and representation: In step 1 of the present invention, affinity and / or anti-affinity rules can be directly input, and / or these rules can be imported from existing scheduler configurations (such as Kubernetes or Apache Yarn). The present invention converts these rules into a unified intermediate representation. Through a unified intermediate representation, the grammatical and semantic differences in the expression of affinity and / or anti-affinity between different schedulers are shielded, which facilitates subsequent processing and verification. The structure of the intermediate representation is as follows Figure 3 As shown, a constraint of an intermediate representation includes two parts: entity and metadata. Specifically, the entity includes source entity, operator, target entity, and the metadata includes general metadata and scheduler-specific metadata.

[0015] ○ The source entity and target entity define the entity objects involved in the constraint. The source entity refers to the application and / or service that initiates the constraint, and the target entity refers to the object of the constraint; in a cloud data center, for example, the logical core of a single physical machine, a socket, a single physical machine, a rack, and an area consisting of multiple computer rooms are all legal entities.

[0016] The operator represents the relationship between the source entity and the target entity, which is expressed by require and exclude. Require indicates that there is an affinity relationship between the source entity and the target entity, and Exclude indicates that there is an anti-affinity relationship between the source entity and the target entity.

[0017] ○ General metadata includes the source file of the original rules (i.e., the input and / or imported affinity and / or anti-affinity rules), the line number defined in the source file, and other information, which is used to locate the original rules in case of conflicts. Scheduler-specific metadata includes the scheduling domain, tags, etc., which are used to subsequently generate specific configuration files and / or configuration information for different new schedulers.

[0018] Step 2: Convert the intermediate representation obtained in step 1 into a format that can be recognized and processed by the solver, and convert the affinity and / or anti-affinity management problem into a formalized satisfiability problem;

[0019] Specifically, the constraint satisfaction problem is transformed: the converted unified intermediate representation is converted into a format that can be recognized and processed by the solver. The solvers involved in the present invention are divided into two categories: constraint-based solving and mathematical programming-based solving:

[0020] 1. Based on constraint solving:

[0021] 1.1. Reduce affinity and / or anti-affinity rules to propositional logic and solve them using a Boolean Satisfiability Problem (SAT) solver. The SAT solvers involved include but are not limited to: Microsoft Z3 Prover, Google OR-Tools, and MiniSAT;

[0022] 1.2. Reduce affinity and / or anti-affinity rules to propositional logic and solve them using a constraint satisfaction problem (CSP) solver. The CSP solvers involved include but are not limited to: Choco, etc.

[0023] 1.3. Reduce affinity and / or anti-affinity rules to first-order predicate logic and solve them using Satisfiability Modulo Theories (SMT) solvers. The SMT solvers involved include but are not limited to: Microsoft Z3 Prover, Barcelogic, CVC3, MathSAT, Yices, etc.;

[0024] 1.4. Reduce affinity and / or anti-affinity rules to first-order predicate logic and solve them using a CSP solver. The CSP solvers involved include but are not limited to: Choco, etc.

[0025] 2. Solving based on mathematical programming:

[0026] 2.1. Reduce the affinity and / or anti-affinity rules to a set of equations and solve them using a mixed integer programming (MIP) solver. The MIP solvers involved include but are not limited to: Microsoft Z3Prover, Google OR-Tools, and IBM ILOG CPLEX.

[0027] 2.2. Reduce the affinity and / or anti-affinity rules to a set of equations and solve them using an integer linear programming (ILP) solver. The ILP solvers involved include but are not limited to: Microsoft Z3Prover, Google OR-Tools, and IBM ILOG CPLEX.

[0028] The unified intermediate representation converted from the affinity and / or anti-affinity rules configured by the scheduler can be expressed in any of the three forms: propositional logic, first-order predicate logic, and system of equations. Different solvers can be selected for solving the problem according to the specific form of formalization: propositional logic expression, first-order predicate logic expression, and system of equations. Different formalization methods can achieve the goal of detecting conflicting rules. Different solvers are used to verify the satisfiability of the rules by converting the unified intermediate representation into a representation acceptable to the corresponding solver, and checking the satisfiability of the rules through the solver. For unsatisfied rules, their minimum conflict set (minimum unsatisfiable core) is output.

[0029] When performing rule verification, if conflicts are found between certain rules, that is, they cannot be satisfied at the same time, the system will find the minimum set that causes this conflict, that is, the minimum rule subset, and the rules in this subset combined lead to the overall unsatisfiability; the system provides the minimum rule subset to help developers identify and resolve conflicts in the rule system.

[0030] Step 3: Check the satisfiability of the constraint set through the solver. For unsatisfied rules, locate the source file of the conflicting rule and the position of the unsatisfied rule in the source file according to the metadata.

[0031] The present invention can realize automatic configuration generation: for the constraint set verified to be satisfied, the present invention can automatically generate the configuration file of the scheduler. These configuration files will contain a verified set of rules and can be directly applied to the scheduling system of the cloud data center, thereby simplifying the work of operation and maintenance personnel and reducing deployment problems caused by human errors;

[0032] The method of the present invention can also realize deployment automation: the present invention provides a complete set of solutions, which is not limited to rule verification and configuration file generation. It can also be integrated with the scheduling system of the cloud data center to realize the automation of application deployment and ensure that all deployment actions comply with pre-defined constraint rules. For example, this plug-in can be used as a plug-in for the Kubernetes scheduler. When new affinity and / or anti-affinity constraint configurations are added or old affinity and / or anti-affinity constraints are updated, the above-mentioned conflict detection process is automatically executed. When the change will cause a rule conflict, the change is rejected from being applied to the scheduler, and the conflict information is fed back to the operation and maintenance personnel.

[0033] Through the above technical solution, the present invention solves the problems encountered by cloud data centers when processing affinity and / or anti-affinity constraints, improves the workflow of cluster operation and maintenance, reduces the complexity of affinity and / or anti-affinity rule constraints, improves deployment efficiency, and reduces the risks and costs caused by rule configuration errors.

[0034] The present invention also provides a cloud scheduler conflict detection system based on formal methods, the system comprising: an operation and maintenance management module, a rule parsing module, a logic conversion module, a logic verification module, a conflict location module, a configuration generation module, and an automatic deployment control module;

[0035] like Figure 1 As shown, the system architecture of the present invention ensures the effective implementation of its functions and guarantees good compatibility with the existing cloud data center environment. It is mainly composed of the following key components:

[0036] 1. Operation and maintenance management module: provides users with an easy-to-interact interface for inputting and / or importing affinity and / or anti-affinity rules. This interface supports direct rule entry and importing rule configurations from other scheduling systems, helping users quickly start the constraint management process;

[0037] 2. Rule parsing module: Receives rules submitted by the input interface and converts the user-defined or imported rule set into an intermediate representation that can be processed by the present invention. This intermediate layer provides standardization of rule processing, shields the syntax and semantic differences in affinity and / or anti-affinity expressions between different schedulers, and lays the foundation for subsequent verification and deployment processes;

[0038] 3. Logical conversion module: converts the parsed unified intermediate representation into a logical expression that can be recognized and processed by the solver. This module ensures the accuracy of the conversion process and the transparency of the operation;

[0039] 4. Logic verification module: Integrates an efficient solver and is responsible for verifying the transformed expression. This module is an executor for judging the satisfiability of rules and needs to be able to process logical operations quickly and accurately;

[0040] 5. Conflict location module: If some rules are verified to be unsatisfiable, this module will locate the specific applications and rule contents of these rule conflicts and provide users with detailed detection reports to facilitate operation and maintenance personnel to quickly troubleshoot and fix problems;

[0041] 6. Configuration building module: This module automatically generates the corresponding scheduler configuration file for the verified rules. Automated configuration generation reduces manual intervention, reduces the error rate, and enhances the accuracy and efficiency of the deployment process;

[0042] 7. Automatic deployment control module: It is closely integrated with the scheduling system of the cloud data center to ensure that the automated deployment process strictly complies with the verified constraints, thus optimizing the deployment quality and consistency of the application.

[0043] Through such a system architecture, the present invention realizes the automation of affinity and / or anti-affinity constraint management, provides an effective integrated solution from rule input to application deployment, and further optimizes the operation and maintenance process of the cloud data center.

[0044] like Figure 2 As shown in the figure, the system implementation process covers the entire process from inputting rules to deploying applications. The following are the detailed steps:

[0045] 1. Input rules:

[0046] o The user submits specific affinity and / or anti-affinity rules through the input service interface provided by the invention, or imports rule configurations from an existing scheduler (such as Kubernetes or Apache Yarn).

[0047] oThe input rules can be either rule syntax in intermediate representation or scheduler specific configuration files.

[0048] 2. Parsing rules:

[0049] oThe rule parser receives user-supplied rules.

[0050] oThe rules are syntax checked to ensure they are correct and converted into a unified intermediate representation.

[0051] 3. Convert to a format that can be recognized by the solver:

[0052] oThe conversion logic module converts the rules in the intermediate representation into expressions suitable for the solver.

[0053] oThe core of this step is to ensure that the rules can be expressed in a way that the solver can recognize and process.

[0054] 4. Verification rules:

[0055] oLogic validators use solvers to determine the satisfiability of rules.

[0056] oThe output of this step is a decision on whether the rule set is satisfiable.

[0057] 5. Manage unsatisfied rules:

[0058] oIf there are unsatisfied rules, the conflict locator will identify and report the specific conflicting rules and applications to the operation and maintenance management interface.

[0059] oOperation and maintenance personnel can perform conflict repairs and rule adjustments based on the information provided.

[0060] 6. Generate configuration files:

[0061] After confirming that all rules are satisfied, the configuration generator converts them into configuration files that can be directly applied to the scheduler.

[0062] oThis step reduces human errors during deployment and speeds up the deployment process.

[0063] 7.Automated deployment execution:

[0064] oThe automatic deployment controller receives the generated new configuration file and applies it to the actual scheduling system to perform application deployment.

[0065] oThis module ensures that application deployment strictly follows verified rules to ensure deployment quality.

[0066] 8. Monitoring and management:

[0067] oThe operation and maintenance management interface provides users with monitoring of the entire process, including but not limited to rule verification status, conflict reports, deployment progress, etc.

[0068] oIn this interface, operation and maintenance personnel can perform necessary interactive operations, such as adjusting rules or triggering new deployment processes.

[0069] The entire process design takes into account user experience and operational convenience, and will play an important role in maintaining efficient operation of data centers and reducing operational errors. Through automated verification and deployment processes, the present invention greatly reduces the probability of errors and improves the rationality of resource allocation and the efficiency of application deployment.

[0070] Figure 4 A sample process using a SAT solver is given. Figure 4 The lower part of the above is an example of the upper part. The overall process can be divided into three main steps: "import and inject", "formalize and solve satisfiability" and "generate". The steps of using SMT, ILP and MIP solvers are different only in the expression submitted to the solver after formalization. For SMT solver, it is first-order logic, and for ILP and MIP solvers, it is a system of equations. The rest of the steps are the same. Figure 4 Same as shown.

[0071] 1. “Import and Inject”: This step imports affinity and / or anti-affinity rules from the cloud scheduler configuration file, or injects new rules written in intermediate representation into the imported rule set;

[0072] 2. "Formal solution and satisfiability": This step formalizes the rule set unified into the intermediate representation into a form that can be recognized and processed by the solver, and checks the satisfiability of the constraint set through the solver; for unsatisfied rules, locate the source file of the conflicting rule and its location in the source file based on metadata;

[0073] 3. “Generate”: For a set of constraints that can be satisfied and there are newly injected rules, a new scheduler configuration file is generated and applied to the scheduler.

[0074] The method and system of the present invention provide an automated solution for managing complex constraints on the cloud, and its innovative features are as follows:

[0075] 1. The differences in the expression of affinity and / or anti-affinity constraints between different schedulers are shielded through a unified intermediate representation, so that the affinity and / or anti-affinity constraints between different schedulers can be uniformly expressed and managed;

[0076] 2. By formalizing the unified intermediate representation into propositional logic / first-order predicate logic / equation system, the affinity and / or anti-affinity management problem is converted into a formalized satisfiability problem. The correctness of the satisfiability judgment is guaranteed by the corresponding solver, thereby ensuring the correctness of the conflict detection.

[0077] The present invention also provides a hardware system for implementing the above-mentioned conflict detection method, the hardware system comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned method is implemented.

[0078] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned conflict detection method is implemented.

[0079] The present invention also provides applications of the above-mentioned conflict detection method, the above-mentioned conflict detection system, the above-mentioned hardware system or the above-mentioned computer-readable storage medium in a single Kubernetes scheduler scenario in a public cloud scenario, a hybrid Kubernetes and Apache Yarn scheduler scenario in a public cloud scenario, a single Kubernetes scheduler scenario in a private cloud scenario, a hybrid Kubernetes and Apache Yarn scheduler scenario in a private cloud scenario, etc.

[0080] The beneficial effects of the present invention include:

[0081] The implementation of the present invention will bring significant beneficial effects to the operation and maintenance management of cloud data centers, which are mainly reflected in the following aspects:

[0082] 1. Automated constraint management: By automatically converting, verifying, and applying affinity and / or anti-affinity rules, the burden on operations and maintenance personnel is greatly reduced, the deployment process is simplified, and configuration issues caused by manual management are reduced;

[0083] 2. Improve deployment efficiency and accuracy: The use of configuration generators and automatic deployment controllers improves deployment speed and ensures the consistency and accuracy of deployment steps, thereby reducing the risk of deployment failure;

[0084] 3. Optimize resource utilization: Reasonable affinity and / or anti-affinity rules help improve resource allocation and utilization, prevent resource waste, and improve application performance and stability;

[0085] 4. Reduce service interruption: An effective rule conflict management mechanism can effectively reduce the potential risk of service interruption and ensure the continuity and reliability of data center services;

[0086] 5. Enhanced troubleshooting capabilities: When the rules are not met, the conflict locator can provide accurate positioning information to assist operation and maintenance personnel in quickly troubleshooting problems and shorten the time to restore services;

[0087] 6. Compatibility across schedulers: Supporting rule input and conversion functions of multiple schedulers makes the present invention compatible with various scheduling environments, providing flexibility for users;

[0088] 7. Improve user experience: Applications with guaranteed stability and performance directly affect the end-user service experience. Through effective resource scheduling and application deployment, users can get more reliable and efficient services.

[0089] In summary, the implementation of the present invention has important practical and potential value in improving the operation and maintenance efficiency of cloud data centers, optimizing resource allocation, and ensuring service quality. In the growing cloud service market, the present invention helps cloud service providers consolidate their competitive advantages and improve their market position and customer satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0091] Figure 1 It is a system architecture diagram of the present invention.

[0092] Figure 2 It is a swimming lane diagram of the system of the present invention.

[0093] Figure 3 It is a schematic diagram of the intermediate representation definition of the present invention.

[0094] Figure 4 It is the overall flow chart of the present invention.

[0095] Figure 5 This is a reference flow chart of Example 1 of the present invention.

[0096] Figure 6 This is a reference flow chart of Example 2 of the present invention.

[0097] Figure 7 This is a reference flow chart of Example 3 of the present invention.

[0098] Figure 8 This is a reference flow chart of Example 4 of the present invention.

[0099] Fig. 9 This is a reference flow chart of Example 5 of the present invention.

[0100] Fig.10 This is a reference flow chart of Example 6 of the present invention.

[0101] Fig.11 This is a reference flow chart of Example 7 of the present invention.

[0102] Fig.12 This is a reference flow chart of Example 8 of the present invention.

[0103] Fig.13 This is a reference flow chart of Example 9 of the present invention.

[0104] Fig.14 This is a reference flow chart of Example 10 of the present invention.

[0105] Fig.15 This is a reference flow chart of Example 11 of the present invention.

[0106] Fig.16 This is a reference flow chart of Example 12 of the present invention. DETAILED DESCRIPTION

[0107] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0108] The present invention provides a cloud scheduler conflict detection method based on a formal method, the method comprising the following steps:

[0109] Step 1: Input and / or import affinity and / or anti-affinity rules, convert them into constraints in a unified intermediate representation, and shield the syntax and semantic differences of affinity and / or anti-affinity expressions between different schedulers;

[0110] Step 2: Convert the intermediate representation obtained in step 1 into a format that can be recognized and processed by the solver, and convert the affinity and / or anti-affinity management problem into a formalized satisfiability problem;

[0111] Step 3: Check the satisfiability of the constraint set through the solver. For unsatisfied rules, locate the source file of the conflicting rule and the position of the unsatisfied rule in the source file according to the metadata.

[0112] In step 1, the constraints of the intermediate representation include two parts: entity and metadata; the entity includes source entity, operator, target entity, and the metadata includes general metadata and scheduler-specific metadata;

[0113] The source entity and the target entity define entity objects involved in the constraint;

[0114] The source entity refers to the application and / or service that initiates the constraint;

[0115] The target entity refers to the object of the constraint;

[0116] The operator represents an affinity and / or anti-affinity relationship between the source entity and the target entity;

[0117] The general metadata includes source information of the affinity and / or anti-affinity rules input and / or imported;

[0118] The scheduler-specific metadata includes different scheduler-specific configuration files and / or configuration information.

[0119] In step 2, the solver is divided into two categories: constraint-based solving and mathematical programming-based solving;

[0120] In the constraint-solving based solver,

[0121] reducing affinity and / or anti-affinity rules to propositional logic, solved by a Boolean satisfiability problem solver and / or a constraint satisfiability problem solver; and / or,

[0122] reducing affinity and / or anti-affinity rules to first-order predicate logic, solved by a satisfiability modulo theory solver and / or a constraint satisfiability problem solver;

[0123] In the solver based on mathematical programming,

[0124] The affinity and / or anti-affinity rules are reduced to a system of equations and solved using a mixed integer programming solver and / or an integer linear programming solver.

[0125] In step 2, the unified intermediate representation converted from the affinity and / or anti-affinity rules configured by the scheduler is expressed in one of the three forms: propositional logic, first-order predicate logic, and system of equations. Different solvers are selected for solving the problem according to the specific form of formalization: propositional logic expression, first-order predicate logic expression, and system of equations. Different formalization methods can achieve the goal of detecting conflicting rules. Different solvers are used to verify the satisfiability of the rules by converting the unified intermediate representation into a representation acceptable to the corresponding solver, and checking the satisfiability of the rules through the solver. For the unsatisfied rules, their minimum conflict set is output.

[0126] Step three also includes automatically generating a scheduler configuration file for the constraint set verified to be satisfied; and / or automatically deploying applications that meet the constraint rules.

[0127] Take the Kubernetes scheduler as an example to illustrate the rules from input and / or import affinity and / or anti-affinity rules to the unified intermediate representation:

[0128]

[0129]

[0130] Take the configuration file of the Kubernetes scheduler shown above as an example. The configuration file contains two fields: podAffinity and podAntiAffinity. Among them, requiredDuringSchedulingIgnoredDuringExecution of podAffinity indicates that the rule is only effective during the scheduling process and is invalid during the application running process. This field contains an affinity rule, indicating that the application has affinity with the application containing the S1 field value in the app label under the topology.kubernetes.io / hostname scheduling domain. Similarly, podAntiAffinity represents an anti-affinity rule, indicating that the application has affinity with the application containing the S2 field value in the app label under the topology.kubernetes.io / hostname scheduling domain. If the operator value is NotIn, it means it is not included.

[0131] After receiving the above configuration file, the system converts the above two affinity and / or anti-affinity rules into the following two unified intermediate representations.

[0132] S0requireS1 / / typology=Node; File=S0.yaml; Line=9; key=app; operator=In; / / Container S0 needs to be scheduled on the same node as the Pod with label app=S1; File and Line indicate the location of the original rule in the configuration file, which is line 9 of the S0.yaml file

[0133] S0excludeS2 / / typology=Node; File=S0.yaml; Line=18; key=app; operator=In; / / Container S0 should avoid being scheduled on the same node as Pods with label app=S2; File and Line mark the location of this rule in the configuration file, which is line 18 of the S0.yaml file

[0134] Among them, typology represents the scheduling domain when the original rule is defined. In Kubernetes, the scheduling domain of topology.kubernetes.io / hostname will be converted to Node. File represents the file where the original rule is defined. Line represents the line number of the original rule defined in the original file. Key and Operator are metadata fields unique to the Kubernetes scheduler, corresponding to the key and operator fields in the original configuration file.

[0135] For other schedulers that support affinity and / or anti-affinity constraints, similar conversions are performed according to the semantic specifications defined by the corresponding schedulers.

[0136] In step three, rule verification: through the solver, the present invention checks whether the converted rule set is satisfiable. If the rule set is not satisfiable, a constraint conflict is detected. The solver will output the minimum unsatisfiable rule set (MinimalUnsatisfiable Core). The present invention can identify and locate conflicting applications and specific constraint rules by combining the metadata information of the minimum unsatisfiable rule set and the rules stored in the unified intermediate representation (for example: the file where the original rule definition is located, the file line number where the original rule definition is located), providing clear conflict information for operation and maintenance personnel, helping to quickly diagnose problems and take corresponding repair measures.

[0137] The format of a sample conflict information is as follows, which includes the specific name of the unschedulable application, the original file of the conflicting rule, the line number in the original file, and the corresponding content in the original file. The operation and maintenance personnel can quickly locate the conflicting rule based on the above information:

[0138] Unschedulable application S1:

[0139] Conflict Rules:

[0140]

[0141]

[0142] Example 1: Implementation based on SAT solver in a single scheduler

[0143] like Figure 5 As shown, for a cloud data center using a single scheduler for affinity and / or anti-affinity rules, the present invention provides the following specific implementation steps based on a SAT solver:

[0144] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0145] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0146] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0147] 4. Formalize the unified intermediate representation into a conjunctive normal form in propositional logic that can be accepted by SAT solvers;

[0148] 5. Check the satisfiability of the formalized scheduler constraint set using a SAT solver;

[0149] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0150] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0151] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0152] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0153] Example 2: Implementation of SAT solver in hybrid scheduler

[0154] like Figure 6 As shown, for a cloud data center using multiple hybrid schedulers, the present invention provides the following specific implementation steps based on a SAT solver:

[0155] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0156] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports multiple existing hybrid scheduler configuration files;

[0157] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0158] 4. Formalize the unified intermediate representation into a conjunctive normal form in propositional logic that can be accepted by SAT solvers;

[0159] 5. Check the satisfiability of the formalized scheduler constraint set using a SAT solver;

[0160] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0161] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0162] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0163] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0164] Example 3: Implementation based on SMT solver in a single scheduler

[0165] like Figure 7 As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on an SMT solver:

[0166] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0167] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0168] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0169] 4. Formalize the unified intermediate representation into a first-order logic formula that can be accepted by the SMT solver;

[0170] 5. Check the satisfiability of the formalized scheduler constraint set through the SMT solver;

[0171] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0172] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0173] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0174] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0175] Example 4: Implementation of SMT solver in hybrid scheduler

[0176] like Figure 8 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on an SMT solver:

[0177] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0178] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports existing multiple hybrid original scheduler configuration files;

[0179] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0180] 4. Formalize the unified intermediate representation into a first-order logic formula that can be accepted by the SMT solver;

[0181] 5. Check the satisfiability of the formalized scheduler constraint set through the SMT solver;

[0182] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0183] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0184] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0185] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0186] Example 5: Implementation in a single scheduler based on CSP solver and propositional logic

[0187] like Fig. 9As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on a CSP solver and propositional logic expression:

[0188] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0189] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0190] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0191] 4. Formalize the unified intermediate representation into a propositional logic formula that can be accepted by the CSP solver;

[0192] 5. Check the satisfiability of the formalized scheduler constraint set through the CSP solver;

[0193] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0194] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0195] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0196] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0197] Example 6: Implementation of a hybrid scheduler based on CSP solver and propositional logic

[0198] like Fig.10 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on a CSP solver and propositional logic expression:

[0199] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0200] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports existing multiple hybrid original scheduler configuration files;

[0201] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0202] 4. Formalize the unified intermediate representation into a propositional logic formula that can be accepted by the CSP solver;

[0203] 5. Check the satisfiability of the formalized scheduler constraint set through the CSP solver;

[0204] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0205] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0206] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0207] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0208] Example 7: Implementation in a single scheduler based on CSP solver and first-order predicate logic

[0209] like Fig.11 As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on a CSP solver and first-order predicate logic expression:

[0210] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0211] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0212] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0213] 4. Formalize the unified intermediate representation into a first-order predicate logic formula that can be accepted by the CSP solver;

[0214] 5. Check the satisfiability of the formalized scheduler constraint set through the CSP solver;

[0215] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0216] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0217] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0218] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0219] Example 8: Implementation of a hybrid scheduler based on CSP solver and propositional logic

[0220] like Fig.12 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on a CSP solver and first-order predicate logic expression:

[0221] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0222] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports existing multiple hybrid original scheduler configuration files;

[0223] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0224] 4. Formalize the unified intermediate representation into a first-order predicate logic formula that can be accepted by the CSP solver;

[0225] 5. Check the satisfiability of the formalized scheduler constraint set through the CSP solver;

[0226] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0227] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0228] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0229] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0230] Example 9: Implementation of an ILP solver in a single scheduler

[0231] like Fig.13 As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on an ILP solver:

[0232] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0233] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0234] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0235] 4. Formalize the unified intermediate representation into a system of equations that can be accepted by the ILP solver;

[0236] 5. Check the satisfiability of the formalized system of equations using the ILP solver;

[0237] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0238] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0239] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0240] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0241] Example 10: Implementation of ILP solver in hybrid scheduler

[0242] like Fig.14 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on an ILP solver:

[0243] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0244] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing hybrid multiple original scheduler configuration file;

[0245] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0246] 4. Formalize the unified intermediate representation into a system of equations that can be accepted by the ILP solver;

[0247] 5. Check the satisfiability of the formalized system of equations using the ILP solver;

[0248] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0249] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0250] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0251] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0252] Example 11: Implementation based on MIP solver in a single scheduler

[0253] like Fig.15 As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on a MIP solver:

[0254] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0255] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0256] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0257] 4. Formalize the unified intermediate representation into a set of equations that can be accepted by the MIP solver;

[0258] 5. Check the satisfiability of the formalized system of equations using the MIP solver;

[0259] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0260] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0261] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0262] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0263] Example 12: Implementation of MIP solver in hybrid scheduler

[0264] like Fig.16 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on a MIP solver:

[0265] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service strategy generation services;

[0266] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing hybrid multiple original scheduler configuration file;

[0267] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0268] 4. Formalize the unified intermediate representation into a set of equations that can be accepted by the MIP solver;

[0269] 5. Check the satisfiability of the formalized system of equations using the MIP solver;

[0270] 6. If the rules can be satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0271] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0272] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0273] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0274] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.

Claims

1. A cloud scheduler conflict detection method based on formal methods, characterized in that: The method comprises the following steps: Step 1: Input and / or import affinity and / or anti-affinity rules, convert them into constraints in a unified intermediate representation, and shield the syntax and semantic differences of affinity and / or anti-affinity expressions between different schedulers; Step 2: Convert the intermediate representation obtained in step 1 into a format that can be recognized and processed by the solver, and convert the affinity and / or anti-affinity management problem into a formalized satisfiability problem; Step 3: Check the satisfiability of the constraint set through the solver. For unsatisfied rules, locate the source file of the conflicting rule and the position of the unsatisfied rule in the source file according to the metadata.

2. The conflict detection method according to claim 1, characterized in that: In step 1, the constraints of the intermediate representation include two parts: entity and metadata; the entity includes source entity, operator, target entity, and the metadata includes general metadata and scheduler-specific metadata; The source entity and the target entity define entity objects involved in the constraint; The source entity refers to the application and / or service that initiates the constraint; The target entity refers to the object of the constraint; The operator represents an affinity and / or anti-affinity relationship between the source entity and the target entity; The general metadata includes source information of the affinity and / or anti-affinity rules input and / or imported; The scheduler-specific metadata includes different scheduler-specific configuration files and / or configuration information.

3. The conflict detection method according to claim 1, characterized in that: In step 2, the solver is divided into two categories: constraint-based solving and mathematical programming-based solving; In the constraint-solving based solver, reducing affinity and / or anti-affinity rules to propositional logic, solved by a Boolean satisfiability problem solver and / or a constraint satisfiability problem solver; and / or, reducing affinity and / or anti-affinity rules to first-order predicate logic, solved by a satisfiability modulo theory solver and / or a constraint satisfiability problem solver; In the solver based on mathematical programming, The affinity and / or anti-affinity rules are reduced to a system of equations and solved using a mixed integer programming solver and / or an integer linear programming solver.

4. The conflict detection method according to claim 1, characterized in that: In step 2, the unified intermediate representation converted from the affinity and / or anti-affinity rules configured by the scheduler is expressed in one of the three forms: propositional logic, first-order predicate logic, and system of equations. Different solvers are selected for solving the problem according to the specific form of formalization: propositional logic expression, first-order predicate logic expression, and system of equations. Different formalization methods can achieve the goal of detecting conflicting rules. Different solvers are used to verify the satisfiability of the rules by converting the unified intermediate representation into a representation acceptable to the corresponding solver, and checking the satisfiability of the rules through the solver. For the unsatisfied rules, their minimum conflict set is output.

5. The conflict detection method according to claim 1, characterized in that: Step three also includes automatically generating a scheduler configuration file for the constraint set verified to be satisfied; and / or automatically deploying applications that meet the constraint rules.

6. A conflict detection system implementing the conflict detection method according to any one of claims 1 to 5, characterized in that: The conflict detection system includes: an operation and maintenance management module, a rule parsing module, a logic conversion module, a logic verification module, a conflict location module, a configuration generation module, and an automatic deployment control module; The operation and maintenance management module is used to directly enter rules and / or import rule configurations from an existing scheduling system; The rule parsing module is used to receive the rules submitted by the user through the operation and maintenance management module, and convert the rules into a unified intermediate representation that can be processed by the system; The logic conversion module is used to convert the parsed unified intermediate representation into a logic expression that can be recognized and processed by the solver; The logic verification module is used to verify the converted logic expression and determine the satisfiability of the rule; The conflict location module is responsible for locating the specific application and rule content of the conflict and providing a detailed conflict report; The configuration building module is used to automatically generate corresponding scheduler configuration files for the rules that have passed the verification; The automatic deployment control module is combined with the scheduling system of the cloud data center to ensure that the entire deployment process automatically and strictly follows the verified constraints, thereby optimizing the deployment quality and consistency of the application.

7. A hardware system for implementing the method according to any one of claims 1 to 5, characterized in that: The hardware system comprises: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

9. Application of the conflict detection method according to any one of claims 1 to 5, the conflict detection system according to claim 6, the hardware system according to claim 7, or the computer-readable storage medium according to claim 8 in a single Kubernetes scheduler scenario in a public cloud scenario, a hybrid Kubernetes and Apache Yarn scheduler scenario in a public cloud scenario, a single Kubernetes scheduler scenario in a private cloud scenario, and a hybrid Kubernetes and Apache Yarn scheduler scenario in a private cloud scenario.

Images