A cloud-based scheduler conflict detection method, system and application based on formal methods

Abstract

The application discloses a cloud-based scheduler conflict detection method based on a formal method, and comprises the following steps: step one, input and / or import affinity and / or anti-affinity rules, and convert the rules into a unified intermediate representation form of constraints, so as to shield the syntax and semantic differences of affinity and / or anti-affinity expressions between different schedulers; step two, convert the intermediate representation form obtained in step one into a format recognizable by a solver again, so as to convert the affinity and / or anti-affinity management problem into a formal satisfiability problem; and step three, check the satisfiability of the constraint set by the solver, and for the rules that do not satisfy the constraints, locate the source file of the conflict rules and the position of the rules in the source file according to metadata. The application further discloses a conflict detection system for realizing the conflict detection method, and application of the method or system.

Description

Technical Field

[0001] The present invention belongs to the technical field of cloud computing and relates to a cloud scheduling conflict detection method, system and application based on formal methods. Background Art

[0002] In today's cloud data center environments, deploying a mix of different applications on the same physical machine has become common practice. This strategy improves data center operational efficiency and reduces operating costs by fully utilizing computing resources. However, mixed application deployment can lead to the "noisy neighbor" problem, where different applications may compete for computing resources such as CPU and memory. When an application shares the same computing resources with other applications, it may experience performance anomalies such as increased response time or decreased throughput, which can affect the application's Quality of Service (QoS) and impair the end-user experience.

[0003] To alleviate resource contention and ensure stable application performance, a primary goal of cloud data center operations and management teams is to carefully plan and manage resource sharing between applications and appropriately isolate applications that are sensitive to resource contention. Therefore, modern cloud schedulers, such as Kubernetes and Apache Yarn, have introduced affinity and anti-affinity deployment constraints. Affinity constraints ensure that collaborative applications can be deployed within the same scheduling domain (e.g., the same physical server or rack), optimizing inter-application communication efficiency. In contrast, anti-affinity constraints isolate applications with high demand for the same resources, preventing them from competing for resources within the same scheduling domain. These rules allow the scheduler to comprehensively consider inter-application dependencies and competition when deploying applications, allowing for more precise application deployment strategies and ensuring application stability and performance.

[0004] However, cloud data centers typically need to manage a large number of diverse applications. The differing rule expressions between schedulers, the ever-increasing number of applications, and the evolving constraints between them make it difficult for operations personnel to manually ensure the satisfiability of all constraints. Unsatisfiable scheduling constraints can prevent applications from deploying properly, and in severe cases, can cause service outages and operational incidents, resulting in significant financial losses and reputational risks for cloud service providers. Consequently, a system that can automatically detect and resolve constraint issues is urgently needed to support the efficient and stable deployment of applications within data centers. Such a system would simplify cluster management, reduce the risk of service degradation and outages caused by unsatisfied constraints, and help maintain high availability and service continuity in data centers. Currently, the satisfiability of affinity and anti-affinity constraints in cloud data centers relies on manual verification by operations teams, and the industry lacks effective automated management and conflict detection systems. Summary of the Invention

[0005] This invention aims to address a range of constraint maintenance issues encountered in cloud data center management, specifically affinity and / or anti-affinity constraints for application deployment. With the continued growth in the number and types of applications, as well as the complexity of constraints, manual verification and management of these constraints in large-scale cloud data centers is becoming increasingly difficult, leading to constraint conflicts that can cause application deployment failures and service interruptions.

[0006] Therefore, the purpose of this invention is to provide a cloud scheduler conflict detection method and system based on formal methods, which can efficiently and accurately manage and verify such constraints in cloud data centers. The problems to be solved by this invention are as follows:

[0007] 1) Different workloads in production environments use different schedulers. For example, long-running services are scheduled using Kubernetes, while batch tasks are scheduled using Apache Yarn. The syntax and semantics of the rules described by different schedulers vary, making unified management difficult.

[0008] 2) Cloud data centers have a large number of applications, numerous scheduling rules, and complex constraints, making manual analysis and verification difficult. Furthermore, manual analysis cannot guarantee the accuracy of verification results, resulting in low verification efficiency.

[0009] 3) The scheduling rules in cloud data centers are constantly updated due to application updates and other factors. Each update requires manual re-verification of the constrained satisfiability, which is labor-intensive and inefficient.

[0010] In view of the above problems, the present invention is committed to reducing the occurrence of operation and maintenance accidents caused by affinity and / or anti-affinity constraints, ensuring the continuity and stability of user services, and providing competitive advantages for cloud service providers in the fiercely competitive cloud computing market.

[0011] The specific technical solution for achieving the purpose of the present invention is:

[0012] The present invention provides a cloud scheduler conflict detection method based on a formal method, the method comprising the following steps:

[0013] Step 1: Input and / or import affinity and / or anti-affinity rules and convert them into a unified intermediate representation to mask the syntactic and semantic differences in affinity and / or anti-affinity expressions between different schedulers.

[0014] Specifically, rule conversion and representation: In step 1 of the present invention, affinity and / or anti-affinity rules can be directly input, and / or these rules can be imported from existing scheduler configurations (such as Kubernetes or Apache Yarn). The present invention converts these rules into a unified intermediate representation. Through a unified intermediate representation, the grammatical and semantic differences in affinity and / or anti-affinity expressions between different schedulers are shielded, which facilitates subsequent processing and verification. The structure of the intermediate representation is as follows: Figure 3 As shown, a constraint in an intermediate representation includes two parts: entity and metadata. Specifically, the entity includes source entity, operator, and target entity, and the metadata includes general metadata and scheduler-specific metadata.

[0015] The source and target entities define the entities involved in the constraint. The source entity is the application and / or service that initiates the constraint, and the target entity is the object of the constraint. In a cloud data center, for example, a single physical machine's logical core, a socket, a single physical machine, a rack, and a region consisting of multiple computer rooms are all legal entities.

[0016] The operator represents the relationship between the source entity and the target entity, which is expressed by require and exclude. Require indicates that there is an affinity relationship between the source entity and the target entity, while Exclude indicates that there is an anti-affinity relationship between the source entity and the target entity.

[0017] General metadata includes the source files of the original rules (i.e., input and / or imported affinity and / or anti-affinity rules), as well as information such as the line numbers defined in the source files, which is used to locate the original rules in the event of conflicts. Scheduler-specific metadata includes scheduling domains, tags, and other information used to subsequently generate specific configuration files and / or configuration information for different new schedulers.

[0018] Step 2: Convert the intermediate representation obtained in step 1 into a format that can be recognized and processed by the solver, and convert the affinity and / or anti-affinity management problem into a formal satisfiability problem;

[0019] Specifically, the constraint satisfaction problem is transformed: the converted unified intermediate representation is converted into a format that can be recognized and processed by the solver. The solvers involved in this invention are divided into two categories: constraint-based solving and mathematical programming-based solving:

[0020] 1. Based on constraint solving:

[0021] 1.1. Reduce affinity and / or anti-affinity rules to propositional logic and solve them using a Boolean Satisfiability Problem (SAT) solver. SAT solvers include, but are not limited to, Microsoft Z3 Prover, Google OR-Tools, and MiniSAT.

[0022] 1.2. Reduce affinity and / or anti-affinity rules to propositional logic and solve them using a Constraint Satisfaction Problem (CSP) solver. Examples of CSP solvers include, but are not limited to, Choco.

[0023] 1.3. Reduce affinity and / or anti-affinity rules to first-order predicate logic and solve them using a Satisfiability Modulo Theories (SMT) solver. Examples of SMT solvers include, but are not limited to, Microsoft Z3 Prover, Barcelogic, CVC3, MathSAT, and Yices.

[0024] 1.4. Reduce affinity and / or anti-affinity rules to first-order predicate logic and solve them using a CSP solver. CSP solvers include, but are not limited to, Choco.

[0025] 2. Solving based on mathematical programming:

[0026] 2.1. Reduce the affinity and / or anti-affinity rules to a system of equations and solve them using a Mixed Integer Programming (MIP) solver. Suitable MIP solvers include, but are not limited to, Microsoft Z3Prover, Google OR-Tools, and IBM ILOG CPLEX.

[0027] 2.2. Reduce the affinity and / or anti-affinity rules to a system of equations and solve them using an integer linear programming (ILP) solver. Suitable ILP solvers include, but are not limited to, Microsoft Z3Prover, Google OR-Tools, and IBM ILOG CPLEX.

[0028] The unified intermediate representation (IR) converted from the affinity and / or anti-affinity rules configured by the scheduler can be expressed in any of three forms: propositional logic, first-order predicate logic, or a system of equations. Depending on the specific form in which the problem is formalized (propositional logic, first-order predicate logic, or a system of equations), different solvers can be selected for solving the problem. Different formalization methods can all achieve the goal of detecting conflicting rules. Verifying the satisfiability of rules using different solvers involves converting the unified IR into a representation acceptable to the corresponding solver, and then checking the satisfiability of the rules through the solver. For unsatisfied rules, the minimum unsatisfiable core is output.

[0029] During rule verification, if conflicts are found between certain rules, that is, they cannot be satisfied at the same time, the system will find the minimum set that causes this conflict, that is, the minimum rule subset, the combination of the rules in this subset leads to the overall unsatisfiability; the system provides the said minimum rule subset to help developers identify and resolve conflicts in the rule system.

[0030] Step 3: Check the satisfiability of the constraint set through the solver. For unsatisfied rules, locate the source file of the conflicting rule and the position of the unsatisfied rule in the source file based on the metadata.

[0031] The present invention enables automatic configuration generation: for a set of constraints that are verified to be satisfied, the present invention can automatically generate a scheduler configuration file. These configuration files will contain a verified set of rules and can be directly applied to the scheduling system of the cloud data center, thereby simplifying the work of operations and maintenance personnel and reducing deployment issues caused by human error.

[0032] The method of the present invention can also realize deployment automation: the present invention provides a complete set of solutions, which is not limited to rule verification and configuration file generation. It can also be integrated with the scheduling system of the cloud data center to realize the automation of application deployment and ensure that all deployment actions comply with pre-defined constraint rules. For example, this plug-in can be used as a plug-in for the Kubernetes scheduler. When new affinity and / or anti-affinity constraint configurations are added or old affinity and / or anti-affinity constraints are updated, the above-mentioned conflict detection process is automatically executed. When the change will cause a rule conflict, the change will be rejected from being applied to the scheduler, and the conflict information will be fed back to the operation and maintenance personnel.

[0033] Through the above technical solution, the present invention solves the problems encountered by cloud data centers when processing affinity and / or anti-affinity constraints, improves the workflow of cluster operation and maintenance, reduces the complexity of affinity and / or anti-affinity rule constraints, improves deployment efficiency, and reduces the risks and costs caused by rule configuration errors.

[0034] The present invention also provides a cloud scheduler conflict detection system based on formal methods, the system comprising: an operation and maintenance management module, a rule parsing module, a logic conversion module, a logic verification module, a conflict location module, a configuration generation module, and an automatic deployment control module;

[0035] like Figure 1 As shown, the system architecture of the present invention ensures the effective implementation of its functions and good compatibility with the existing cloud data center environment. It mainly consists of the following key components:

[0036] 1. Operation and Maintenance Management Module: Provides users with an easy-to-interact interface for entering and / or importing affinity and / or anti-affinity rules. This interface supports direct rule entry and importing rule configurations from other scheduling systems, helping users quickly start the constraint management process;

[0037] 2. Rule parsing module: Receives rules submitted via the input interface and converts user-defined or imported rule sets into an intermediate representation that can be processed by the present invention. This intermediate layer standardizes rule processing, shields the syntactic and semantic differences in affinity and / or anti-affinity expressions between different schedulers, and lays the foundation for subsequent verification and deployment processes.

[0038] 3. Logic Conversion Module: This module converts the parsed unified intermediate representation into a logical expression that the solver can recognize and process. This module ensures the accuracy of the conversion process and the transparency of the operation.

[0039] 4. Logic Verification Module: This module integrates an efficient solver and is responsible for verifying the transformed expressions. This module is the executor for determining rule satisfiability and must be able to process logical operations quickly and accurately.

[0040] 5. Conflict Location Module: If certain rules are verified as unsatisfiable, this module will locate the specific applications and rule content of these rule conflicts and provide users with a detailed detection report to facilitate operations and maintenance personnel to quickly troubleshoot and fix the problem;

[0041] 6. Configuration Building Module: This module automatically generates the corresponding scheduler configuration files for the rules that have passed verification. Automated configuration generation reduces manual intervention, lowers the error rate, and enhances the accuracy and efficiency of the deployment process.

[0042] 7. Automatic deployment control module: Tightly integrated with the cloud data center's scheduling system, it ensures that the automated deployment process strictly complies with verified constraints, optimizing the quality and consistency of application deployment.

[0043] Through such a system architecture, the application realizes the automation of affinity and / or anti-affinity constraint management, provides an integrated solution from rule input to application deployment, and further optimizes the operation and maintenance process of the cloud data center.

[0044] As shown in Figure 2 , the implementation process of the system covers the whole process from inputting rules to deploying applications. The following are the detailed steps:

[0045] 1. Input rules:

[0046] o The user submits specific affinity and / or anti-affinity rules through the input service interface provided by the application, or imports rule configurations from existing schedulers (such as Kubernetes or Apache Yarn).

[0047] o The input rules can be rule syntax in intermediate representation or scheduler-specific configuration files.

[0048] 2. Parse rules:

[0049] o The rule parser receives the rules provided by the user.

[0050] o Syntax check is performed on the rules to ensure that there are no errors, and the rules are converted into a unified intermediate representation form.

[0051] 3. Convert to a format recognizable by the solver:

[0052] o The conversion logic module converts the rules in intermediate representation into expressions suitable for the solver.

[0053] o The core of this step is to ensure that the rules can be expressed in a way that the solver can recognize and process.

[0054] 4. Verify rules:

[0055] o The logical verifier uses the solver to determine the satisfiability of the rules.

[0056] o The output of this step is the determination result of whether the rule set is satisfiable.

[0057] 5. Manage unsatisfied rules:

[0058] o If there are unsatisfied rules, the conflict locator identifies and reports the specific conflicting rules and applications to the operation and maintenance interface.

[0059] o The operation and maintenance personnel can perform conflict repair and rule adjustment based on the provided information.

[0060] 6. Generate configuration files:

[0061] After confirming that all rules are satisfied, the configuration generator converts them into a configuration file that can be directly applied to the scheduler.

[0062] oThis step reduces human errors during deployment and speeds up the deployment process.

[0063] 7.Automated deployment execution:

[0064] oThe automatic deployment controller receives the generated new configuration file and applies it to the actual scheduling system to perform application deployment.

[0065] oThis module ensures that application deployment strictly follows verified rules to ensure deployment quality.

[0066] 8. Monitoring and management:

[0067] The operation and maintenance management interface provides users with monitoring of the entire process, including but not limited to rule verification status, conflict reports, deployment progress, etc.

[0068] On this interface, operations personnel can perform necessary interactive operations, such as adjusting rules or triggering new deployment processes.

[0069] The entire process is designed with user experience and operational convenience in mind, playing a vital role in maintaining efficient data center operations and reducing operational errors. By automating the verification and deployment process, this invention significantly reduces the probability of errors and improves the rationality of resource allocation and the efficiency of application deployment.

[0070] Figure 4 A sample process using a SAT solver is given. Figure 4 The second half of the article is an example to illustrate the first half. The overall process can be divided into three main steps: "import and injection", "formalization and solving satisfiability" and "generation". The steps of using SMT, ILP and MIP solvers are different only in the expression submitted to the solver after formalization. For SMT solver, it is first-order logic, and for ILP and MIP solvers, it is a system of equations. The rest of the steps are the same. Figure 4 Same as shown.

[0071] 1. "Import and Inject": This step imports affinity and / or anti-affinity rules from the cloud scheduler configuration file, or injects new rules written in the intermediate representation into the imported rule set;

[0072] 2. "Formal Solution and Satisfiability": This step formalizes the rule set, unified into an intermediate representation, into a form recognizable and processable by the solver. The solver then checks the satisfiability of the constraint set. For unsatisfied rules, the source file and location of the conflicting rule are located based on metadata.

[0073] 3. “Generate”: For a set of constraints that can be satisfied and there are newly injected rules, a new scheduler configuration file is generated and applied to the scheduler.

[0074] The method and system of the present invention provide an automated solution for managing complex constraints on the cloud. Its innovations are as follows:

[0075] 1. A unified intermediate representation shields the differences in how affinity and / or anti-affinity constraints are expressed between different schedulers, allowing affinity and / or anti-affinity constraints to be uniformly expressed and managed across schedulers.

[0076] 2. By formalizing the unified intermediate representation into propositional logic / first-order predicate logic / a system of equations, the affinity and / or anti-affinity management problem is converted into a formalized satisfiability problem. The correctness of the satisfiability judgment and, therefore, the correctness of the conflict detection is guaranteed by the corresponding solver.

[0077] The present invention also provides a hardware system for implementing the above-mentioned conflict detection method, the hardware system comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned method is implemented.

[0078] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned conflict detection method is implemented.

[0079] The present invention also provides applications of the above-mentioned conflict detection method, the above-mentioned conflict detection system, the above-mentioned hardware system or the above-mentioned computer-readable storage medium in a single Kubernetes scheduler scenario in a public cloud scenario, a hybrid Kubernetes and Apache Yarn scheduler scenario in a public cloud scenario, a single Kubernetes scheduler scenario in a private cloud scenario, a hybrid Kubernetes and Apache Yarn scheduler scenario in a private cloud scenario, etc.

[0080] The beneficial effects of the present invention include:

[0081] The implementation of this invention will bring significant benefits to the operation and maintenance management of cloud data centers, mainly reflected in the following aspects:

[0082] 1. Automated constraint management: By automatically converting, validating, and applying affinity and / or anti-affinity rules, the burden on operations personnel is greatly reduced, the deployment process is simplified, and configuration issues caused by manual management are reduced.

[0083] 2. Improve deployment efficiency and accuracy: The use of configuration generators and automatic deployment controllers increases deployment speed and ensures the consistency and accuracy of deployment steps, thereby reducing the risk of deployment failure;

[0084] 3. Optimize resource utilization: Reasonable affinity and / or anti-affinity rules help improve resource allocation and utilization, prevent resource waste, and enhance application performance and stability.

[0085] 4. Reduce service interruptions: An effective rule conflict management mechanism can effectively reduce potential service interruption risks and ensure the continuity and reliability of data center services;

[0086] 5. Enhanced troubleshooting capabilities: When rules are not met, the conflict locator can provide accurate location information, assisting operations personnel in quickly troubleshooting the problem and shortening the time to restore service.

[0087] 6. Cross-scheduler compatibility: Supporting rule input and conversion functions for multiple schedulers makes the present invention compatible with various scheduling environments, providing flexibility for users;

[0088] 7. Improve user experience: Applications with guaranteed stability and performance directly affect the end-user service experience. Through effective resource scheduling and application deployment, users receive more reliable and efficient services.

[0089] In summary, the implementation of this invention has significant practical and potential value in improving cloud data center operational efficiency, optimizing resource allocation, and ensuring service quality. In the growing cloud service market, this invention helps cloud service providers consolidate their competitive advantage and improve their market position and customer satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0091] Figure 1 It is a system architecture diagram of the present invention.

[0092] Figure 2 It is a lane diagram of the system of the present invention.

[0093] Figure 3 It is a schematic diagram of the intermediate representation definition of the present invention.

[0094] Figure 4 It is the overall flow chart of the present invention.

[0095] Figure 5 This is a reference flow chart of Example 1 of the present invention.

[0096] Figure 6 This is a reference flow chart of Example 2 of the present invention.

[0097] Figure 7 This is a reference flow chart of Example 3 of the present invention.

[0098] Figure 8 This is a reference flow chart of Example 4 of the present invention.

[0099] Figure 9 This is a reference flow chart of Example 5 of the present invention.

[0100] Figure 10 This is a reference flow chart of Example 6 of the present invention.

[0101] Figure 11 This is a reference flow chart of Example 7 of the present invention.

[0102] Figure 12 This is a reference flow chart of Example 8 of the present invention.

[0103] Figure 13 This is a reference flow chart of Example 9 of the present invention.

[0104] Figure 14 This is a reference flow chart of Example 10 of the present invention.

[0105] Figure 15 This is a reference flow chart of Example 11 of the present invention.

[0106] Figure 16 This is a reference flow chart of Example 12 of the present invention. DETAILED DESCRIPTION

[0107] The present invention is further described in detail with reference to the following specific examples and accompanying drawings. The processes, conditions, experimental methods, etc. for implementing the present invention, except for those specifically mentioned below, are common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0108] The present invention provides a cloud scheduler conflict detection method based on a formal method, the method comprising the following steps:

[0109] Step 1: Input and / or import affinity and / or anti-affinity rules and convert them into constraints in a unified intermediate representation, masking the syntactic and semantic differences in affinity and / or anti-affinity expressions between different schedulers;

[0110] Step two, the intermediate representation obtained in step one is converted again into a format recognizable by a solver, and the affinity and / or anti-affinity management problem is converted into a formal satisfiability problem;

[0111] Step three, the satisfiability of the constraint set is checked by a solver, and for the rules that do not satisfy, the source file of the conflicting rules and the location of the rules in the source file are located according to the metadata.

[0112] In step one, the constraints of the intermediate representation include two parts of entities and metadata; the entities include source entities, operators, target entities, the metadata includes general metadata and scheduler-specific metadata;

[0113] The source entity and the target entity define the entity object involved in the constraint;

[0114] The source entity refers to the application and / or service that initiates the constraint;

[0115] The target entity refers to the object of the constraint;

[0116] The operator represents the affinity and / or anti-affinity relationship between the source entity and the target entity;

[0117] The general metadata includes the source information of the imported affinity and / or anti-affinity rules;

[0118] The scheduler-specific metadata includes different scheduler-specific configuration files and / or configuration information.

[0119] In step two, the solver is divided into two categories based on constraint solving and mathematical programming solving;

[0120] In the constraint-based solver,

[0121] The affinity and / or anti-affinity rules are reduced to propositional logic, and are solved by a Boolean satisfiability problem solver and / or a constraint satisfiability problem solver; and / or,

[0122] The affinity and / or anti-affinity rules are reduced to first-order predicate logic, and are solved by a satisfiability module theory solver and / or a constraint satisfiability problem solver;

[0123] In the mathematical programming-based solver,

[0124] The affinity and / or anti-affinity rules are reduced to a system of equations, and are solved by a mixed integer programming solver and / or an integer linear programming solver.

[0125] In step two, the converted uniform intermediate representation of the affinity and / or anti-affinity rules configured by the scheduler is expressed in the form of propositional logic, first-order predicate logic, or equation set. Depending on the specific form of the problem formalization: propositional logic expression, first-order predicate logic expression, or equation set expression, different solvers are selected for solving. Different formalization methods can achieve the goal of detecting conflicting rules. Different solvers verify the satisfiability of the rules by converting the uniform intermediate representation into a representation acceptable to the solver and checking the satisfiability of the rules. For rules that do not satisfy, the smallest conflict set is output.

[0126] Step three also includes automatically generating a scheduler configuration file for the constraint set verified as satisfiable, and / or automatically deploying the application that meets the constraint rules.

[0127] The following illustrates the rules from input and / or imported affinity and / or anti-affinity rules to uniform intermediate representation using the Kubernetes scheduler as an example:

[0128]

[0129]

[0130] Taking one of the configuration files of the Kubernetes scheduler shown in the above text as an example, the configuration file contains two fields: podAffinity and podAntiAffinity. Among them, requiredDuringSchedulingIgnoredDuringExecution of podAffinity indicates that the rule only takes effect during the scheduling process and is invalid during the application running process. The field contains an affinity rule indicating that the application has affinity with the application containing the S1 field value in the app label under the topology.kubernetes.io / hostname scheduling domain. Similarly, podAntiAffinity indicates an anti-affinity rule, indicating that the application has affinity with the application containing the S2 field value in the app label under the topology.kubernetes.io / hostname scheduling domain. If the operator value is NotIn, it means not containing.

[0131] After receiving the above configuration file, the system converts the above two affinity and / or anti-affinity rules into the following two uniform intermediate representations.

[0132] S0requireS1 / / typology=Node; File=S0.yaml; Line=9; key=app; operator=In; / / Container S0 needs to be scheduled on the same node as the Pod with label app=S1; File and Line indicate the location of the original rule in the configuration file, which is line 9 of the S0.yaml file

[0133] S0excludeS2 / / typology=Node;File=S0.yaml;Line=18;key=app;operator=In; / / Container S0 should avoid being scheduled on the same node as Pods with label app=S2; File and Line indicate the location of this rule in the configuration file, which is line 18 of the S0.yaml file

[0134] Typology represents the scheduling domain when the original rule was defined. In Kubernetes, the scheduling domain of topology.kubernetes.io / hostname is converted to Node. File represents the file where the original rule is defined. Line represents the line number where the original rule is defined in the original file. Key and Operator are metadata fields specific to the Kubernetes scheduler and correspond to the key and operator fields in the original configuration file.

[0135] For other schedulers that support affinity and / or anti-affinity constraints, similar conversions are performed according to the semantic specifications defined by the corresponding schedulers.

[0136] In step three, rule verification: the present invention uses a solver to check whether the converted rule set is satisfiable. If the rule set is not satisfiable, a constraint conflict is detected. The solver will output the minimum unsatisfiable rule set (MinimalUnsatisfiable Core). The present invention can identify and locate conflicting applications and specific constraint rules by combining the minimum unsatisfiable rule set and the metadata information of the rules stored in the unified intermediate representation (for example: the file where the original rule definition is located, the file line number where the original rule definition is located), providing clear conflict information to operation and maintenance personnel, helping to quickly diagnose problems and take corresponding repair measures.

[0137] The following example shows conflict information, which includes the name of the unschedulable application, the original file of the conflicting rule, the line number in the original file, and the corresponding content in the original file. Operations and maintenance personnel can quickly locate the conflicting rule based on this information:

[0138] Unschedulable application S1:

[0139] Conflict Rules:

[0140]

[0141]

[0142] Example 1: Implementation in a single scheduler based on a SAT solver

[0143] like Figure 5 As shown, for a cloud data center using a single scheduler for affinity and / or anti-affinity rules, the present invention provides the following specific implementation steps based on a SAT solver:

[0144] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0145] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0146] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0147] 4. Formalize the unified intermediate representation into a conjunctive normal form in propositional logic that can be accepted by SAT solvers;

[0148] 5. Check the satisfiability of the formalized scheduler constraint set using a SAT solver;

[0149] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0150] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0151] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0152] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0153] Example 2: Implementation of a SAT solver in a hybrid scheduler

[0154] like Figure 6 As shown, for a cloud data center using multiple hybrid schedulers, the present invention provides the following specific implementation steps based on a SAT solver:

[0155] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0156] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports multiple existing hybrid scheduler configuration files;

[0157] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0158] 4. Formalize the unified intermediate representation into a conjunctive normal form in propositional logic that can be accepted by SAT solvers;

[0159] 5. Check the satisfiability of the formalized scheduler constraint set using a SAT solver;

[0160] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0161] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0162] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0163] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0164] Example 3: Implementation in a single scheduler based on an SMT solver

[0165] like Figure 7 As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on an SMT solver:

[0166] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0167] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0168] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0169] 4. Formalize the unified intermediate representation into a first-order logic formula that can be accepted by the SMT solver;

[0170] 5. Check the satisfiability of the formalized scheduler constraint set using an SMT solver;

[0171] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0172] 7. The newly generated configuration files are deployed to the scheduler cluster, and the deployment of the application is completed;

[0173] 8. If the rules cannot be satisfied, the application and the conflicting rules are located by the application, and the service interface outputs a conflict report. The user needs to adjust the rules according to the feedback;

[0174] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until the rules are all satisfied.

[0175] Example 4: Implementation of SMT solver-based in hybrid scheduler

[0176] As shown in Figure 8 , for a cloud data center using a hybrid scheduler, the application provides the following specific implementation steps based on the SMT solver:

[0177] 1. The user deploys the application in a cluster of single schedulers to provide a scheduling service policy generation service;

[0178] 2. The user inputs the affinity and / or anti-affinity rules through the service interface, or imports existing multiple hybrid original scheduler configuration files;

[0179] 3. The rule parsing module of the application parses the input rules, and converts the rule set into a unified intermediate representation form;

[0180] 4. The unified intermediate representation form is formalized into a first-order logic formula that can be accepted by the SMT solver;

[0181] 5. The satisfiability of the formalized scheduler constraint set is checked by the SMT solver;

[0182] 6. If the rules are satisfiable, the configuration generation module generates configuration files that satisfy the scheduler specification;

[0183] 7. The newly generated configuration files are deployed to the scheduler cluster, and the deployment of the application is completed;

[0184] 8. If the rules cannot be satisfied, the application and the conflicting rules are located by the application, and the service interface outputs a conflict report. The user needs to adjust the rules according to the feedback;

[0185] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until the rules are all satisfied.

[0186] Example 5: Implementation of CSP solver and propositional logic in a single scheduler

[0187] As shown in Figure 9As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on a CSP solver and propositional logic expression:

[0188] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0189] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0190] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0191] 4. Formalize the unified intermediate representation into a propositional logic formula that can be accepted by the CSP solver;

[0192] 5. Check the satisfiability of the formalized scheduler constraint set using a CSP solver;

[0193] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0194] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0195] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0196] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0197] Example 6: Implementation of a hybrid scheduler based on CSP solver and propositional logic

[0198] like Figure 10 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on a CSP solver and propositional logic expression:

[0199] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0200] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports existing multiple hybrid original scheduler configuration files;

[0201] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0202] 4. Formalize the unified intermediate representation into a propositional logic formula that can be accepted by the CSP solver;

[0203] 5. Check the satisfiability of the formalized scheduler constraint set using a CSP solver;

[0204] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0205] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0206] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0207] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0208] Example 7: Implementation in a single scheduler based on CSP solver and first-order predicate logic

[0209] like Figure 11 As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on a CSP solver and first-order predicate logic expression:

[0210] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0211] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0212] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0213] 4. Formalize the unified intermediate representation into a first-order predicate logic formula that can be accepted by the CSP solver;

[0214] 5. Check the satisfiability of the formalized scheduler constraint set using a CSP solver;

[0215] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0216] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0217] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0218] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0219] Example 8: Implementation of a hybrid scheduler based on CSP solver and propositional logic

[0220] like Figure 12 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on a CSP solver and first-order predicate logic expression:

[0221] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0222] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports existing multiple hybrid original scheduler configuration files;

[0223] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0224] 4. Formalize the unified intermediate representation into a first-order predicate logic formula that can be accepted by the CSP solver;

[0225] 5. Check the satisfiability of the formalized scheduler constraint set using a CSP solver;

[0226] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0227] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0228] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0229] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0230] Example 9: Implementation of an ILP solver in a single scheduler

[0231] like Figure 13 As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on an ILP solver:

[0232] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0233] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0234] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0235] 4. Formalize the unified intermediate representation into a system of equations that can be accepted by the ILP solver;

[0236] 5. Check the satisfiability of the formalized system of equations using an ILP solver;

[0237] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0238] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0239] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0240] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0241] Example 10: Implementation of ILP solver in hybrid scheduler

[0242] like Figure 14 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on an ILP solver:

[0243] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0244] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing hybrid multiple original scheduler configuration file;

[0245] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0246] 4. Formalize the unified intermediate representation into a system of equations that can be accepted by the ILP solver;

[0247] 5. Check the satisfiability of the formalized system of equations using an ILP solver;

[0248] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0249] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0250] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0251] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0252] Example 11: Implementation of a MIP solver in a single scheduler

[0253] like Figure 15 As shown, for a cloud data center using a single scheduler, the present invention provides the following specific implementation steps based on a MIP solver:

[0254] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0255] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing raw scheduler configuration file;

[0256] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0257] 4. Formalize the unified intermediate representation into a set of equations that can be accepted by the MIP solver;

[0258] 5. Check the satisfiability of the formalized system of equations using the MIP solver;

[0259] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0260] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0261] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0262] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0263] Example 12: Implementation of MIP solver in hybrid scheduler

[0264] like Figure 16 As shown, for a cloud data center using a hybrid scheduler, the present invention provides the following specific implementation steps based on a MIP solver:

[0265] 1. The user deploys the present invention in a cluster of a single scheduler to provide scheduling service policy generation services;

[0266] 2. The user enters affinity and / or anti-affinity rules through the service interface, or imports an existing hybrid multiple original scheduler configuration file;

[0267] 3. The rule parsing module of the present invention parses the input rules and converts the rule set into a unified intermediate representation;

[0268] 4. Formalize the unified intermediate representation into a set of equations that can be accepted by the MIP solver;

[0269] 5. Check the satisfiability of the formalized system of equations using the MIP solver;

[0270] 6. If the rules are satisfied, the configuration generation module generates a configuration file that meets the scheduler specifications;

[0271] 7. Deploy the newly generated configuration file to the scheduler cluster to complete the application deployment;

[0272] 8. If the rules cannot be satisfied, the present invention will locate the conflicting applications and conflicting rules, and the service interface will output a conflict report. The user needs to adjust the rules based on the feedback;

[0273] 9. The user resubmits the adjusted rules and repeats steps 3 to 8 until all rules are satisfied.

[0274] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the appended claims.

Claims

1. A cloud scheduler conflict detection method based on formal methods, characterized in that: The method comprises the following steps: Step 1: Input and / or import affinity and / or anti-affinity rules and convert them into constraints in a unified intermediate representation, masking the syntactic and semantic differences in affinity and / or anti-affinity expressions between different schedulers; Step 2: Convert the intermediate representation obtained in step 1 into a format that can be recognized and processed by the solver, and convert the affinity and / or anti-affinity management problem into a formal satisfiability problem; Step 3: Check the satisfiability of the constraint set through the solver. For unsatisfied rules, locate the source file of the conflicting rule and the position of the unsatisfied rule in the source file based on the metadata.

2. The conflict detection method according to claim 1, wherein: In step 1, the constraints of the intermediate representation include entities and metadata; the entities include source entities, operators, and target entities, and the metadata include general metadata and scheduler-specific metadata; The source entity and the target entity define entity objects involved in the constraint; The source entity refers to the application and / or service that initiates the constraint; The target entity refers to the object of the constraint; The operator represents an affinity and / or anti-affinity relationship between the source entity and the target entity; The general metadata includes source information of the affinity and / or anti-affinity rules input and / or imported; The scheduler-specific metadata includes different scheduler-specific configuration files and / or configuration information.

3. The conflict detection method according to claim 1, wherein: In step 2, the solver is divided into two categories: constraint-based solving and mathematical programming-based solving; In the constraint-based solver, Reducing affinity and / or anti-affinity rules to propositional logic, solved by a Boolean satisfiability problem solver and / or a constraint satisfiability problem solver; and / or, Reduce affinity and / or anti-affinity rules to first-order predicate logic, solved by a satisfiability modulo theory solver and / or a constraint satisfiability problem solver; In the solver based on mathematical programming, The affinity and / or anti-affinity rules are reduced to a system of equations, which are solved using a mixed integer programming solver and / or an integer linear programming solver.

4. The conflict detection method according to claim 1, wherein: In step 2, the unified intermediate representation after conversion of the affinity and / or anti-affinity rules configured by the scheduler is expressed in one of three forms: propositional logic, first-order predicate logic, and system of equations. Depending on the specific form of the problem formalization: propositional logic expression, first-order predicate logic expression, or system of equations expression, different solvers are selected for solving. Different formalization methods can achieve the goal of detecting conflicting rules. Different solvers are used to verify the satisfiability of the rules by converting the unified intermediate representation into a representation acceptable to the corresponding solver, and checking the satisfiability of the rules through the solver. For unsatisfied rules, their minimum conflict set is output.

5. The conflict detection method according to claim 1, wherein: Step three also includes automatically generating a scheduler configuration file for the constraint set that is verified to be satisfied; and / or automatically deploying applications that meet the constraint rules.

6. A conflict detection system implementing the conflict detection method according to any one of claims 1 to 5, characterized in that: The conflict detection system includes: an operation and maintenance management module, a rule parsing module, a logic conversion module, a logic verification module, a conflict location module, a configuration generation module, and an automatic deployment control module; The operation and maintenance management module is used to directly enter rules and / or import rule configurations from an existing scheduling system; The rule parsing module is used to receive the rules submitted by the user through the operation and maintenance management module and convert the rules into a unified intermediate representation that can be processed by the system; The logic conversion module is used to convert the parsed unified intermediate representation into a logic expression that can be recognized and processed by the solver; The logic verification module is used to verify the converted logical expression and determine the satisfiability of the rules; The conflict location module is responsible for locating the specific applications and rule contents of the conflict and providing a detailed conflict report; The configuration generation module is used to automatically generate corresponding scheduler configuration files for the rules that have passed the verification; The automatic deployment control module is combined with the scheduling system of the cloud data center to ensure that the entire deployment process automatically and strictly follows the verified constraints, optimizing the deployment quality and consistency of the application.

7. A hardware system for implementing the method according to any one of claims 1 to 5, characterized in that: The hardware system includes: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

9. Application of the conflict detection method according to any one of claims 1 to 5, the conflict detection system according to claim 6, the hardware system according to claim 7, or the computer-readable storage medium according to claim 8 in a single Kubernetes scheduler scenario in a public cloud scenario, a hybrid Kubernetes and Apache Yarn scheduler scenario in a public cloud scenario, a single Kubernetes scheduler scenario in a private cloud scenario, and a hybrid Kubernetes and Apache Yarn scheduler scenario in a private cloud scenario.

Images