Multi-machine-room parallel scheduling method and device, computer equipment and readable storage medium

Abstract

The invention relates to the technical field of automatic control, can be applied to the field of medical treatment and finance, and discloses a multi-machine-room parallel scheduling method and device, computer equipment and a readable storage medium. The method comprises the steps that when a to-be-processed task is received, task attribute information of the to-be-processed task is analyzed; determining a plurality of available machine rooms, and collecting operation state information of each available machine room in real time; obtaining a plurality of preset scheduling strategies, and screening a target scheduling strategy from the plurality of scheduling strategies based on the task attribute information and the operation state information of each available machine room; and according to the target scheduling strategy, selecting a target available machine room from the plurality of available machine rooms, and scheduling the target available machine room to process the to-be-processed task. In a complex business scene, machine room resources can be utilized in a balanced manner, waste of the machine room resources is avoided, normal development of financial businesses and medical businesses and high efficiency of task execution are ensured, and efficient, stable and flexible machine room scheduling requirements of the financial insurance businesses and the medical businesses are met.

Description

Technical Field

[0001] This invention relates to the field of automatic control technology and can be applied to the medical and financial fields. In particular, it relates to a multi-computer room parallel scheduling method, device, computer equipment, and readable storage medium. Background Technology

[0002] In recent years, cloud computing and distributed systems, with their powerful elastic scalability, high reliability, and low cost, have become critical infrastructures for financial and insurance companies and medical institutions to build core business systems and support massive data processing and complex business logic. To meet the stringent requirements of high availability, low latency, and strict disaster recovery for financial and medical businesses, many financial and insurance companies and medical institutions have deployed their business systems in multiple geographically dispersed data centers (computer rooms). This not only ensures the continuous and stable operation of business systems in the event of natural disasters, network failures, or other emergencies in localized areas, guaranteeing the continuity of financial and medical transactions and the safety of customer funds, but also reduces user access latency, improves customer experience, and enhances the company's competitiveness in the market by deploying services closer to the user. In a multi-computer room environment, reasonable data center scheduling can accurately allocate various financial transaction tasks, data analysis tasks, and risk assessment tasks to appropriate computing resources based on the resource conditions and business needs of different data centers. Taking the financial and insurance industry as an example, this allows for optimal resource utilization, efficient task execution, and maximization of overall system performance.

[0003] In related technologies, methods such as round-robin scheduling, random allocation scheduling, and minimum load scheduling are commonly used for data center scheduling. However, the applicant recognizes that, on the one hand, different data centers have different hardware resource configurations and network bandwidths, and the types of services and load levels they carry vary. It is difficult to fully consider these factors, leading to uneven resource utilization. Some data centers have idle and wasted resources, while others are overloaded, affecting task execution efficiency. On the other hand, tasks in financial insurance and medical services often have different priorities, complex dependencies, and strict execution time requirements. This can easily lead to high-priority tasks not being processed in a timely manner, and dependent tasks not being executed in the correct order, resulting in task execution delays and affecting the normal operation of financial or medical services. This makes it difficult to meet the efficient, stable, and flexible data center scheduling requirements of financial insurance and medical services. Summary of the Invention

[0004] This invention provides a multi-datacenter parallel scheduling method, apparatus, computer equipment, and readable storage medium to solve the technical problems that cause high-priority tasks to be unable to be processed in a timely manner, and dependent tasks to be executed in the wrong order, resulting in task execution delays, affecting the normal operation of financial or medical businesses, and making it difficult to meet the needs of financial insurance and medical businesses for efficient, stable, and flexible datacenter scheduling.

[0005] Firstly, a multi-datacenter parallel scheduling method is provided, including:

[0006] When a task to be processed is received, the task attribute information of the task to be processed is parsed.

[0007] Identify multiple available data centers and collect real-time operational status information for each of the available data centers;

[0008] Obtain multiple preset scheduling strategies, and based on the task attribute information and the operating status information of each available data center, filter the target scheduling strategy from the multiple scheduling strategies;

[0009] According to the target scheduling strategy, a target available data center is selected from the multiple available data centers, and the target available data center is scheduled to process the task to be processed.

[0010] Secondly, a multi-datacenter parallel scheduling device is provided, comprising:

[0011] The parsing module is used to parse the task attribute information of the task to be processed when a task to be processed is received.

[0012] The data acquisition module is used to identify multiple available data centers and collect the operational status information of each available data center in real time.

[0013] The filtering module is used to obtain multiple preset scheduling strategies, and filter the target scheduling strategy from the multiple scheduling strategies based on the task attribute information and the operating status information of each available data center.

[0014] The scheduling module is used to select a target available data center from the plurality of available data centers according to the target scheduling strategy, and to schedule the target available data center to process the task to be processed.

[0015] Thirdly, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the multi-data center parallel scheduling method as described in any of the first aspects above.

[0016] Fourthly, a readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of any of the multi-data center parallel scheduling methods described in the first aspect above.

[0017] In the above-mentioned scheme implemented by the multi-datacenter parallel scheduling method, device, computer equipment, and readable storage medium, when a task to be processed is received, the task attribute information of the task to be processed is parsed, multiple available datacenters are determined, the operating status information of each available datacenter is collected in real time, multiple preset scheduling strategies are obtained, and a target scheduling strategy is selected from multiple scheduling strategies based on the task attribute information and the operating status information of each available datacenter. According to the target scheduling strategy, a target available datacenter is selected from multiple available datacenters, and the target available datacenter is scheduled to process the task to be processed. In this invention, the task attribute information of the task to be processed and the operating status information of each available datacenter are combined, and a suitable target scheduling strategy is selected from multiple scheduling strategies for datacenter scheduling. This makes it possible to consider not only the task's own attributes but also the real-time status of the datacenter during the datacenter scheduling process, which can adapt to complex business scenarios. It can make balanced use of datacenter resources even in complex business scenarios, avoid waste of datacenter resources, ensure the normal operation of financial and medical businesses and the high efficiency of task execution, and meet the needs of financial insurance and medical businesses for efficient, stable, and flexible datacenter scheduling. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of an application environment for a multi-data center parallel scheduling method according to an embodiment of the present invention;

[0020] Figure 2 This is a flowchart illustrating a multi-data center parallel scheduling method according to an embodiment of the present invention;

[0021] Figure 3 This is a flowchart illustrating a specific implementation of step S10;

[0022] Figure 4 This is a flowchart illustrating a specific implementation of step S20;

[0023] Figure 5 This is a flowchart illustrating a specific implementation of step S30;

[0024] Figure 6This is a flowchart illustrating a specific implementation of step S31;

[0025] Figure 7 This is a schematic diagram of a multi-data center parallel scheduling device in one embodiment of the present invention;

[0026] Figure 8 This is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;

[0027] Figure 9 This is another structural schematic diagram of a computer device according to one embodiment of the present invention. Detailed Implementation

[0028] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0029] The multi-datacenter parallel scheduling method provided in this invention can be applied to applications such as... Figure 1 In this application environment, the client communicates with the server via a network. The server receives tasks from the client, parses the task attribute information, identifies multiple available data centers, collects the real-time operating status information of each available data center, obtains multiple preset scheduling strategies, and, based on the task attribute information and the operating status information of each available data center, selects a target scheduling strategy from among the multiple scheduling strategies. Following the target scheduling strategy, it selects a target available data center from among the multiple available data centers and schedules the target available data center to process the task. In this invention, the task attribute information of the task to be processed and the operating status information of each available data center are combined, and a suitable target scheduling strategy is selected from multiple scheduling strategies for data center scheduling. This ensures that the data center scheduling process considers not only the task's own attributes but also the real-time status of the data center, adapting to complex business scenarios. It enables balanced utilization of data center resources even in complex business scenarios, avoiding waste of data center resources, ensuring the normal operation of financial and medical businesses and high efficiency in task execution, and meeting the needs of financial insurance and medical businesses for efficient, stable, and flexible data center scheduling. The client can be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The server can be implemented using a standalone server or a server cluster consisting of multiple servers. The invention will now be described in detail through specific embodiments.

[0030] Please see Figure 2 As shown, Figure 2A flowchart illustrating a multi-data center parallel scheduling method provided in an embodiment of the present invention includes the following steps:

[0031] S10: When a task to be processed is received, parse the task attribute information of the task to be processed.

[0032] The multi-data center parallel scheduling method provided by this invention can be applied to data center scheduling systems in various application scenarios, such as financial insurance scenarios, medical scenarios, etc. Data center scheduling systems are usually implemented through a server, which can receive pending tasks from clients in real time.

[0033] When the data center scheduling system receives a task to be processed, it parses the task attribute information. Task attribute information is a series of data describing the characteristics and requirements of the task, covering the task type (e.g., data computation, data storage, or data transmission); task priority (indicating the task's urgency among other tasks); and the resource requirements (including computing and storage resources), which determine the resource consumption of the data center during the task's execution. By extracting and analyzing this key information, the data center scheduling system can comprehensively understand the specific details of the task to be processed.

[0034] For example, in financial insurance operations, if a claims calculation task is received, the data center scheduling system will determine that the task belongs to the data calculation type, set its priority to high according to business rules, and finally parse the task's attribute information. As another example, in medical insurance business processing scenarios, if a task for reviewing and approving chronic disease benefits is received, the data center scheduling system will determine that the task belongs to the data review and logical judgment type. Based on the requirements of medical insurance policies regarding the chronic disease identification process and timeliness, it will set its priority to high, then parse the task to obtain its attribute information, including the size of the detailed medical records submitted by the patient, the complex judgment logic involving multiple chronic diseases, and the diversity of data formats due to the data sources being different levels of medical institutions. In this way, accurately parsing the task attribute information provides a foundation for subsequent data center scheduling, enabling the scheduling process to fully consider the characteristics and needs of the task itself, ensuring that tasks of different types and priorities are processed appropriately.

[0035] Among them, such as Figure 3 As shown, step S10, which involves parsing the task attribute information of the task to be processed, includes the following steps:

[0036] S11: Parse the task metadata of the task to be processed to obtain multiple attribute parameters.

[0037] In this embodiment of the invention, the data center scheduling system parses the task metadata of the task to be processed, obtaining multiple attribute parameters. Task metadata is basic descriptive information about the task to be processed, containing various key characteristics of the task. When the data center scheduling system receives a task to be processed, it parses the task metadata. During this process, multiple attribute parameters are extracted, including task type, task priority, task dependencies, and task execution time limit.

[0038] Among them, task type refers to the business category to which the task belongs. For example, in the financial insurance field, it may be a claims calculation task, a policy generation task, or a customer risk assessment task. Task priority indicates the urgency of the task among many tasks, and is divided into high, medium, and low levels. High-priority tasks need to be processed first to ensure the timeliness and accuracy of business. Task dependency refers to the execution order or data interaction relationship between the current task and other tasks. For example, some claims calculation tasks may depend on the integrity check task of the claims application data submitted by customers in the early stage. Task execution time limit specifies how long the task must be completed to ensure the smooth operation of the business process.

[0039] For example, in financial insurance operations, a high-priority claims calculation task, explicitly defined as claims calculation with a high priority, may depend on claims application data entry and review tasks, with an execution time limit of 24 hours. By parsing the task's metadata, these attribute parameters can be accurately extracted. Similarly, in medical insurance operations, a high-priority out-of-town inpatient expense reimbursement review task, explicitly defined as expense review with a high priority, may depend on the completion of inpatient expense details uploaded by out-of-town medical institutions and the verification of insured individuals' basic information in the local medical insurance system, with an execution time limit of 48 hours. By parsing the task's metadata, these key attribute parameters can be accurately extracted. Thus, by deeply analyzing task metadata, the data center scheduling system can comprehensively obtain multiple key attribute parameters of the task, laying the foundation for accurate subsequent organization of task attribute information, enabling the scheduling system to clearly understand the essential characteristics and business requirements of the task.

[0040] S12: Organize multiple attribute parameters to obtain the task attribute information of the task to be processed.

[0041] After acquiring multiple attribute parameters, the data center scheduling system systematically organizes these parameters. Specifically, it categorizes, sorts, and performs correlation analysis on the attribute parameters according to preset rules and logic. For example, parameters such as task type, task priority, task dependencies, and task execution time limits are integrated according to a certain format to form a structured set of task attribute information. This set clearly presents the complete picture of the task to be processed, enabling the data center scheduling system to quickly and accurately obtain key task information.

[0042] For example, in financial insurance operations, the task attribute information obtained after processing, such as the claims calculation task mentioned earlier, will clearly show that the task is a claims calculation type, has high priority, depends on the data entry and review of claims application materials, and has a 24-hour execution deadline. Similarly, in medical insurance operations, the task attribute information for the received outpatient special disease benefit assessment task, after processing, will clearly indicate that the task is a qualification assessment type, has high priority, depends on the review of outpatient special disease application materials submitted by the insured, and must be completed within 7 working days. By processing multiple attribute parameters, scattered information is integrated into structured task attribute information, facilitating unified management and analysis by the scheduling system in the data center. This provides clear and accurate data support for subsequent data center scheduling based on task attribute information, helping to improve the scientific and rational nature of scheduling.

[0043] S20: Identify multiple available data centers and collect real-time operational status information for each available data center.

[0044] The data center scheduling system first identifies multiple available data centers. An available data center is one that currently meets the operational requirements and is capable of handling pending tasks. After identifying available data centers, monitoring agents deployed in each data center can collect real-time operational status information. This operational status information includes several aspects, such as CPU utilization of servers within the data center, reflecting the current computing load; memory utilization, indicating the amount of memory resources remaining that can be used to process tasks; and network bandwidth utilization, indicating the level of data transmission activity on the data center's network.

[0045] For example, in financial insurance operations, real-time monitoring of the operational status of data centers handling large volumes of customer transaction data can promptly identify server performance bottlenecks, network congestion, and other issues. Similarly, in medical insurance operations, real-time monitoring of the operational status of data centers handling massive amounts of medical expense settlement data for insured individuals can promptly detect issues such as database storage nearing its limit or system response delays. If a data center's CPU utilization is found to be excessively high, new insurance application or medical expense settlement tasks can be avoided by assigning them to that data center. Instead, a data center with relatively less resource consumption can be selected, ensuring that online insurance applications or settlements can respond quickly to customer requests and improve customer experience. Thus, by monitoring the operational status of available data centers in real time, we can dynamically understand the resource usage and performance of data centers, providing accurate data support for selecting appropriate data centers to process tasks. This avoids assigning tasks to resource-constrained or poorly performing data centers, improving the efficiency and stability of task processing.

[0046] Among them, such as Figure 4 As shown, step S20, which involves identifying multiple available data centers and collecting the operational status information of each available data center in real time, includes the following steps:

[0047] S21: Query the multiple data centers that have been successfully connected and are operating normally as multiple available data centers.

[0048] The data center scheduling system can be configured with a data center management database, which records detailed information about all data centers connected to the system, including their access and operational status. When it is necessary to determine available data centers, the system queries the database to select those that have successfully connected and are operating normally. Successful access means that the data center has established a stable and reliable communication connection with the scheduling system, enabling it to receive and execute scheduling commands. Operating normally indicates that the data center's hardware, network infrastructure, and other components are functioning correctly and can perform business operations as usual.

[0049] For example, in the financial insurance business, a large insurance group has multiple data center computer rooms across the country. Through the computer room management database, it can quickly and accurately query the currently available computer rooms. In the medical insurance business, a department has deployed multiple medical insurance data center computer rooms in different cities within the province. With the help of the medical insurance computer room management database, it can quickly and accurately query the computer rooms that are currently operating normally and can handle new business processing, providing a basis for subsequent task scheduling, avoiding the assignment of tasks to computer rooms that are not working properly, and improving the accuracy and reliability of task scheduling.

[0050] S22: For each available data center, collect the total running time and idle time of the processors in the available data center in real time, and calculate the processor utilization rate of the available data center using the total running time and idle time of the processors.

[0051] The data center scheduling system deploys a monitoring agent program in each available data center. This program can collect the processor's operating data in real time. For each available data center, the data center scheduling system uses this program to collect the total processor running time and processor idle time in real time. The total processor running time refers to the total working time from processor startup to the current moment, reflecting the overall workload of the processor; specifically, it can be the total CPU running time. The processor idle time refers to the total time the processor is in an idle state within a certain time interval, that is, the time when it does not execute any tasks; specifically, it can be the CPU idle time.

[0052] After obtaining this data, the data center scheduling system can calculate the processor utilization rate of each available data center using the following formula 1:

[0053] Formula 1: Processor utilization = (Total processor runtime - Processor idle time) / Total processor runtime × 100%

[0054] For example, if the total running time of processors in an available data center is 100 hours and the idle time is 20 hours, then the processor utilization rate of the data center is (100-20) / 100×100%=80%. By accurately calculating the processor utilization rate, the data center scheduling system can intuitively understand the busy level of each available data center processor, providing an important reference for subsequent judgment on whether the data center's computing resources are sufficient, and helping to rationally allocate computationally intensive tasks.

[0055] S23: Real-time statistics on available data center idle memory, using the total available data center memory and idle memory to calculate the memory utilization rate of available data centers.

[0056] In this embodiment of the invention, the data center scheduling system, also utilizing a monitoring agent deployed in available data centers, can statistically analyze the idle memory within the data center in real time. By combining the total available data center memory with the idle memory, the system can calculate the memory utilization rate of the available data center. The total data center memory refers to the total amount of physical memory configured in the data center, which determines the amount of data and task scale the data center can process simultaneously.

[0057] After obtaining the available free memory in the available computer rooms, the computer room scheduling system can calculate the memory utilization rate of each available computer room according to the following formula 2:

[0058] Formula 2: Memory utilization rate = (Total memory in the data center - Free memory) / Total memory in the data center × 100%

[0059] For example, if a data center has a total available memory of 100GB and currently has 30GB of free memory, then the memory utilization rate of the data center is (100-30) / 100×100%=70%. In this way, the data center scheduling system can accurately grasp the memory utilization rate and understand the data center's memory resource usage. For financial and insurance business tasks with high memory requirements, such as large-scale data storage and analysis tasks, it can rationally select data centers with sufficient memory resources, avoiding task processing failures or performance degradation due to insufficient memory.

[0060] S24: Periodically send test packets to available data centers and record the round-trip time of the test packets. Use the round-trip time to calculate the network latency of available data centers.

[0061] The data center scheduling system sends test packets to each available data center at preset time intervals. A test packet is a specific data packet used to test network connection performance. Once an available data center receives a test packet, it immediately returns it to the data center scheduling system. The system records the round-trip time from sending to receiving the test packet; this round-trip time reflects network latency. Network latency refers to the time required for data to travel from the sender to the receiver and back, directly affecting data transmission efficiency and real-time performance.

[0062] For example, the data center scheduling system sends a test packet to available data centers every 5 minutes. By recording and statistically analyzing round-trip times multiple times, the average network latency is calculated, allowing for timely understanding of the network latency of each available data center. For financial and insurance businesses with high real-time network requirements, such as online transaction services, data centers with low network latency can be selected to ensure that transaction data can be transmitted quickly and accurately, guaranteeing the smooth progress of transactions. For medical insurance businesses with high requirements for network stability and real-time performance, such as direct settlement of medical expenses in different locations, data centers with sufficient network bandwidth and low packet loss rates can be selected to ensure that medical insurance settlement data can be transmitted in a timely and error-free manner, ensuring that insured individuals can successfully complete expense settlement when seeking medical treatment in different locations.

[0063] S25: Query the number of pending tasks in available data centers, and calculate the task queue length of available data centers based on the number of pending tasks.

[0064] The data center scheduling system establishes a communication connection with the task management system of each available data center. Through this connection, the system can query the current number of pending tasks for each available data center in real time. The task queue length refers to the number of tasks currently waiting in the data center's task queue, reflecting the data center's task load. For example, if the task management system of an available data center shows 20 tasks waiting to be processed, then the task queue length for that data center is 20. Understanding the task queue length helps the data center scheduling system determine the task load level of a data center, avoiding assigning new tasks to data centers with excessively long task queues and high processing pressure, ensuring timely task processing, and improving overall business processing efficiency.

[0065] Taking policy review tasks in the financial insurance field as an example, if the task queue in a certain data center is too long, the data center scheduling system can allocate newly received policy review tasks to other data centers with shorter task queues, thereby speeding up the review process. Taking medical insurance reimbursement document review tasks in the medical insurance field as an example, if a certain data center has too many pending reimbursement document tasks and a long task queue, the medical insurance data center scheduling system can intelligently allocate newly received reimbursement document review tasks to other data centers with less backlog and shorter queues, thereby improving overall review efficiency and allowing insured individuals to receive reimbursement payments faster.

[0066] S26: Use processor utilization, memory utilization, network latency, and task queue length as operational status information for available data centers.

[0067] The data center scheduling system integrates the processor utilization, memory utilization, network latency, and task queue length data calculated and statistically analyzed in the previous steps to form complete operational status information for each available data center. This operational status information comprehensively reflects the current performance and load of the available data centers, providing rich decision-making basis for subsequent task scheduling.

[0068] By integrating information from multiple sources to form complete operational status information, the data center scheduling system can more comprehensively and accurately assess the real-time status of each available data center. This enables more scientific and rational scheduling decisions, improving the efficiency and quality of parallel scheduling across multiple data centers and better meeting the needs of the financial and insurance industries for efficient, stable, and flexible data center scheduling. For example, during peak periods for financial and insurance businesses, by comprehensively analyzing this operational status information, the data center scheduling system can rationally allocate different types of tasks to the most suitable data centers, ensuring the smooth operation of all businesses. Similarly, during peak periods for medical businesses, such as the flu season when hospital registration, consultations, and examinations surge, by comprehensively analyzing the server load, network bandwidth usage, and remaining storage space of each medical data center, the medical data center scheduling system can rationally allocate different types of tasks, such as image diagnosis tasks, medical record entry tasks, and drug dispensing tasks, to the most suitable data centers, ensuring the efficient and orderly operation of all medical services.

[0069] S30: Obtain multiple preset scheduling strategies, and select the target scheduling strategy from the multiple scheduling strategies based on task attribute information and the operating status information of each available data center.

[0070] The data center scheduling system pre-stores multiple scheduling strategies. These strategies are sets of rules formulated based on different business scenarios and needs, guiding how to allocate tasks to suitable data centers. For example, a scheduling strategy could prioritize allocating high-priority tasks to data centers with sufficient resources and stable performance. After obtaining task attribute information and the operational status information of each available data center, the data center scheduling system analyzes and compares this information. Based on factors such as task type and priority in the task attribute information, and combined with indicators such as CPU utilization and memory utilization in the operational status information of each available data center, the system selects the target scheduling strategy that best suits the current task and data center status from multiple scheduling strategies.

[0071] For example, in financial insurance operations, for a high-priority fund transaction data storage task, the data center scheduling system analyzes and finds that a certain data center has high-performance storage devices and ample available storage resources. Based on the pre-defined scheduling strategy's allocation rules for high-priority data storage tasks, the system selects the target scheduling strategy to allocate the task to that data center, ensuring timely and accurate processing and storage of fund transaction data and guaranteeing the stable operation of the financial market. In healthcare operations, for a high-priority emergency patient electronic medical record rapid storage and retrieval task, the data center scheduling system analyzes and finds that a certain data center has fast server processing speeds, excellent storage device read / write performance, and sufficient storage resources. Based on the pre-defined scheduling strategy's allocation rules for high-priority medical data storage tasks, the system selects the target scheduling strategy to allocate the task to that data center, ensuring timely and accurate processing and storage of emergency patient electronic medical records, providing strong data support for subsequent emergency treatment.

[0072] In this way, by comprehensively considering task attribute information and data center operation status information, the target scheduling strategy can be accurately selected from multiple scheduling strategies, making scheduling decisions more scientific and reasonable, adapting to the complex needs of different business scenarios, and improving the efficiency of data center resource utilization and task processing.

[0073] Among them, such as Figure 5 As shown, step S30, which involves selecting a target scheduling strategy from multiple scheduling strategies based on task attribute information and the operational status information of each available data center, includes the following steps:

[0074] S31: Check among multiple scheduling policies whether there is a scheduling policy that matches the task attribute information.

[0075] The data center scheduling system first analyzes task attribute information, including task priority, execution time limit, dependencies, and type. The system then searches through multiple pre-defined scheduling strategy libraries to determine if a matching strategy exists. This allows for the rapid utilization of existing targeted scheduling strategies, improving scheduling efficiency and ensuring tasks are allocated to the appropriate data center in the most suitable way, meeting the timeliness and accuracy requirements of financial insurance and medical services. It's important to note that when a matching strategy is identified among multiple options, the system will directly use that strategy as the target scheduling strategy.

[0076] Among them, such as Figure 6 As shown, step S31, which involves querying among multiple scheduling policies to see if a scheduling policy matches the task attribute information, includes the following steps:

[0077] S311: Read the task priority and task execution time limit from the task attribute information.

[0078] The data center scheduling system first extracts two key parameters from the task attribute information: task priority and task execution time limit. Task priority measures the importance and urgency of a task; for example, in financial insurance, urgent customer claims tasks may have a higher priority. The task execution time limit specifies how long a task must be completed; for example, some real-time transaction processing tasks need to be completed within a short period. By clearly defining the key attributes of the tasks, a foundation is laid for selecting appropriate scheduling strategies based on different attributes, ensuring that important and urgent tasks are prioritized and meeting the timeliness and importance requirements of financial insurance operations.

[0079] S312: When the task priority is less than the priority threshold and the task execution time is greater than or equal to the time limit threshold, read the task dependency relationship from the task attribute information.

[0080] Specifically, when the task priority is greater than or equal to the priority threshold or the task execution time limit is less than the time limit threshold, the scheduling strategy among multiple scheduling strategies used to indicate that the data center is scheduled according to the task priority is used as the scheduling strategy that matches the task attribute information.

[0081] The data center scheduling system determines task priority and execution time based on preset priority and time limit thresholds. When a task has a low priority and a relatively long execution time, it indicates that the task's priority is not high and the timeframe is not overly tight. Therefore, no priority- and time-limit-related scheduling strategies are needed. The system then further examines task dependencies to determine whether data center scheduling should be performed based on these dependencies. Task dependencies indicate whether a task depends on the completion of other tasks. For example, in financial insurance, certain complex insurance product pricing tasks may depend on prior data collection and analysis tasks.

[0082] If the task priority is greater than or equal to the priority threshold or the task execution time limit is less than the time limit threshold, it indicates that the task has a high degree of urgency or importance. In this case, the data center scheduling system will select the scheduling strategy that indicates data center scheduling according to task priority from among multiple scheduling strategies as the scheduling strategy that matches the task attribute information. In this way, the data center will be scheduled for the task to be processed according to its priority. This realizes the flexible selection of scheduling strategies based on the different urgency and importance of tasks, ensuring that high-priority and short-time tasks can be processed first, and guaranteeing the smooth completion of critical tasks in financial insurance and medical services.

[0083] S313: When the task dependency relationship indicates that there are no tasks that the task to be processed depends on, read the task type from the task attribute information.

[0084] In this embodiment of the invention, after obtaining the task dependencies, the data center scheduling system parses the task dependencies. When a task has no dependency, it means that the task to be processed does not depend on another task. Therefore, the data center scheduling system continues to read the task type and, referring to the task type, continues to select a scheduling strategy for the task to be processed. The task type is used to distinguish the specific nature and characteristics of the task. For example, in financial insurance business, task types may include data processing tasks, model training tasks, report generation tasks, etc.; in medical business, task types may cover medical image analysis tasks, patient electronic medical record organization tasks, medical expense accounting report generation tasks, etc. Different tasks have different requirements for data center resources and processing procedures.

[0085] If the task dependency relationship indicates that the task to be processed has tasks that it depends on, it means that the execution of the task needs to wait for other tasks to complete. In this case, the data center scheduling system will use the scheduling strategy among multiple scheduling strategies that indicates data center scheduling according to task dependency relationship as the scheduling strategy that matches the task attribute information. This fully considers the dependency relationship between tasks, reasonably arranges the task execution order and data center allocation, ensures that the task dependency relationship is satisfied in the scheduling, avoids processing errors or delays due to task dependency issues, and ensures the smooth operation of financial insurance business processes.

[0086] Taking financial insurance business as an example, payment transactions and refund transactions are two different tasks, but they are interdependent. A refund transaction will only occur if a payment transaction exists. Therefore, when scheduling data centers for refund transaction tasks, the data center that previously processed the payment transaction task will be prioritized. Similarly, in medical business, patient appointment registration and appointment cancellation are two different tasks, but they are interdependent. Only after the appointment registration is completed will the subsequent cancellation request arise. Therefore, when scheduling data centers for cancellation registration tasks, the data center that previously processed the patient's appointment registration task will be prioritized to ensure accurate information flow and smooth business processing.

[0087] S314: If the task type indicates that the amount of task data to be processed is less than or equal to the task data volume threshold, then the task type is parsed to determine whether the task to be processed is allowed to be processed in chunks.

[0088] In this embodiment of the invention, the data center scheduling system identifies the task data volume of the task to be processed according to the task type indication, and judges the task data volume based on a preset task data volume threshold. When the task data volume is small, the data center scheduling system further parses the task type to determine whether the task allows for sharding. Sharding refers to splitting a large task into multiple smaller tasks, processing them separately on different data centers or computing nodes, and finally merging the results. For example, for some large-scale data analysis tasks, allowing sharding can greatly improve processing efficiency.

[0089] If the task type indicates that the data volume of the task to be processed exceeds the task data volume threshold, it means that the task data volume is large. To avoid large amounts of data being transmitted over long distances for extended periods, the data center scheduling system will use the scheduling strategy that indicates data center scheduling based on the geographical location of the task's origin as the matching scheduling strategy with the task attribute information. This ensures that the task is processed in the data center closest to it. In this way, based on the size of the task data volume and the characteristics of the task type, a reasonable processing method is selected. For large-volume tasks, geographical location factors are considered during scheduling; for small-volume tasks, it is further determined whether they can be processed in chunks. This improves the flexibility and efficiency of task processing, meeting the processing needs of financial and insurance businesses for tasks of different sizes.

[0090] S315: When it is determined that the task to be processed is not allowed to be processed in segments, determine that there is no scheduling policy among the multiple scheduling policies that matches the task attribute information.

[0091] In this embodiment of the invention, if a task is not allowed to be processed in segments, it means that the task needs to be processed as a whole. At this time, the data center scheduling system may not be able to find a strategy that completely matches the task attribute information among the current preset multiple scheduling strategies. The attributes of the task to be processed are not too strict for the scheduling of the data center. Therefore, the data center scheduling system will determine that there is no scheduling strategy that matches the task attribute information among the multiple scheduling strategies, so that it can continue to refer to the data center operation information to schedule multiple available data centers.

[0092] When it is determined that the task to be processed can be processed in segments, the data center scheduling system will use the scheduling strategy among multiple scheduling strategies that indicates data center scheduling according to the segmentation results of the task to be processed as the scheduling strategy that matches the task attribute information, split the task to be processed into multiple sub-tasks, and distribute them to multiple data centers in parallel for execution according to the scheduling strategy, thereby improving the accuracy and efficiency of task processing and ensuring the quality of financial and insurance business.

[0093] S32: When the query determines that there is no scheduling policy that matches the task attribute information among multiple scheduling policies, extract the processor utilization rate and memory utilization rate corresponding to each available data center from the running status information of each available data center.

[0094] If step S31 yields no results, the data center scheduling system will turn its attention to the operational status information of available data centers. This operational status information includes key indicators such as processor utilization, memory utilization, network latency, and task queue length. At this point, the data center scheduling system first extracts the processor utilization and memory utilization of each available data center from this information. Processor utilization reflects the workload of the data center's processors, while memory utilization reflects the usage of the data center's memory resources.

[0095] For example, during peak periods for financial and insurance business, some data centers may experience high processor and memory usage due to processing large amounts of data. Obtaining these key metrics provides fundamental data for assessing the data center's load, allowing for more rational task allocation and preventing tasks from being assigned to overloaded data centers, thus ensuring the stable operation of financial and insurance services. Similarly, during peak periods for medical visits, some data centers may experience high processor and disk I / O usage due to simultaneously processing numerous patients' electronic medical record inquiries, examination report generation, and image data retrieval. Obtaining these key metrics provides fundamental data for assessing the data center's load, enabling more rational allocation of newly received tasks such as remote medical consultation data transmission, preventing tasks from being assigned to overloaded data centers, and ensuring the orderly and stable operation of various medical services.

[0096] S33: Based on the processor utilization and memory utilization of each available data center, query whether there is an overloaded data center among multiple available data centers.

[0097] The data center scheduling system assesses the processor and memory utilization of each available data center based on preset processor and memory utilization thresholds. If the processor utilization or memory utilization of an available data center exceeds the threshold, it is considered an overloaded data center. For example, assuming a processor utilization threshold of 80% and a memory utilization threshold of 70%, a data center with a processor utilization of 85% or a memory utilization of 75% will be identified as an overloaded data center. This system promptly identifies overloaded data centers, preventing new tasks from being assigned to them and avoiding task processing failures or performance degradation due to data center overload, thus ensuring efficient processing of financial and insurance business.

[0098] S34: If the query determines that there are no overloaded data centers among the multiple available data centers, then select the scheduling strategy used for weighted round-robin scheduling of the multiple available data centers as the target scheduling strategy from the multiple scheduling strategies.

[0099] When step S33 determines that there are no overloaded data centers, it indicates that the load of each available data center is relatively balanced. At this time, the data center scheduling system selects a weighted round-robin scheduling strategy as the target scheduling strategy. Weighted round-robin scheduling assigns different weights to each available data center based on its performance indicators (such as processor performance, memory capacity, etc.), and then distributes tasks to each data center in sequence according to the weight ratio.

[0100] For example, if data center A has a performance weight of 0.4 and data center B has a performance weight of 0.6, then data center B will have a relatively higher probability of receiving tasks during task allocation. Thus, under load balancing conditions, weighted round-robin scheduling can fully utilize the resources of each data center, achieving reasonable resource allocation, improving overall task processing efficiency, and meeting the needs of financial and insurance businesses for large-scale task processing.

[0101] S35: If the query determines that there is an overloaded data center among multiple available data centers, then select the scheduling strategy among multiple scheduling strategies that is used to select the available data center with the smallest load among the multiple available data centers for scheduling as the target scheduling strategy.

[0102] When step S33 determines that an overloaded data center exists, the weighted round-robin scheduling strategy cannot be applied to the data center scheduling. Therefore, the data center scheduling system will select the minimum load scheduling strategy as the target scheduling strategy. This strategy will compare the load status of each available data center in real time and select the data center with the lowest processor utilization and memory utilization to allocate new tasks.

[0103] For example, if data center C has the lowest processor utilization rate (30%) and memory utilization rate (20%) among multiple available data centers, then the tasks to be processed will be assigned to data center C. This effectively avoids assigning tasks to overloaded data centers, ensuring that tasks can be processed quickly in data centers with lower loads, improving task response speed and processing success rate, and guaranteeing the smooth operation of financial and insurance businesses.

[0104] S40: According to the target scheduling strategy, select the target available data center from multiple available data centers, and schedule the target available data center to process the task to be processed.

[0105] After selecting the target scheduling strategy, the data center scheduling system will choose the target available data center from multiple available data centers according to the specific rules of the strategy. The target scheduling strategy clearly specifies how to determine the most suitable data center for handling the current task based on task attribute information and data center operating status information. For example, it prioritizes data centers with abundant remaining resources and stable performance, in descending order of task priority.

[0106] After selecting a target available data center, the data center scheduling system sends scheduling instructions to that data center, which then initiates the corresponding processing program to handle the pending tasks. For example, in financial insurance operations, if the target scheduling strategy is to allocate high-priority claims calculation tasks to data centers with abundant computing resources and stable networks, the data center scheduling system, based on this strategy, selects a data center as the target available data center and sends it detailed information and processing instructions for the claims calculation tasks. Upon receiving the instructions, the data center immediately starts the calculation program to analyze and calculate the claims data.

[0107] In this way, by accurately selecting available data centers and scheduling tasks based on the selected target scheduling strategy, it is possible to ensure that tasks are processed efficiently in the most suitable data center environment, give full play to the advantages of data center resources, avoid resource waste, ensure the normal operation of financial and medical services, improve the efficiency and quality of task execution, and promote the healthy development of financial insurance and medical services.

[0108] In another optional implementation, during the operation of the task scheduling system, the data center scheduling system continuously monitors the processing status of tasks to be processed and the data center status of the target available data center in real time. The processing status of a task to be processed refers to whether the task is executed smoothly as expected in the target available data center. If a situation occurs that prevents the task from being completed normally, such as a calculation error or program crash, it is judged as a processing failure. The data center status of the target available data center includes the operation status of the data center's hardware facilities, network connection status, etc. When the data center experiences hardware failure, network interruption, or other problems that prevent it from continuing to provide normal computing resources and services for tasks, the data center status will change to an unavailable state.

[0109] When a task fails to process or the status of a target available data center changes to unavailable, the data center scheduling system will immediately initiate a rescheduling process. This involves re-collecting the operational status information of each available data center, and then, based on the task attribute information and the operational status information of each available data center, selecting a new target scheduling strategy from multiple scheduling policies. Following the new target scheduling strategy, a new target available data center is selected from multiple available data centers, and the new target available data center is scheduled to process the task again.

[0110] In this way, when anomalies occur in task processing or the status of the data center changes, the data center scheduling system can respond and adjust in a timely manner. By re-collecting information and filtering strategies, it can reselect a suitable data center for the task to be processed, thereby improving the reliability and stability of task processing. It avoids the interruption of the entire business process due to a single data center failure or task processing failure, ensuring the continuous operation of the system and the normal operation of the business. It is especially suitable for scenarios with high requirements for the timeliness and accuracy of task processing, ensuring high availability of the system.

[0111] In another optional implementation, the data center scheduling system is equipped with a scheduled task mechanism. The scheduled task reads a preset key-value database at pre-set time intervals, such as every 5 minutes, to check if an abnormal data center identifier is stored. The preset key-value database is a data storage structure that stores data in key-value pairs, where the key is a unique identifier and the value is the data information associated with that key. In this embodiment, the preset key-value database stores an abnormal data center identifier, which is a symbol or code used to uniquely identify a data center that has experienced an anomaly.

[0112] There are two ways to obtain abnormal data center identifiers: One is that the data center itself has anomaly detection capabilities. When hardware devices (such as servers and storage devices) within the data center malfunction, network connections are interrupted, or software systems crash, the data center will automatically detect these anomalies and report its abnormal data center identifier to a preset key-value database for storage. The other method is for the business side to initiate a probe task. The business side will periodically send probe requests to various data centers to check their availability and operational status. If the probe task finds that a data center cannot respond to the request normally or the response result is abnormal, the business side will determine that the data center is abnormal and report its abnormal data center identifier to the preset key-value database.

[0113] After retrieving the abnormal data center identifier from the preset key-value database, the data center scheduling system will query multiple available data centers to identify the abnormal data center as the one indicated by the identifier. This abnormal data center will then be taken offline. Taking it offline means removing it from the currently available data center set and ceasing to provide service for new task assignments, thus preventing tasks from being assigned to problematic data centers and causing processing failures. Simultaneously, the data center scheduling system will allocate other available data centers (excluding the abnormal data center) to process the tasks assigned to the abnormal data center, ensuring successful task execution.

[0114] In this way, by periodically reading the abnormal data center identifiers from the preset key-value database, abnormal situations in data centers can be detected in a timely manner, whether reported by the data center itself or detected by the business side. Abnormal data centers can be taken offline and tasks can be rescheduled, which can effectively avoid assigning tasks to faulty data centers, improve the success rate of task processing and the stability of the system. At the same time, this automated processing method reduces manual intervention, improves the efficiency and timeliness of fault handling, and ensures the normal operation of the entire data center scheduling system.

[0115] As can be seen, the above solution combines the task attribute information of the task to be processed with the operating status information of each available data center, and selects a suitable target scheduling strategy from multiple scheduling strategies for data center scheduling. This ensures that the data center scheduling process considers not only the task's own attributes but also the real-time status of the data center, adapting to complex business scenarios. It enables balanced utilization of data center resources even in complex business scenarios, avoids waste of data center resources, ensures the normal operation of financial and medical businesses and high efficiency of task execution, and meets the needs of financial insurance and medical businesses for efficient, stable and flexible data center scheduling.

[0116] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0117] In one embodiment, a multi-datacenter parallel scheduling device is provided, which corresponds one-to-one with the multi-datacenter parallel scheduling method described in the above embodiments. For example... Figure 7 As shown, the multi-datacenter parallel scheduling device includes a parsing module 701, a data acquisition module 702, a filtering module 703, and a scheduling module 704. Detailed descriptions of each functional module are as follows:

[0118] The parsing module 701 is used to parse the task attribute information of the task to be processed when a task to be processed is received;

[0119] The data acquisition module 702 is used to identify multiple available computer rooms and collect the operating status information of each available computer room in real time.

[0120] The filtering module 703 is used to obtain multiple preset scheduling strategies, and filter the target scheduling strategy from the multiple scheduling strategies based on the task attribute information and the operating status information of each available computer room.

[0121] The scheduling module 704 is used to select a target available data center from the plurality of available data centers according to the target scheduling strategy, and schedule the target available data center to process the task to be processed.

[0122] In one embodiment, the parsing module 701 is used to parse the task metadata of the task to be processed to obtain multiple attribute parameters, including task type, task priority, task dependency relationship, and task execution time limit; and to organize the multiple attribute parameters to obtain the task attribute information of the task to be processed.

[0123] In one embodiment, the acquisition module 702 is used to query multiple data centers that have been successfully connected and are operating normally as the multiple available data centers; for each available data center, the module collects the total processor running time and processor idle time of the available data center in real time, and calculates the processor utilization rate of the available data center using the total processor running time and the processor idle time; the module also counts the idle memory of the available data center in real time, and calculates the memory utilization rate of the available data center using the total memory of the available data center and the idle memory; the module periodically sends test packets to the available data centers and records the round-trip time of the test packets, and calculates the network latency of the available data centers using the round-trip time; the module queries the number of pending tasks of the available data centers, and calculates the task queue length of the available data centers based on the number of pending tasks; and uses the processor utilization rate, the memory utilization rate, the network latency, and the task queue length as the operating status information of the available data centers.

[0124] In one embodiment, the filtering module 703 is used to query among the plurality of scheduling strategies whether a scheduling strategy matching the task attribute information exists. When the query determines that a scheduling strategy matching the task attribute information exists among the plurality of scheduling strategies, the scheduling strategy matching the task attribute information is taken as the target scheduling strategy. When the query determines that no scheduling strategy matching the task attribute information exists among the plurality of scheduling strategies, the processor utilization rate and memory utilization rate corresponding to each available data center are extracted from the operating status information of each available data center. Based on the processor utilization rate and memory utilization rate corresponding to each available data center... The system queries whether any of the available data centers are overloaded, wherein the processor utilization rate of the overloaded data center is greater than a processor utilization rate threshold or the memory utilization rate of the overloaded data center is greater than a memory utilization rate threshold. If the query determines that there is no overloaded data center among the available data centers, then a scheduling strategy for weighted round-robin scheduling of the available data centers is selected from the multiple scheduling strategies as the target scheduling strategy. If the query determines that there is an overloaded data center among the available data centers, then a scheduling strategy for selecting the available data center with the lowest load among the available data centers for scheduling is selected from the multiple scheduling strategies as the target scheduling strategy.

[0125] In one embodiment, the filtering module 703 is configured to read task priority and task execution time limit from the task attribute information; when the task priority is less than a priority threshold and the task execution time limit is greater than or equal to a time limit threshold, read task dependencies from the task attribute information, wherein when the task priority is greater than or equal to the priority threshold or the task execution time limit is less than the time limit threshold, the scheduling strategy among the plurality of scheduling strategies used to indicate data center scheduling according to task priority is selected as the scheduling strategy matching the task attribute information; when the task dependency indicates that the task to be processed does not have any dependent tasks, read the task type from the task attribute information, wherein when the task dependency indicates that the task to be processed has dependent tasks, the scheduling strategy among the plurality of scheduling strategies used to indicate data center scheduling according to task dependencies is selected as the scheduling strategy matching the task attribute information. The scheduling strategy matches the attribute information. If the task type indicates that the task data volume of the task to be processed is less than or equal to the task data volume threshold, then the task type is parsed to determine whether the task to be processed is allowed to be sharded. Specifically, when the task type indicates that the task data volume of the task to be processed is greater than the task data volume, the scheduling strategy among the multiple scheduling strategies that indicates data center scheduling based on the geographical location of the task's origin is selected as the scheduling strategy matching the task attribute information. When it is determined that the task to be processed is not allowed to be sharded, it is determined that there is no scheduling strategy among the multiple scheduling strategies that matches the task attribute information. Specifically, when it is determined that the task to be processed is allowed to be sharded, the scheduling strategy among the multiple scheduling strategies that indicates data center scheduling based on the sharding result of the task to be processed is selected as the scheduling strategy matching the task attribute information.

[0126] In one embodiment, the scheduling module 704 is further configured to, when detecting that the task to be processed has failed or the status of the target available data center has changed to unavailable, re-collect the operating status information of each available data center, and, based on the task attribute information and the operating status information of each available data center, filter a new target scheduling strategy from the plurality of scheduling strategies, and, according to the new target scheduling strategy, select a new target available data center from the plurality of available data centers, and schedule the new target available data center to reprocess the task to be processed.

[0127] In one embodiment, the scheduling module 704 is further configured to periodically read whether an abnormal data center identifier is stored in a preset key-value database. The abnormal data center identifier stored in the preset key-value database is either reported to the database by the data center itself when an abnormality is detected, or reported to the database by the business party when an abnormality is detected in the data center through a detection task. The module also queries the available data centers indicated by the abnormal data center identifier among the multiple available data centers as abnormal data centers, takes the abnormal data center offline, and schedules other available data centers (excluding the abnormal data center) to process the tasks assigned to the abnormal data center.

[0128] This invention provides a multi-datacenter parallel scheduling device that combines the task attribute information of the task to be processed with the operating status information of each available datacenter. It selects a suitable target scheduling strategy from multiple scheduling strategies for datacenter scheduling, ensuring that the scheduling process considers not only the task's own attributes but also the real-time status of the datacenter. This adapts to complex business scenarios, enabling balanced utilization of datacenter resources even in complex environments, avoiding resource waste, ensuring the normal operation of financial and medical services, and high efficiency in task execution. It meets the efficient, stable, and flexible datacenter scheduling requirements of financial insurance and medical services.

[0129] Specific limitations regarding the multi-datacenter parallel scheduling device can be found in the limitations of the multi-datacenter parallel scheduling method described above, and will not be repeated here. Each module in the aforementioned multi-datacenter parallel scheduling device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0130] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. When the computer program is executed by the processor, it implements the functions or steps of a multi-datacenter parallel scheduling method on the server side.

[0131] In one embodiment, a computer device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the client-side functions or steps of a multi-datacenter parallel scheduling method.

[0132] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0133] When a task to be processed is received, the task attribute information of the task to be processed is parsed.

[0134] Identify multiple available data centers and collect real-time operational status information for each of the available data centers;

[0135] Obtain multiple preset scheduling strategies, and based on the task attribute information and the operating status information of each available data center, filter the target scheduling strategy from the multiple scheduling strategies;

[0136] According to the target scheduling strategy, a target available data center is selected from the multiple available data centers, and the target available data center is scheduled to process the task to be processed.

[0137] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0138] When a task to be processed is received, the task attribute information of the task to be processed is parsed.

[0139] Identify multiple available data centers and collect real-time operational status information for each of the available data centers;

[0140] Obtain multiple preset scheduling strategies, and based on the task attribute information and the operating status information of each available data center, filter the target scheduling strategy from the multiple scheduling strategies;

[0141] According to the target scheduling strategy, a target available data center is selected from the multiple available data centers, and the target available data center is scheduled to process the task to be processed.

[0142] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions on the server side and client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0144] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0145] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A multi-datacenter parallel scheduling method, characterized in that, include: When a task to be processed is received, the task attribute information of the task to be processed is parsed. Identify multiple available data centers and collect real-time operational status information for each of the available data centers; Obtain multiple preset scheduling strategies, and based on the task attribute information and the operating status information of each available data center, filter the target scheduling strategy from the multiple scheduling strategies; According to the target scheduling strategy, a target available data center is selected from the multiple available data centers, and the target available data center is scheduled to process the task to be processed.

2. The method according to claim 1, characterized in that, The process of parsing the task attribute information of the task to be processed includes: The task metadata of the task to be processed is parsed to obtain multiple attribute parameters, including task type, task priority, task dependency, and task execution time limit; The multiple attribute parameters are organized to obtain the task attribute information of the task to be processed.

3. The method according to claim 1, characterized in that, The process of identifying multiple available data centers and collecting real-time operational status information for each available data center includes: Queries the multiple data centers that have been successfully connected and are operating normally as the multiple available data centers; For each available data center, the total running time and idle time of the processors in the available data center are collected in real time. The processor utilization rate of the available data center is calculated using the total running time and idle time of the processors. The available data center's idle memory is monitored in real time. Using the total available data center memory and the idle memory, the memory utilization rate of the available data center is calculated. Test packets are periodically sent to the available data centers and the round-trip time of the test packets is recorded. The round-trip time is used to calculate the network latency of the available data centers. Query the number of pending tasks in the available data center, and calculate the task queue length of the available data center based on the number of pending tasks; The processor utilization rate, the memory utilization rate, the network latency, and the task queue length are used as the operating status information of the available data center.

4. The method according to claim 1, characterized in that, The step of filtering a target scheduling strategy from the multiple scheduling strategies based on the task attribute information and the operational status information of each available data center includes: The system queries among the plurality of scheduling strategies to determine if there is a scheduling strategy that matches the task attribute information. When the query determines that there is a scheduling strategy that matches the task attribute information among the plurality of scheduling strategies, the scheduling strategy that matches the task attribute information is taken as the target scheduling strategy. When the query determines that there is no scheduling strategy that matches the task attribute information among the multiple scheduling strategies, the processor utilization rate and memory utilization rate corresponding to each available data center are extracted from the running status information of each available data center; Based on the processor utilization and memory utilization of each available data center, query whether there is an overloaded data center among the multiple available data centers, wherein the processor utilization of the overloaded data center is greater than the processor utilization threshold or the memory utilization is greater than the memory utilization threshold. If the query determines that there is no overloaded data center among the multiple available data centers, then the scheduling strategy used for weighted round-robin scheduling of the multiple available data centers is selected from the multiple scheduling strategies as the target scheduling strategy. If the query determines that there is an overloaded data center among the multiple available data centers, then the scheduling strategy selected from the multiple scheduling strategies for scheduling the available data center with the lowest load among the multiple available data centers is selected as the target scheduling strategy.

5. The method according to claim 4, characterized in that, The step of querying among the multiple scheduling policies to see if there is a scheduling policy that matches the task attribute information includes: Read the task priority and task execution time limit from the task attribute information; When the task priority is less than the priority threshold and the task execution time is greater than or equal to the time limit threshold, the task dependency relationship is read from the task attribute information. When the task priority is greater than or equal to the priority threshold or the task execution time is less than the time limit threshold, the scheduling strategy among the multiple scheduling strategies used to indicate data center scheduling according to task priority is used as the scheduling strategy that matches the task attribute information. When the task dependency relationship indicates that the task to be processed does not have any dependent tasks, the task type is read from the task attribute information. When the task dependency relationship indicates that the task to be processed has dependent tasks, the scheduling strategy among the multiple scheduling strategies used to indicate data center scheduling according to task dependency relationship is used as the scheduling strategy that matches the task attribute information. If the task type indicates that the task data volume of the task to be processed is less than or equal to the task data volume threshold, then the task type is parsed to determine whether the task to be processed is allowed to be processed in segments. When the task type indicates that the task data volume of the task to be processed is greater than the task data volume, the scheduling strategy among the multiple scheduling strategies that indicates data center scheduling according to the geographical location of the source of the task to be processed is used as the scheduling strategy that matches the task attribute information. When it is determined that the task to be processed is not allowed to be processed in segments, it is determined that there is no scheduling strategy among the multiple scheduling strategies that matches the task attribute information. However, when it is determined that the task to be processed is allowed to be processed in segments, the scheduling strategy among the multiple scheduling strategies that indicates data center scheduling according to the segmentation result of the task to be processed is taken as the scheduling strategy that matches the task attribute information.

6. The method according to claim 1, characterized in that, The method further includes: When the processing failure of the pending task is detected or the status of the target available data center changes to unavailable, the operating status information of each available data center is collected again, and a new target scheduling strategy is selected from the multiple scheduling strategies based on the task attribute information and the operating status information of each available data center. According to the new target scheduling strategy, a new target available data center is selected from the multiple available data centers, and the new target available data center is scheduled to reprocess the pending task.

7. The method according to claim 1, characterized in that, The method further includes: The system periodically reads whether there is an abnormal data center identifier stored in the preset key-value database. The abnormal data center identifier stored in the preset key-value database is either reported to the preset key-value database by the data center itself when it recognizes an abnormality, or reported to the preset key-value database by the business party when it detects an abnormality in the data center through a detection task. The system queries the available data centers indicated by the abnormal data center identifier among the multiple available data centers, identifies the abnormal data center as the abnormal data center, takes the abnormal data center offline, and schedules other available data centers (excluding the abnormal data center) to process the tasks assigned to the abnormal data center.

8. A multi-room parallel scheduling device, characterized in that, include: The parsing module is used to parse the task attribute information of the task to be processed when a task to be processed is received. The data acquisition module is used to identify multiple available data centers and collect the operational status information of each available data center in real time. The filtering module is used to obtain multiple preset scheduling strategies, and filter the target scheduling strategy from the multiple scheduling strategies based on the task attribute information and the operating status information of each available data center. The scheduling module is used to select a target available data center from the plurality of available data centers according to the target scheduling strategy, and to schedule the target available data center to process the task to be processed.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the multi-data center parallel scheduling method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the multi-data center parallel scheduling method as described in any one of claims 1 to 7.

Images