Task scheduling method, device and equipment for server cluster

Abstract

The invention provides a task scheduling method, device and equipment for a server cluster. The method comprises the following steps: acquiring a current task of any target server in the server cluster; loading a target function corresponding to the current task through a decorator function to obtain a task identifier of the current task; querying the task identifier in a target database to obtain a query result; acquiring attributes of the target function; generating a task instruction according to the query result and the attribute of the target function; and sending the task instruction to the target server, so that the target server executes the current task or acquires the next target task according to the task instruction. According to the scheme, the task scheduling problem of the server cluster can be solved.

Description

Technical Field

[0001] The present invention relates to the field of computer information technology, and in particular to a task scheduling method, device and equipment for a server cluster. Background Art

[0002] There are a large number of scheduled tasks in the background server to pull data. Apschedular (a lightweight task scheduling framework) is currently widely used to schedule these scheduled tasks. As a mainstream scheduled task scheduler, Apschedular is widely used in a single-machine environment to schedule various scheduled tasks.

[0003] However, due to the increasing amount of task data, a single server can no longer meet our needs, and we must use multiple servers, that is, to solve the performance problem through server clusters. However, the Apschedular framework itself does not provide the task scheduling function of server clusters, and currently only supports single server scheduling tasks; therefore, it is impossible to use Apschedular to perform reasonable task scheduling in a cluster environment. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a task scheduling method, device and equipment for a server cluster, which can solve the task scheduling problem of the server cluster.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows:

[0006] A task scheduling method for a server cluster, comprising:

[0007] Get the current task of any target server in the server cluster;

[0008] The target function corresponding to the current task is loaded and processed by the decorator function to obtain the task identifier of the current task;

[0009] Query the task identifier in the target database to obtain a query result;

[0010] Obtaining properties of the objective function;

[0011] Generate a task instruction according to the query result and the attribute of the objective function;

[0012] The task instruction is sent to the target server, so that the target server executes the current task or obtains the next target task according to the task instruction.

[0013] Optionally, the target function corresponding to the current task is loaded and processed by a decorator function to obtain a task identifier of the current task, including:

[0014] The target function is input into the decorator function as a parameter of the decorator function to obtain a function signature returned by the decorator function; and the function signature is used as a task identifier of the current task.

[0015] Optionally, querying the task identifier in the target database to obtain query results includes:

[0016] Compressing the task identifier to obtain a target character string;

[0017] The target character string is used as a target key, and the target key is searched in a target database. When the target key is found, a first search result is output; otherwise, a second search result is output.

[0018] Optionally, obtaining properties of the objective function includes:

[0019] Obtaining configuration information of the objective function;

[0020] A first attribute or a second attribute of the objective function is obtained from the configuration information, wherein the first attribute is an interval execution function, and the second attribute is that the objective function is a repeated execution function.

[0021] Optionally, generating a task instruction according to the query result and the attribute of the objective function includes:

[0022] When the target function is an interval execution function and the query result is a first query result, obtaining a value corresponding to the target key stored in the target database;

[0023] When the difference between the value and the current timestamp is greater than or equal to the interval time of the target function, a first task instruction is generated; the first task instruction is used to instruct the target server to execute the current task;

[0024] When the difference between the value and the current timestamp is less than the interval time of the target function, a second task instruction is generated, and the second task instruction is used to instruct the target server to obtain the next target task.

[0025] Optionally, generating a task instruction according to the query result and the attribute of the objective function includes:

[0026] When the target function is an interval execution function and the query result is the second query result, a third task instruction is generated, and the current timestamp is stored in the target database as the value of the target key. The third task instruction is used to instruct the target server to execute the current task.

[0027] Optionally, generating a task instruction according to the query result and the attribute of the objective function includes:

[0028] When the target function is a repeated execution function and the query result is the first query result, a fourth task instruction is generated, and the fourth task instruction is used to instruct the target server to obtain a next target task;

[0029] When the target function is a repeated execution function and the query result is the second query result, a fifth task instruction is generated, and the fifth task instruction is used to instruct the target server to execute the current task and store the target key in the target database.

[0030] The present invention also provides a task scheduling device for a server cluster, comprising:

[0031] The acquisition module is used to obtain the current task of any target server in the server cluster;

[0032] A processing module is used to load and process the target function corresponding to the current task through a decorator function to obtain the task identifier of the current task; query the task identifier in a target database to obtain a query result; obtain the properties of the target function; generate a task instruction based on the query result and the properties of the target function; and send the task instruction to the target server so that the target server executes the current task or obtains the next target task according to the task instruction.

[0033] The present invention also provides a computing device, comprising: a processor and a memory storing a computer program, wherein when the computer program is run by the processor, the method as described above is executed.

[0034] The present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above method.

[0035] The above solution of the present invention includes at least the following beneficial effects:

[0036] The above scheme of the present invention can solve the task scheduling problem of the server cluster by obtaining the current task of any target server in the server cluster; loading the target function corresponding to the current task through the decorator function to obtain the task identifier of the current task; querying the task identifier in the target database to obtain the query result; obtaining the attribute of the target function; generating a task instruction according to the query result and the attribute of the target function; and sending the task instruction to the target server so that the target server executes the current task or obtains the next target task according to the task instruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1is a first flow chart of a task scheduling method for a server cluster according to an embodiment of the present invention;

[0038] Figure 2 is a second flow chart of the task scheduling method of a server cluster according to an embodiment of the present invention;

[0039] Figure 3 It is a structural diagram of a task scheduling device for a server cluster according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0041] like Figure 1 As shown, an embodiment of the present invention provides a task scheduling method for a server cluster, comprising:

[0042] Step 11, obtaining the current task of any target server in the server cluster;

[0043] Step 12, loading the target function corresponding to the current task through the decorator function to obtain the task identifier of the current task;

[0044] Step 13, querying the task identifier in the target database to obtain a query result;

[0045] Step 14, obtaining the properties of the objective function;

[0046] Step 15, generating a task instruction according to the query result and the properties of the objective function;

[0047] Step 16: Send the task instruction to the target server, so that the target server executes the current task or obtains the next target task according to the task instruction.

[0048] In this embodiment, the target task is a data pulling task of the target server on the target data platform. The target task usually corresponds to a target function, through which data is pulled on the target data platform. The target task can be an operation to synchronize the target data of the target data platform, where the target data platform can be a different advertising data platform or a data platform of the target application; specifically, the target task can be as follows:

[0049] Task 1: Synchronize the target data platform account amount data

[0050] scheduler.add_job(

[0051] id='gdt_account_fund_main',func=

[0052] gdt_account_fund_main,trigger='interval',hours=1,replace_existing=True,jobstore='default' )

[0054] Task 2: Synchronize target data platform account data

[0055] scheduler.add_job(

[0056] id='gdt_advertiser_info_main',func=

[0057] gdt_advertiser_info_main,trigger='interval',hours=1,replace_existing=True,jobstor e='default' )

[0059] Task 3: Get the daily balance of funds account

[0060] scheduler.add_job(

[0061] id='gdt_account_daily_balance_main',func=

[0062] gdt_account_daily_balance_main,trigger='cron',hours=9,replace_existing=True,jo bstore='cron' )

[0064] Task 4: Update target data platform materials

[0065] scheduler.add_job(

[0066] id='gdt_adgroup_daily_report_main', func=

[0067] gdt_adgroup_daily_report_main,trigger='

[0068] interval',minutes=3,replace_existing=True,jobstore='default'

[0069] Among the four target tasks listed above, the task that the target server is currently about to execute is the current task. In this embodiment, hundreds of target tasks are deployed on the target server, and these target tasks are scheduled by Apschedular (a lightweight task scheduling framework) deployed on each target server. Apschedular on each target server refers to the same task scheduling configuration to learn which functions need to be executed at this moment, that is, to determine the current task of the target server.

[0070] After determining the current task of the target server, such as Figure 2 As shown, the target function corresponding to the current task is decorated by a decorator function to obtain the signature of the target function, and the function signature is used as the task identifier of the current task. The task identifier is queried in the target database to determine whether the task identifier has been stored in the target database to obtain the query result. According to the query result, combined with the properties of the target function corresponding to the current task, it is determined whether the target server executes the current task. In this embodiment, preferably, the target database is a Redis database, which is a high-performance (key-value) distributed memory database, and its data model is a key-value pair (key-value). In this embodiment, when all servers in the server cluster execute the target task, the task identifier of the target task is used as the key key, and the current timestamp is used as the value value, and the key-value pair is stored in the target database. When the target task is executed, according to the properties of the target function corresponding to the target task, the corresponding key-value pair stored in the database is deleted or the key-value pair stored in the database is retained. When a server is to execute a current task, it can be determined by querying the task identifier in the database whether other servers in the cluster are executing the current task, and the execution of the target function corresponding to the current task by other servers in the cluster in the past. The target database can determine whether the target server should execute the current task according to the execution status of other servers, thereby realizing task scheduling for the server cluster.

[0071] In this embodiment, the target database acts as a unified task scheduling center, uses the function signature as the unique identifier of task execution, and determines the previous and current execution status of the task by querying the function signature, thereby realizing task scheduling for the target server in the server cluster. This solves the problem of using the Apschedular (scheduled task) framework to perform task scheduling in a server cluster. Since the unified center Redis manages all task states, and since Redis itself has atomic access, there will be no concurrency problems, that is, the number of cluster servers can be dynamically added or reduced at any time without affecting the overall execution effect.

[0072] In an optional embodiment of the present invention, step 12 may include:

[0073] Step 121, input the target function as a parameter of the decorator function into the decorator function, obtain the function signature returned by the decorator function; and use the function signature as the task identifier of the current task.

[0074] In this embodiment, each target function in the Apschedular configuration has already used decorator logic in advance to dynamically increase function content and load function signatures. The decorator is a high-order function that accepts the target function as a parameter. In this embodiment, an internal function is defined inside the decorator function, and the internal function can access the parameters of the decorator function and the external environment (including the decorated function). The internal function implements the functional extension of the decorated function. Finally, the decorator function returns the internal function, that is, the expanded function replaces the original function.

[0075] In this embodiment, all target functions configured in the Apschedular framework are passed to the decorator count_func_execute_seconds, and the step of obtaining the function signature is task_name=func.__name__.

[0076] Since functions are first-class objects and have attributes, __name__ is the name of the function, and this function signature is used as the unique identifier for task execution. Since the Python interpreter will naturally check whether there are duplicate functions, there is no need to add additional business logic code to ensure the uniqueness of function execution.

[0077] The wrapped function, that is, all scheduled task functions, are replaced with inner functions and returned. Finally, the task functions called by the Apschedular framework have all the functions written in the decorator code.

[0078] In an optional embodiment of the present invention, step 13 may include:

[0079] Step 131, compressing the task identifier to obtain a target character string;

[0080] Step 132, using the target character string as a target key, searching for the target key in a target database, and when the target key is found, outputting a first query result, otherwise, outputting a second query result.

[0081] In this embodiment, the target database server is connected in the decorator logic. Since a large number of keys are resident in the target database, the length of the key needs to be compressed to save storage space. The task identifier is compressed, and the compressed result is used as the key to query the target database. The compression method includes:

[0082] 1. Data block segmentation: divide the function signature into multiple data blocks (usually 256 bytes or smaller), each of which can be compressed independently;

[0083] 2. Compress each data block. Compress the data by looking for repeated byte sequences and creating a sliding window (usually 32KB) to find the match of the current byte. For each byte, output the matching offset and length, or directly output the literal value;

[0084] 3. Count the compression results of each data block, generate a Huffman tree and generate a Huffman code, and replace each symbol in the compression result of each data block with its corresponding Huffman code;

[0085] 4. The compressed data finally output is the compressed data after Huffman coding.

[0086] In an optional embodiment of the present invention, step 14 may include:

[0087] Step 141, obtaining configuration information of the objective function;

[0088] Step 142: Acquire a first attribute or a second attribute of the objective function from the configuration information, wherein the first attribute is an interval execution function, and the second attribute is that the objective function is a repeated execution function.

[0089] In this embodiment, the target function is configured in the configuration information as cron mode (expression task scheduling) or interval mode (periodic task scheduling). When the target function is in interval mode, the target function is determined to be an interval execution function, that is, an event is triggered after a fixed time interval. When the target function is in cron mode, since the cron expression set in this embodiment is that the target function is always executed, the target function is determined to be a repeated execution function.

[0090] In an optional embodiment of the present invention, step 15 may include:

[0091] Step 151, when the target function is an interval execution function and the query result is a first query result, obtaining a value corresponding to the target key stored in the target database;

[0092] Step 152, when the difference between the value and the current timestamp is greater than or equal to the interval time of the target function, a first task instruction is generated; the first task instruction is used to instruct the target server to execute the current task;

[0093] Step 153: When the difference between the value and the current timestamp is less than the interval time of the target function, a second task instruction is generated, and the second task instruction is used to instruct the target server to obtain the next target task.

[0094] In this embodiment, when it is determined that the target function is an interval execution function and the key key corresponding to the target function is stored in the target database, the value value corresponding to the key key is queried. The value value is the corresponding start time when the server in the cluster last executed the target function. According to the difference between the value value and the current timestamp, the interval time from the last execution of the target function is judged. When the interval time is greater than or equal to the interval time set by the target function, it means that the target server is required to execute the target function. At this time, the database server sends a first task instruction to the target server, and the target server executes the current task according to the first task instruction. When the value value shows that the interval time from the last execution is less than the interval time set by the target function, it means that the target server does not need to execute this current task. At this time, the database server sends a second task instruction to the target server, and the target server does not execute this current task according to the second task instruction, and obtains the next target task, and repeats the judgment whether to execute the next target task.

[0095] In an optional embodiment of the present invention, step 15 may further include:

[0096] Step 154, when the target function is an interval execution function and the query result is the second query result, a third task instruction is generated, and the current timestamp is stored in the target database as the value of the target key, and the third task instruction is used to instruct the target server to execute the current task.

[0097] In this embodiment, when the target function is an interval execution function and the task identifier of the current task is not found in the target database, it means that no server in the cluster has executed the target function. At this time, a third task instruction is generated, and the target server executes the current task. The task identifier of the current task is compressed as the key and the current timestamp is used as the value and stored in the target database.

[0098] When compressing the task identifier here, you can use the following method to compress it:

[0099] The reference of duplicate data is achieved by searching for the longest prefix that matches the current sequence of characters (task identifiers) to be compressed in the current window (usually a fixed-size buffer). Specifically, assuming that the current sequence of characters to be processed is S and the window size is W, the position and length of the longest prefix that matches the beginning of S in the search window are as follows: Let the current matching position be match_pos and the matching length be match_len. Initially, match_pos is negative infinity and match_len is 0. Starting from the current position of the characters to be processed, search from the back to the front in the window for the part that matches the beginning of the current sequence of characters to be processed, and find the longest matching position and length.

[0100] Furthermore, the difference between the original data and the matching part and some control information (such as a flag bit, etc.) are encoded.

[0101] Suppose there is a set of symbols {s1, s2, s3, ..., sn}, and their respective frequencies of occurrence are {f1, f2, f3, ..., fn}. Step 1, first form each symbol and its frequency into a series of nodes, and each node can be regarded as a binary tree with only one node.

[0102] Step 2: Select two nodes with the smallest frequencies from these nodes, create a new parent node whose frequency is the sum of the frequencies of the two child nodes, remove the two child nodes from the node set, and add the newly created parent node to the node set.

[0103] Repeat step 2 until there is only one node left in the node set, which is the root node of the tree.

[0104] For each symbol, we start from the root node and encode it according to its path in the tree. Going left is encoded as 0, and going right is encoded as 1. Finally, we get the encoding of each symbol. The above search, matching and encoding work together to achieve efficient data compression.

[0105] In an optional embodiment of the present invention, step 15 may further include:

[0106] Step 155, when the target function is a repeated execution function and the query result is the first query result, a fourth task instruction is generated, and the fourth task instruction is used to instruct the target server to obtain the next target task;

[0107] Step 156, when the target function is a repeated execution function and the query result is the second query result, a fifth task instruction is generated, and the fifth task instruction is used to instruct the target server to execute the current task and store the target key in the target database.

[0108] In this embodiment, when the target function is a repeatedly executed function and the function signature of the target function is queried in the target database, it means that another server in the cluster is currently executing the target function. In order to improve the efficiency of pulling data from the server cluster, the target server does not need to repeatedly execute the target function. Instead, it can obtain the next target task and determine whether the next target task should be executed based on the same judgment logic.

[0109] When the target function is a repeatedly executed function and the function signature of the target function is not found in the target database, it means that no server in the server cluster is executing the target function. Since the set target function is repeatedly executed, it needs to be executed all the time. Therefore, the target server executes the current task, calculates the key according to the target function corresponding to the current task, and stores the current timestamp as the value in the target database. When the target server finishes executing the current task, it deletes the corresponding key-value pair stored in the target database and continues to obtain the next target task.

[0110] The above method of the present invention uses a decorator to execute business logic, including operations such as extracting function signatures, and then uses Redis's single thread and atomic access to store task function signatures, which can prevent other machines in the cluster from erroneous access and repeated execution. Without modifying the code function of Apschedular itself, Apschedular can be used to schedule tasks in a cluster environment, saving a lot of computing resources. The task execution performance of pulling data is improved, and when pulling data from various platforms, the timeliness and accuracy of pulling data are improved.

[0111] The implementation process of the above method is further described below with a specific embodiment:

[0112] Example 1: The target data platform is an advertising data platform

[0113] 1. Start the Apschedular scheduling service in each cluster server;

[0114] 2. Each Apschedular will refer to the same task scheduling configuration to learn which functions need to be executed at the moment. The following provides some function code configuration examples;

[0115] Task 1: Synchronize account balance data on the advertising data platform

[0116] scheduler.add_job(

[0117] id='gdt_account_fund_main',func=

[0118] gdt_account_fund_main,trigger='interval',hours=1,replace_existing=True,jobstore='default' )

[0120] Task 2: Synchronize advertising data platform account data

[0121] scheduler.add_job(

[0122] id='gdt_advertiser_info_main',func=

[0123] gdt_advertiser_info_main,trigger='interval',hours=1,replace_existing=True,jobstor e='default' )

[0125] Task 3: Get the daily balance of funds account

[0126] scheduler.add_job(

[0127] id='gdt_account_daily_balance_main',func=

[0128] gdt_account_daily_balance_main,trigger='cron',hours=9,replace_existing=True,jo bstore='cron' )

[0130] Task 4: Update advertising data platform materials

[0131] scheduler.add_job(

[0132] id='gdt_adgroup_daily_report_main', func=

[0133] gdt_adgroup_daily_report_main,trigger='

[0134] interval',minutes=3,replace_existing=True,jobstore='default'

[0135] The above code is part of the code for pulling data from an advertising data platform, where:

[0136] The gdt_account_fund_main function is used to pull account balance data from the advertising data platform;

[0137] The gdt_advertiser_info_main function is used to pull account report data from the advertising data platform;

[0138] The gdt_account_daily_balance_main function is used to retrieve the financial flow data of the advertising data platform account;

[0139] The gdt_adgroup_daily_report_main function is used to pull advertising creative material data from the advertising data platform;

[0140] These four functions are called using cron and interval modes respectively. There are about 400 similar function configurations. The average execution time of a single function is 10-20 minutes. In other words, in a single machine, Apschedular cannot meet our requirements for function scheduling and execution efficiency.

[0141] 3. Each function to be executed in the configuration has already used decorator logic to dynamically increase the function content and load the function signature in advance. The following are some code examples:

[0142] Decorator configuration:

[0143] @count_func_execute_seconds

[0144] def gdt_account_daily_balance_main(record_date:str=None):

[0145] ””

[0146] Get the daily balance of the fund account (new, including the end-of-day balance data)

[0147] :param record_date:

[0148] :return:

[0149] ””

[0150] Decorator specific logic:

[0151]

[0152] In the code example, all the task functions configured in the Apschedular framework are passed to the decorator count_func_execute_seconds, and the step of obtaining the function signature is task_name = func.__name__;

[0153] Since functions are first-class objects, they also have attributes, where __name__ is the name of the function. This function signature is used as the unique identifier for task execution. The advantage is that the Python interpreter will naturally check whether there are duplicate functions when executing, so there is no need to add additional business logic code to ensure the uniqueness of function execution;

[0154] The wrapped function, that is, all our scheduled task functions, are replaced with the inner function and returned. Finally, the task functions called by the Apschedular framework have all the functions written in the decorator code.

[0155] 4. Connect the Redis server in the decorator logic, and calculate the key from the function signature for query. Since there are a large number of keys in Redis, it is very necessary to compress the length of the key to save storage space;

[0156] 5. Check whether the key in step 4 exists in the Redis server, and then determine whether to continue execution based on the last execution time saved in the value; if the key is not obtained at this moment, write the key to Redis and execute it, and write the value to the current time, and the process ends.

[0157] like Figure 3 As shown, an embodiment of the present invention further provides a task scheduling device 30 for a server cluster, comprising:

[0158] An acquisition module 31 is used to acquire the current task of any target server in the server cluster;

[0159] The processing module 32 is used to load the target function corresponding to the current task through the decorator function to obtain the task identifier of the current task; query the task identifier in the target database to obtain the query result; obtain the properties of the target function; generate a task instruction based on the query result and the properties of the target function; send the task instruction to the target server, so that the target server executes the current task or obtains the next target task according to the task instruction.

[0160] Optionally, the target function corresponding to the current task is loaded and processed by a decorator function to obtain a task identifier of the current task, including:

[0161] The target function is input into the decorator function as a parameter of the decorator function to obtain a function signature returned by the decorator function; and the function signature is used as a task identifier of the current task.

[0162] Optionally, querying the task identifier in the target database to obtain query results includes:

[0163] Compressing the task identifier to obtain a target character string;

[0164] The target character string is used as a target key, and the target key is searched in a target database. When the target key is found, a first search result is output; otherwise, a second search result is output.

[0165] Optionally, obtaining properties of the objective function includes:

[0166] Obtaining configuration information of the objective function;

[0167] A first attribute or a second attribute of the objective function is obtained from the configuration information, wherein the first attribute is an interval execution function, and the second attribute is that the objective function is a repeated execution function.

[0168] Optionally, generating a task instruction according to the query result and the attribute of the objective function includes:

[0169] When the target function is an interval execution function and the query result is a first query result, obtaining a value corresponding to the target key stored in the target database;

[0170] When the difference between the value and the current timestamp is greater than or equal to the interval time of the target function, a first task instruction is generated; the first task instruction is used to instruct the target server to execute the current task;

[0171] When the difference between the value and the current timestamp is less than the interval time of the target function, a second task instruction is generated, and the second task instruction is used to instruct the target server to obtain the next target task.

[0172] Optionally, generating a task instruction according to the query result and the attribute of the objective function includes:

[0173] When the target function is an interval execution function and the query result is the second query result, a third task instruction is generated, and the current timestamp is stored in the target database as the value of the target key. The third task instruction is used to instruct the target server to execute the current task.

[0174] Optionally, generating a task instruction according to the query result and the attribute of the objective function includes:

[0175] When the target function is a repeated execution function and the query result is the first query result, a fourth task instruction is generated, and the fourth task instruction is used to instruct the target server to obtain a next target task;

[0176] When the target function is a repeated execution function and the query result is the second query result, a fifth task instruction is generated, and the fifth task instruction is used to instruct the target server to execute the current task and store the target key in the target database.

[0177] It should be noted that the device is a device corresponding to the above method, and all implementation methods in the above method embodiments are applicable to the embodiments of the device and can achieve the same technical effects.

[0178] The embodiment of the present invention further provides a computing device, comprising: a processor, a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0179] The embodiment of the present invention further provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the above method. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0180] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0181] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0182] In the embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0185] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, ROM, RAM, magnetic disks, or optical disks.

[0186] In addition, it should be noted that in the apparatus and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. Moreover, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order, and some steps can be performed in parallel or independently of each other. For those of ordinary skill in the art, it is understood that all or any steps or components of the method and apparatus of the present invention can be implemented in any computing device (including processors, storage media, etc.) or a network of computing devices in hardware, firmware, software or a combination thereof, which can be achieved by those of ordinary skill in the art using their basic programming skills after reading the description of the present invention.

[0187] Therefore, the purpose of the present invention can also be achieved by running a program or a group of programs on any computing device. The computing device can be a well-known general device. Therefore, the purpose of the present invention can also be achieved by simply providing a program product containing a program code that implements the method or device. That is to say, such a program product also constitutes the present invention, and the storage medium storing such a program product also constitutes the present invention. Obviously, the storage medium can be any well-known storage medium or any storage medium developed in the future. It should also be pointed out that in the device and method of the present invention, it is obvious that each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent schemes of the present invention. In addition, the steps of performing the above-mentioned series of processing can naturally be performed in chronological order according to the order of description, but it is not necessary to perform them in chronological order. Some steps can be performed in parallel or independently of each other.

[0188] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A task scheduling method for a server cluster, characterized in that: include: Get the current task of any target server in the server cluster; The target function corresponding to the current task is loaded and processed by the decorator function to obtain the task identifier of the current task; Query the task identifier in the target database to obtain a query result; Obtaining properties of the objective function; Generate a task instruction according to the query result and the attribute of the objective function; The task instruction is sent to the target server, so that the target server executes the current task or obtains the next target task according to the task instruction.

2. The task scheduling method for a server cluster according to claim 1, characterized in that: The target function corresponding to the current task is loaded and processed by the decorator function to obtain the task identifier of the current task, including: The target function is input into the decorator function as a parameter of the decorator function to obtain a function signature returned by the decorator function; and the function signature is used as a task identifier of the current task.

3. The task scheduling method for a server cluster according to claim 1, characterized in that: Query the task identifier in the target database to obtain query results, including: Compressing the task identifier to obtain a target character string; The target character string is used as a target key, and the target key is searched in a target database. When the target key is found, a first search result is output; otherwise, a second search result is output.

4. The task scheduling method for a server cluster according to claim 3, characterized in that: Obtaining properties of the objective function, including: Obtaining configuration information of the objective function; A first attribute or a second attribute of the objective function is obtained from the configuration information, wherein the first attribute is an interval execution function, and the second attribute is that the objective function is a repeated execution function.

5. The task scheduling method for a server cluster according to claim 4, characterized in that: Generate a task instruction according to the query result and the attribute of the objective function, including: When the target function is an interval execution function and the query result is a first query result, obtaining a value corresponding to the target key stored in the target database; When the difference between the value and the current timestamp is greater than or equal to the interval time of the target function, a first task instruction is generated; the first task instruction is used to instruct the target server to execute the current task; When the difference between the value and the current timestamp is less than the interval time of the target function, a second task instruction is generated, and the second task instruction is used to instruct the target server to obtain the next target task.

6. The task scheduling method for a server cluster according to claim 4, characterized in that: Generate a task instruction according to the query result and the attribute of the objective function, including: When the target function is an interval execution function and the query result is the second query result, a third task instruction is generated, and the current timestamp is stored in the target database as the value of the target key. The third task instruction is used to instruct the target server to execute the current task.

7. The task scheduling method for a server cluster according to claim 4, characterized in that: Generate a task instruction according to the query result and the attribute of the objective function, including: When the target function is a repeated execution function and the query result is the first query result, a fourth task instruction is generated, and the fourth task instruction is used to instruct the target server to obtain the next target task; When the target function is a repeated execution function and the query result is the second query result, a fifth task instruction is generated, and the fifth task instruction is used to instruct the target server to execute the current task and store the target key in the target database.

8. A task scheduling device for a server cluster, characterized in that: include: The acquisition module is used to obtain the current task of any target server in the server cluster; A processing module, used to load and process the target function corresponding to the current task through a decorator function to obtain the task identifier of the current task; query the task identifier in a target database to obtain a query result; and obtain the properties of the target function; A task instruction is generated according to the query result and the properties of the target function; the task instruction is sent to the target server, so that the target server executes the current task or obtains the next target task according to the task instruction.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the method according to any one of claims 1 to 7 is performed.

10. A computer-readable storage medium, characterized in that: Instructions are stored, and when the instructions are executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 7.

Images