Task processing method and related device

Abstract

The application discloses a task processing method and related equipment, which can determine a distributed cluster system for executing tasks and at least one task to be distributed; obtain a task weight corresponding to each task and set a running instance weight corresponding to each node machine in the distributed cluster system; determine at least one workable node machine from the node machines of the distributed cluster system; when the workable node machine is started, distribute each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task, and output a task processing result. The application can distribute tasks to multiple node machines for processing by increasing the task weight of the tasks and the running instance weight of the node machines, which is beneficial to load balancing of the tasks, improves uniformity of task distribution, and can avoid a situation that a task cannot be executed after a certain machine fails.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a task processing method and related equipment. Background Technology

[0002] Market data in financial market data systems can typically be categorized into multiple product types, such as foreign exchange, investor relations, volatility, futures, stocks, and bonds. Access to each product type is referred to as a task.

[0003] In current technologies, all tasks are typically executed on a single machine. This can lead to these tasks consuming a significant amount of the machine's hardware resources, and if the machine crashes, the tasks that were running on it may be unable to continue executing. Summary of the Invention

[0004] This application provides a task processing method and related equipment. The related equipment may include a task processing device, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the uniformity of task allocation and facilitate the achievement of task load balancing.

[0005] This application provides a task processing method, including:

[0006] Determine a distributed cluster system for performing tasks, and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine;

[0007] Obtain the task weight corresponding to each task, and set the running instance weight corresponding to each node machine in the distributed cluster system;

[0008] Determine at least one workable node machine from the node machines of the distributed cluster system;

[0009] When the workable node machine starts, each task is assigned to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task, and the task processing results are output.

[0010] Accordingly, embodiments of this application provide a task processing apparatus, including:

[0011] The first determining unit is used to determine a distributed cluster system for executing tasks and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine.

[0012] The setting unit is used to obtain the task weight corresponding to each task and set the running instance weight corresponding to each node machine in the distributed cluster system.

[0013] The second determining unit is used to determine at least one workable node machine from the node machines of the distributed cluster system.

[0014] The processing unit is used to allocate each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task when the workable node machine is started, and output the task processing results.

[0015] An electronic device provided in this application includes a processor and a memory. The memory stores multiple instructions, and the processor loads the instructions to execute the steps in the task processing method provided in this application.

[0016] This application also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps in the task processing method provided in this application.

[0017] Furthermore, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps in the task processing method provided in embodiments of this application.

[0018] This application provides a task processing method and related equipment. It can determine a distributed cluster system for executing tasks and at least one task to be assigned. The distributed cluster system includes at least one node machine. The method obtains the task weight corresponding to each task and sets the running instance weight corresponding to each node machine in the distributed cluster system. It determines at least one workable node machine from the node machines in the distributed cluster system. When the workable node machine starts, it assigns each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task, and outputs the task processing result. This application can distribute tasks to multiple node machines for processing by increasing the task weight and the running instance weight of the node machine. This is beneficial for achieving task load balancing, improving the uniformity of task allocation, and avoiding the situation where a machine fails and cannot execute a task. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1aThis is a schematic diagram of a scenario for the task processing method provided in an embodiment of this application;

[0021] Figure 1b This is a flowchart of the task processing method provided in the embodiments of this application;

[0022] Figure 2 This is another flowchart of the task processing method provided in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram of the structure of the task processing device provided in the embodiments of this application;

[0024] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] This application provides a task processing method and related equipment. The related equipment may include a task processing device, an electronic device, a computer-readable storage medium, and a computer program product. Specifically, the task processing device may be integrated into an electronic device, which may be a terminal or a server, etc.

[0027] It is understood that the task processing method of this embodiment can be executed on a terminal, on a server, or jointly by a terminal and a server. The above examples should not be construed as limiting this application.

[0028] like Figure 1a As shown, the method of jointly executing task processing by a terminal and a server is taken as an example. The task processing system provided in this application embodiment includes a terminal 10 and a server 11, etc.; the terminal 10 and the server 11 are connected through a network, such as through a wired or wireless network, etc., wherein the task processing device can be integrated into the server.

[0029] Server 11 can be used to: determine a distributed cluster system for executing tasks and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine; obtain the task weight corresponding to each task and set the running instance weight corresponding to each node machine in the distributed cluster system; determine at least one workable node machine from the node machines of the distributed cluster system; when the workable node machine starts, assign each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task, and output the task processing results. Server 11 can be a single server, a server cluster composed of multiple servers, or a cloud server.

[0030] Terminal 10 can be used to receive task processing results output by server 11. Terminal 10 may include a mobile phone, smart TV, tablet computer, laptop computer, or personal computer (PC), etc. A client can also be configured on terminal 10, which may be an application client or a browser client, etc.

[0031] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0032] This embodiment will be described from the perspective of a task processing device, which can be integrated into an electronic device, such as a server or a terminal.

[0033] like Figure 1b As shown, the specific process of this task processing method can be as follows:

[0034] 101. Determine a distributed cluster system for performing tasks and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine.

[0035] Specifically, the distributed cluster system can be a ZooKeeper cluster, which may include at least two distributed node machines.

[0036] ZooKeeper is a highly available, high-performance, and consistent open-source coordination service framework designed for distributed applications. It can implement distributed basic services such as synchronization services, configuration maintenance, naming services, and distributed locks. Its interface is simple, easy to use, and has stable performance.

[0037] In specific scenarios, market data in financial market data systems (MDS) can typically be divided into multiple product categories, such as foreign exchange (FX), investor relations (IR), volatility (VolSurf), futures, stocks (Equity), and bonds (Bond). Access to each product category is called a task, and the TPS (Transactions Per Second) generally differs for different product categories.

[0038] TPS can represent the system throughput, which is the number of transactions the system can process per second.

[0039] In current technologies, there is typically only one node machine executing tasks. Once the node machine starts, it must handle and execute all tasks. This can lead to tasks consuming excessive hardware resources, or the machine failing and unable to continue executing tasks. In such cases, the only solutions are to restart the application or redeploy the tasks by increasing machine resources.

[0040] The task processing method provided in this application provides a processing technology for dynamically allocating tasks based on multi-task and host weight configuration. This technology can add weight attributes to tasks and node machines, thereby achieving load balancing in a business sense. It can also support the dynamic addition of tasks and the dynamic addition and removal of node machines. When a node machine changes, the task can be smoothly migrated to other node machines for execution. In addition, this application can also support the even distribution of tasks to different node machines for processing according to product type.

[0041] 102. Obtain the task weight corresponding to each task, and set the running instance weight corresponding to each node machine in the distributed cluster system.

[0042] The task weights for each task can be set according to actual conditions. Specifically, the task weight can be determined by the number of tasks a node machine can process within a preset time period. This preset time period can be set according to actual conditions, such as 1 hour or 1 second. Generally, the more tasks that can be processed within the preset time period, the lower the task weight; conversely, the fewer tasks that can be processed within the preset time period, the higher the task weight.

[0043] The running instance weight corresponding to the node machine can include the actual running instance weight and the expected running instance weight. Specifically, the actual running instance weight of the node machine can be determined based on the task weight of each task running on the node machine. For example, the actual running instance weight of the node machine can be obtained by adding the task weights of each task running on the node machine. The expected running instance weight of the node machine can be set according to the actual situation, such as the machine performance and service life of the node machine.

[0044] Optionally, in this embodiment, the step "obtaining the task weight corresponding to each task" may include:

[0045] Determine the type of financial product corresponding to each task;

[0046] The task weights for each task are determined based on the throughput corresponding to the types of financial products.

[0047] The types of financial products can include foreign exchange (FX), investor relations (IR), volatility (VolSurf), futures, stocks (Equity), and bonds (Bond).

[0048] The higher the throughput, the lower the corresponding task weight; conversely, the lower the throughput, the higher the corresponding task weight.

[0049] 103. Determine at least one workable node machine from the node machines of the distributed cluster system.

[0050] Here, the node machines that can perform tasks are specifically the node machines currently selected for executing tasks.

[0051] 104. When the workable node machine starts, according to the running instance weight corresponding to the workable node machine and the task weight of each task, each task is assigned to the workable node machine for processing, and the task processing results are output.

[0052] Specifically, the weight of the running instance corresponding to the jobable node machine can include the actual running instance weight and the expected running instance weight. The actual running instance weight can be determined based on the task weight of each task running on the jobable node machine.

[0053] Optionally, in this embodiment, the step "when the workable node machine starts, according to the running instance weight corresponding to the workable node machine and the task weight of each task, assign each task to the workable node machine for processing, and output the task processing result" may include:

[0054] When the workable node machine starts, determine the startup timestamp of each workable node machine;

[0055] Based on the startup timestamp, determine the sleep waiting time for each workable node machine;

[0056] When the waiting time after the start-up of the workable node machine exceeds the corresponding sleep waiting time, the workable node machine is triggered to compete for tasks based on its corresponding running instance weight and the task weight of each task, so that the workable node machine can process the tasks it wins and output the task processing results.

[0057] Specifically, the startup timestamps of different workable node machines may be different; the startup timestamp is the startup time of the node machine. To prevent the first workable node machine from taking over all tasks, a sleep waiting time can be set. This allows the node machine to not take over tasks immediately after startup, but rather after a certain period of time, which is the sleep waiting time. This allows all workable node machines to compete for tasks simultaneously, preventing all tasks from running on a single node machine.

[0058] Among them, the earlier the start time of a workable node machine, the longer its sleep waiting time; conversely, the later the start time of a workable node machine, the shorter its sleep waiting time.

[0059] In specific scenarios, tasks to be assigned can be retrieved from Apollo, added to the local cache, and then added to ZooKeeper (ZK) in a loop. Additionally, the execution instance weight of the node machine needs to be set (using Apollo's zk.totalWeight configuration), and tasks need to be added with their corresponding task weights set (using Apollo's taskId configuration).

[0060] Apollo is a distributed configuration center that can centrally manage the configurations of applications in different environments and clusters. After the configuration is modified, it can be pushed to the application in real time. It also has standardized permissions, process governance and other features, making it suitable for microservice configuration management scenarios.

[0061] Specifically, in the scenario of initializing and starting node machines, a target start timestamp can be added in advance. The target start timestamp can be a preset time for each node machine to compete for tasks, which can be set according to the actual situation. Then, the start timestamps of all node machines are obtained. If the smallest start timestamp plus the waiting time is less than or equal to the timestamp, the node machine is started directly; if the smallest start timestamp plus the waiting time is greater than the timestamp, the node machine needs to wait for that long before starting and taking over the tasks. By adding the waiting time, the node machine that starts first is prevented from taking over all tasks. The waiting time can be configured through waitSeconds in seconds. For example, zk.waitSeconds can be set to a 60-second preemptive sleep time.

[0062] This also includes starting a caching service and activating a task; specifically, setting a task worker, calling the beforeActive interface, attempting to compete for the task, and gaining the right to run if the competition is successful, and entering an exception branch if the competition fails.

[0063] For example, in a certain startup scenario, if only one machine is started and all six tasks (FX, IR, VolSurf, Futures, Equity, Bond) are activated, then machine 1 will take over these six tasks.

[0064] For example, in another startup scenario, two node machines (machine 1 and machine 2) are started simultaneously. The expected weight of the running instance on each node machine is set to 4. There are tasks to be assigned: FX, IR, VolSurf, Futures, Equity, and Bond. The task weights of tasks FX and Equity are both 2, while the task weights of tasks IR, VolSurf, Futures, and Bond are all 1. Then, tasks FX and Equity can be assigned to machine 1 to run, and tasks IR, VolSurf, Futures, and Bond can be assigned to machine 2 to run.

[0065] Specifically, if machine 1 starts up successfully at 51:02, its sleep waiting time can be 60 seconds, and then machine 1 will start preempting tasks at 52:02; if machine 2 starts up successfully at 51:32, its sleep waiting time can be 30 seconds, and then machine 2 will start preempting tasks at 52:02.

[0066] For example, in another startup scenario, three machines (machine 1, machine 2, and machine 3) are started simultaneously. The expected execution instance weights of machines 1, 2, and 3 are set to 3, 3, and 2, respectively. There are tasks to be assigned: FX, IR, VolSurf, Futures, Equity, and Bond. The task weight of task FX is 3, and the task weights of tasks VolSurf, Futures, Bond, IR, and Equity are all 1. Then, task FX can be assigned to machine 1, tasks VolSurf, Futures, and Bond can be assigned to machine 2, and tasks IR and Equity can be assigned to machine 3.

[0067] Optionally, in this embodiment, the task processing method may further include:

[0068] When the target workable node machine in the at least one workable node machine stops working, the target task processed on the target workable node machine is released.

[0069] Based on the running instance weight and number of machines corresponding to the currently available workable node machines, determine the target node machine to take over the target task from the currently available workable node machines;

[0070] The target task is assigned to the target node machine for processing, and the task processing result is output.

[0071] The target workable node machine can be any node machine from at least one workable node machine. If, after the target workable node machine stops working, the number of remaining currently workable node machines is 1, then all target tasks can be transferred to that currently workable node machine for execution. If, after the target workable node machine stops working, the number of remaining currently workable node machines is greater than 1, then the target tasks can be transferred to which currently workable node machine to continue running based on the running instance weight of the remaining currently workable node machines.

[0072] Optionally, in this embodiment, the step "determining the target node machine to take over the target task from the currently available workable node machines based on the running instance weight and number of machines corresponding to the currently available workable node machines" may include:

[0073] When the number of currently available workable node machines is greater than a preset number, the target node machine to take over the target task is determined from the currently available workable node machines based on the actual running instance weight and the expected running instance weight corresponding to the currently available workable node machine.

[0074] The preset quantity can specifically be 1.

[0075] In one embodiment, if the number of currently available workable node machines is greater than 1, it is determined whether the actual running instance weight of each currently available workable node machine is greater than its expected running instance weight; if the actual running instance weight of a certain currently available workable node machine is greater than or equal to its expected running instance weight, then the currently available workable node machine may not take over the target task; if the actual running instance weight of a certain currently available workable node machine is less than its expected running instance weight, then the currently available workable node machine may take over the target task.

[0076] In one embodiment, if a node machine stops, the tasks on that node machine need to be transferred to other node machines for execution. Therefore, tasks can be reallocated. Specifically, when a node machine stops working, other nodes can compete for the target tasks released by that node machine. For each node machine, a competition time can be set, which can be specifically set as the node machine's current actual running instance weight / expected running instance weight * 1000 milliseconds. When a node machine reaches its competition time, it begins competing for tasks.

[0077] For example, in a scenario where a node machine stops working, if only one machine is started and all six tasks (FX, IR, VolSurf, Futures, Equity, Bond) are activated, then machine 1 will take over these six tasks; if machine 1 stops working, then all tasks will stop running.

[0078] For example, in a scenario where one node machine stops working, two node machines (machine 1 and machine 2) are started simultaneously. The expected weight of the running instances on each node machine is set to 4. Machine 1 runs tasks FX and Equity, while machine 2 runs tasks IR, VolSurf, Futures, and Bond. The task weights for tasks FX and Equity are both 2, while the task weights for tasks IR, VolSurf, Futures, and Bond are all 1. If machine 1 stops working, the tasks (FX and Equity) released by machine 1 are taken over by machine 2. Finally, the tasks running on machine 2 include: FX, IR, VolSurf, Futures, Equity, and Bond.

[0079] For example, in a scenario where another node machine stops working, three machines (machine 1, machine 2, and machine 3) are started simultaneously. The expected execution instance weights of machines 1, 2, and 3 are all 3. Machine 1 runs task FX, machine 2 runs tasks VolSurf, Futures, and Bond, and machine 3 runs tasks IR and Equity. Task FX has a task weight of 3, while tasks VolSurf, Futures, Bond, IR, and Equity each have a task weight of 1. If machine 1 stops working, it releases task FX, which can then be taken over by machine 3. Finally, the tasks running on machine 3 include IR, Equity, and FX.

[0080] Optionally, in this embodiment, the task processing method may further include:

[0081] When a target node machine that is currently in a non-working state is selected from the distributed cluster system as a new node machine for executing tasks, the target task that needs to be released on the working node machine is determined based on the running instance weight of the working node machine and the running instance weight of the target node machine.

[0082] Release the target task on the workable node machine, assign the target task to the target node machine for processing, and output the task processing result.

[0083] The machine status can include job status and non-job status. Job status means that the machine is performing a task, while non-job status means that the machine is not performing a task.

[0084] Optionally, in this embodiment, the step "determine the target task to be released on the workable node machine based on the running instance weight of the workable node machine and the running instance weight of the target node machine" may include:

[0085] The actual running instance weight of the current working node machine is determined based on the task weight corresponding to the task running on the working node machine.

[0086] When the current actual running instance weight of the workable node machine is greater than the expected running instance weight of the workable node machine, the target task that needs to be released on the workable node machine is determined based on the running instance weight of the workable node machine and the running instance weight of the target node machine.

[0087] Specifically, the task weights corresponding to each task running on a workable node machine can be added together to obtain the current actual running instance weight of that workable node machine.

[0088] If the actual running instance weight of a jobable node machine is greater than the expected running instance weight, then the tasks running on that jobable node machine need to be released. Specifically, some tasks running on that jobable node machine can be released so that the actual running instance weight of the jobable node machine is not greater than the expected running instance weight. These released tasks are then set to a waiting state so that other node machines can compete to take over again.

[0089] In one embodiment, if a new node machine is added to perform the task, i.e., a new instance joins the task processing process, the task can be reallocated to make the task allocation more reasonable.

[0090] For example, in a scenario where a node machine is added to process tasks: if only one machine is started and all 6 tasks (FX, IR, VolSurf, Futures, Equity, Bond) are activated, then machine 1 will take over these 6 tasks; if machine 2 is added, then machine 1 can kick out tasks FX, IR, and VolSurf, so that the actual running instance weight of machine 1 is 3, and machine 2 will take over the kicked-out tasks FX, IR, and VolSurf.

[0091] For example, in another scenario involving adding more node machines to process tasks: Two node machines (machine 1 and machine 2) are started simultaneously, each with an actual running instance weight of 4. Machine 1 runs tasks FX and Equity, while machine 2 runs tasks IR, VolSurf, Futures, and Bond. Tasks FX and Equity each have a weight of 2, while tasks IR, VolSurf, Futures, and Bond each have a weight of 1. If machine 3 is added, machine 1 can remove task FX or Equity, ensuring its actual running instance weight does not exceed 3. Machine 2 can remove tasks IR, VolSurf, Futures, or Bond, ensuring its actual running instance weight is 3. Machine 3 can then take over the tasks removed by machines 1 and 2.

[0092] For example, in another scenario where additional node machines are added to handle tasks: Three node machines (machine 1, machine 2, and machine 3) are started simultaneously. The expected execution instance weights of machines 1, 2, and 3 are all 3. Machine 1 runs task FX, machine 2 runs tasks VolSurf, Futures, and Bond, and machine 3 runs tasks IR and Equity. Task FX has a task weight of 3, while tasks VolSurf, Futures, Bond, IR, and Equity each have a task weight of 1. If machine 4 is added, since the actual execution instance weights of machines 1, 2, and 3 do not exceed the expected execution instance weight of 3, there is no need to remove tasks, and machine 4 does not need to take over the tasks.

[0093] As can be seen from the above, this embodiment can determine a distributed cluster system for executing tasks and at least one task to be assigned. The distributed cluster system includes at least one node machine. It obtains the task weight corresponding to each task and sets the running instance weight corresponding to each node machine in the distributed cluster system. It determines at least one workable node machine from the node machines of the distributed cluster system. When the workable node machine starts, according to the running instance weight corresponding to the workable node machine and the task weight of each task, each task is assigned to the workable node machine for processing, and the task processing result is output. This application can distribute tasks to multiple node machines for processing by increasing the task weight and the running instance weight of the node machine. This is beneficial for achieving task load balancing, improving the uniformity of task allocation, and avoiding the situation where a task cannot be executed after a machine fails.

[0094] Based on the method described in the preceding embodiments, the following will provide a more detailed explanation by taking the specific integration of the task processing device into a server as an example.

[0095] This application provides a task processing method, such as... Figure 2 As shown, the specific process of this task processing method can be as follows:

[0096] 201. The server determines a distributed cluster system for executing tasks and at least one task to be assigned, the distributed cluster system comprising at least one node machine.

[0097] Specifically, the distributed cluster system can be a ZooKeeper cluster, which may include at least two distributed node machines.

[0098] In specific scenarios, market data in financial market data systems (MDS) can typically be divided into multiple product categories, such as foreign exchange (FX), investor relations (IR), volatility (VolSurf), futures, stocks (Equity), and bonds (Bond). Access to each product category is called a task, and the TPS (Transactions Per Second) generally differs for different product categories.

[0099] 202. The server obtains the task weight corresponding to each task and sets the running instance weight corresponding to each node machine in the distributed cluster system.

[0100] The task weights for each task can be set according to actual conditions. Specifically, the task weight can be determined by the number of tasks a node machine can process within a preset time period. This preset time period can be set according to actual conditions, such as 1 hour or 1 second. Generally, the more tasks that can be processed within the preset time period, the lower the task weight; conversely, the fewer tasks that can be processed within the preset time period, the higher the task weight.

[0101] The running instance weight corresponding to the node machine can include the actual running instance weight and the expected running instance weight. Specifically, the actual running instance weight of the node machine can be determined based on the task weight of each task running on the node machine. For example, the actual running instance weight of the node machine can be obtained by adding the task weights of each task running on the node machine. The expected running instance weight of the node machine can be set according to the actual situation, such as the machine performance and service life of the node machine.

[0102] 203. The server determines at least one workable node machine from the node machines of the distributed cluster system.

[0103] 204. When the available node machines start, the server determines the startup timestamp of each available node machine.

[0104] 205. The server determines the sleep waiting time for each workable node machine based on the startup timestamp.

[0105] 206. When the waiting time after the start of the workable node machine exceeds the corresponding sleep waiting time, the server triggers the workable node machine to compete for tasks based on its corresponding running instance weight and the task weight of each task, so that the workable node machine can process the tasks it wins and output the task processing results.

[0106] Specifically, the startup timestamps of different workable node machines may be different; the startup timestamp is the startup time of the node machine. To prevent the first workable node machine from taking over all tasks, a sleep waiting time can be set. This allows the node machine to not take over tasks immediately after startup, but rather after a certain period of time, which is the sleep waiting time. This allows all workable node machines to compete for tasks simultaneously, preventing all tasks from running on a single node machine.

[0107] Among them, the earlier the start time of a workable node machine, the longer its sleep waiting time; conversely, the later the start time of a workable node machine, the shorter its sleep waiting time.

[0108] The task processing method provided in this application provides a processing technology for dynamically allocating tasks based on multi-task and host weight configuration. This technology can add weight attributes to tasks and node machines, thereby achieving load balancing in a business sense. It can also support the dynamic addition of tasks and the dynamic addition and removal of node machines. When a node machine changes, the task can be smoothly migrated to other node machines for execution. In addition, this application can also support the even distribution of tasks to different node machines for processing according to product type.

[0109] As can be seen from the above, this embodiment can determine the distributed cluster system for executing tasks and at least one task to be assigned through the server. The distributed cluster system includes at least one node machine. It obtains the task weight corresponding to each task and sets the running instance weight corresponding to each node machine in the distributed cluster system. It determines at least one workable node machine from the node machines in the distributed cluster system. When the workable node machine starts, it determines the start timestamp of each workable node machine. Based on the start timestamp, it determines the sleep waiting time of each workable node machine. When the waiting time of the workable node machine after startup exceeds the corresponding sleep waiting time, it triggers the workable node machine to compete for tasks based on its corresponding running instance weight and the task weight of each task, so that the workable node machine processes the competed task and outputs the task processing result. This application can distribute tasks to multiple node machines for processing by increasing the task weight and the running instance weight of the node machine. This is beneficial for achieving task load balancing, improving the uniformity of task allocation, and avoiding the situation where a machine cannot execute tasks after failure.

[0110] To better implement the above methods, embodiments of this application also provide a task processing device, such as... Figure 3 As shown, the task processing device may include a first determining unit 301, a setting unit 302, a second determining unit 303, and a processing unit 304, as follows:

[0111] (1) First determining unit 301;

[0112] The first determining unit is used to determine a distributed cluster system for executing tasks and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine.

[0113] (2) Set unit 302;

[0114] The setting unit is used to obtain the task weight corresponding to each task and set the running instance weight corresponding to each node machine in the distributed cluster system.

[0115] (3) Second determining unit 303;

[0116] The second determining unit is used to determine at least one workable node machine from the node machines of the distributed cluster system.

[0117] (4) Processing unit 304;

[0118] The processing unit is used to allocate each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task when the workable node machine is started, and output the task processing results.

[0119] Optionally, in some embodiments of this application, the processing unit may include a first determining subunit, a second determining subunit, and a competing subunit, as follows:

[0120] The first determining subunit is used to determine the startup timestamp of each workable node machine when the workable node machine starts.

[0121] The second determining subunit is used to determine the sleep waiting time of each workable node machine based on the start timestamp;

[0122] The competition subunit is used to trigger the workable node machine to compete for tasks based on its corresponding running instance weight and the task weight of each task when the waiting time after the start-up of the workable node machine exceeds the corresponding sleep waiting time. The workable node machine then processes the tasks it wins and outputs the task processing results.

[0123] Optionally, in some embodiments of this application, the task processing device may further include a release unit, a takeover machine determination unit, and an allocation unit, as follows:

[0124] The release unit is used to release the target task processed on the target workable node machine when the target workable node machine in the at least one workable node machine stops working.

[0125] The takeover machine determination unit is used to determine the target node machine to take over the target task from the currently available workable node machines based on the running instance weight and the number of machines corresponding to the currently available workable node machines.

[0126] The allocation unit is used to allocate the target task to the target node machine for processing and output the task processing result.

[0127] Optionally, in some embodiments of this application, the takeover machine determination unit may be specifically used to determine the target node machine to take over the target task from the current workable node machines when the number of currently workable node machines is greater than a preset number, based on the actual running instance weight and the expected running instance weight corresponding to the current workable node machine.

[0128] Optionally, in some embodiments of this application, the task processing apparatus may further include a new machine unit and a reallocation unit, as follows:

[0129] The newly added machine unit is used to determine the target task that needs to be released on the workable node machine when a target node machine whose current machine state is in a non-working state is selected from the distributed cluster system as a newly added node machine for executing tasks, based on the running instance weight of the workable node machine and the running instance weight of the target node machine.

[0130] The reallocation unit is used to release the target task on the workable node machine, allocate the target task to the target node machine for processing, and output the task processing result.

[0131] Optionally, in some embodiments of this application, the newly added machine unit may include a weight determination subunit and a task determination subunit, as follows:

[0132] The weight determination subunit is used to determine the current actual running instance weight of the workable node machine based on the task weight corresponding to the task running on the workable node machine.

[0133] The task determination subunit is used to determine the target task that needs to be released on the workable node machine based on the running instance weight of the workable node machine and the running instance weight of the target node machine when the current actual running instance weight of the workable node machine is greater than the expected running instance weight of the workable node machine.

[0134] As can be seen from the above, in this embodiment, the first determining unit 301 determines the distributed cluster system for executing tasks and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine; the setting unit 302 obtains the task weight corresponding to each task and sets the running instance weight corresponding to each node machine in the distributed cluster system; the second determining unit 303 determines at least one workable node machine from the node machines in the distributed cluster system; and the processing unit 304, when the workable node machine starts, assigns each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task, and outputs the task processing result. This application can distribute tasks to multiple node machines for processing by increasing the task weight of the task and the running instance weight of the node machine, which is beneficial to achieving task load balancing, improving the uniformity of task allocation, and avoiding the situation where a task cannot be executed after a machine fails.

[0135] This application also provides an electronic device, such as... Figure 4 The diagram shows a structural schematic of an electronic device involved in an embodiment of this application. This electronic device can be a terminal or a server, specifically:

[0136] The electronic device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 4 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0137] The processor 401 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 402, and calls data stored in the memory 402, to perform various functions and process data. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0138] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and task processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0139] The electronic device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0140] The electronic device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0141] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0142] The process involves determining a distributed cluster system for executing tasks and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine; obtaining the task weight corresponding to each task and setting the running instance weight corresponding to each node machine in the distributed cluster system; determining at least one workable node machine from the node machines in the distributed cluster system; and when the workable node machine starts, assigning each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task, and outputting the task processing results.

[0143] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0144] As can be seen from the above, this embodiment can determine a distributed cluster system for executing tasks and at least one task to be assigned. The distributed cluster system includes at least one node machine. It obtains the task weight corresponding to each task and sets the running instance weight corresponding to each node machine in the distributed cluster system. It determines at least one workable node machine from the node machines of the distributed cluster system. When the workable node machine starts, according to the running instance weight corresponding to the workable node machine and the task weight of each task, each task is assigned to the workable node machine for processing, and the task processing result is output. This application can distribute tasks to multiple node machines for processing by increasing the task weight and the running instance weight of the node machine. This is beneficial for achieving task load balancing, improving the uniformity of task allocation, and avoiding the situation where a task cannot be executed after a machine fails.

[0145] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0146] Therefore, embodiments of this application provide a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute steps in any of the task processing methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0147] The process involves determining a distributed cluster system for executing tasks and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine; obtaining the task weight corresponding to each task and setting the running instance weight corresponding to each node machine in the distributed cluster system; determining at least one workable node machine from the node machines in the distributed cluster system; and when the workable node machine starts, assigning each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task, and outputting the task processing results.

[0148] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0149] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0150] Since the instructions stored in the computer-readable storage medium can execute the steps of any of the task processing methods provided in the embodiments of this application, the beneficial effects that any of the task processing methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0151] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various alternative implementations of the above-described task processing aspects.

[0152] The above provides a detailed description of a task processing method and related equipment provided by the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A task processing method, characterized in that, include: Determine a distributed cluster system for performing tasks, and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine; Obtain the task weight corresponding to each task, and set the running instance weight corresponding to each node machine in the distributed cluster system; Determine at least one workable node machine from the node machines of the distributed cluster system; When the workable node machine starts, each task is assigned to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task, and the task processing result is output. The step of obtaining the task weight corresponding to each task includes: Determine the type of financial product corresponding to each task; The task weights for each task are determined based on the throughput corresponding to the types of financial products. When the workable node machine starts, according to the running instance weight corresponding to the workable node machine and the task weight of each task, each task is assigned to the workable node machine for processing, and the task processing results are output, including: When the workable node machine starts, determine the startup timestamp of each workable node machine; Based on the startup timestamp, the sleep waiting time of each workable node machine is determined; the sleep waiting time is used to prevent the workable node machine from immediately taking over the task after startup. When the waiting time after the start-up of the workable node machine exceeds the corresponding sleep waiting time, the workable node machine is triggered to compete for tasks based on its corresponding running instance weight and the task weight of each task, so that the workable node machine can process the tasks it wins and output the task processing results.

2. The method according to claim 1, characterized in that, The method further includes: When the target workable node machine in the at least one workable node machine stops working, the target task processed on the target workable node machine is released. Based on the running instance weight and number of machines corresponding to the currently available workable node machines, determine the target node machine to take over the target task from the currently available workable node machines; The target task is assigned to the target node machine for processing, and the task processing result is output.

3. The method according to claim 2, characterized in that, The step of determining the target node machine to take over the target task from the currently available workable node machines based on the running instance weight and number of machines corresponding to the currently available workable node machines includes: When the number of currently available workable node machines is greater than a preset number, the target node machine to take over the target task is determined from the currently available workable node machines based on the actual running instance weight and the expected running instance weight corresponding to the currently available workable node machine.

4. The method according to claim 1, characterized in that, The method further includes: When a target node machine that is currently in a non-working state is selected from the distributed cluster system as a new node machine for executing tasks, the target task that needs to be released on the working node machine is determined based on the running instance weight of the working node machine and the running instance weight of the target node machine. Release the target task on the workable node machine, assign the target task to the target node machine for processing, and output the task processing result.

5. The method according to claim 4, characterized in that, The step of determining the target task to be released on the workable node machine based on the running instance weight of the workable node machine and the running instance weight of the target node machine includes: The actual running instance weight of the current working node machine is determined based on the task weight corresponding to the task running on the working node machine. When the current actual running instance weight of the workable node machine is greater than the expected running instance weight of the workable node machine, the target task that needs to be released on the workable node machine is determined based on the running instance weight of the workable node machine and the running instance weight of the target node machine.

6. A task processing device, characterized in that, include: The first determining unit is used to determine a distributed cluster system for executing tasks and at least one task to be assigned, wherein the distributed cluster system includes at least one node machine. The setting unit is used to obtain the task weight corresponding to each task and set the running instance weight corresponding to each node machine in the distributed cluster system. The second determining unit is used to determine at least one workable node machine from the node machines of the distributed cluster system. The processing unit is used to allocate each task to the workable node machine for processing according to the running instance weight corresponding to the workable node machine and the task weight of each task when the workable node machine is started, and output the task processing result. The processing unit includes a first determining subunit, a second determining subunit, and a competing subunit, as follows: The first determining subunit is used to determine the startup timestamp of each workable node machine when the workable node machine starts. The second determining subunit is used to determine the sleep waiting time of each workable node machine based on the start timestamp; The sleep waiting time is used to prevent the job node machine from immediately taking over the task after it starts up; The competition subunit is used to trigger the workable node machine to compete for tasks based on its corresponding running instance weight and the task weight of each task when the waiting time after the start-up of the workable node machine exceeds the corresponding sleep waiting time. The workable node machine then processes the tasks it wins and outputs the task processing results. The step of obtaining the task weight corresponding to each task includes: Determine the type of financial product corresponding to each task; The task weights for each task are determined based on the throughput corresponding to the types of financial products.

7. An electronic device, characterized in that, It includes a memory and a processor; the memory stores an application program, and the processor is used to run the application program within the memory to perform the operations in the task processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to perform the steps of the task processing method according to any one of claims 1 to 5.

9. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the steps of the task processing method according to any one of claims 1 to 5.

Images