Task scheduling processing method, device, computer equipment and readable storage medium

Abstract

The present application relates to the technical field of computer systems, and provides a task scheduling processing method, apparatus, computer equipment, and readable storage medium, including: receiving task information sent by any service node in a multi-node cluster, and setting the service node that sends the task information as the first node; setting one of the service nodes other than the first node in the multi-node cluster as the second node; controlling the second node to process the task information to obtain the task result, and obtaining the execution log and thread status generated in the process of processing the task information; if the thread status is confirmed to be an execution exception or an execution interruption, then controlling the second node to re-process the task information to obtain the task result, and obtaining the execution log corresponding to the thread status and sending the execution log to the control end. The present application balances the task volume of each service node in the multi-node cluster, and improves the overall efficiency of the multi-node cluster in processing task information.

Description

Technical Field

[0001] The present application relates to the technical field of computer systems, and in particular to a task scheduling processing method, apparatus, computer equipment, and readable storage medium. Background Art

[0002] For task information that needs to be executed in batches, the task information is usually sent to a load balancer in a multi-node cluster, and then sent to each service node in the multi-node cluster through the load balancer; wherein the multi-node cluster is a server cluster constructed by connecting a load balancer to multiple servers for processing task information; each server in the server cluster is a service node.

[0003] However, the inventors have discovered that the current load balancer of a multi-node cluster cannot fully utilize the computing resources of each service node, which easily leads to a situation where some service nodes need to process too much task information while other service nodes need to process too little task information, resulting in the problem of low efficiency in task information processing in the multi-node cluster as a whole. Summary of the invention

[0004] The present application provides a task scheduling processing method, apparatus, computer equipment and readable storage medium to solve the problem that the current load balancer of a multi-node cluster cannot fully utilize the computing resources of each service node, which easily leads to the situation that some service nodes need to process too much task information, while other service nodes need to process too little task information, resulting in low efficiency of task information processing in the multi-node cluster as a whole.

[0005] In a first aspect, the present application provides a task scheduling processing method, the method being applied to a multi-threaded component in a multi-node cluster, the multi-node cluster comprising: the multi-threaded component and a plurality of service nodes; the method comprising:

[0006] Receiving task information sent by any service node in the multi-node cluster, and setting the service node that sends the task information as the first node;

[0007] In the multi-node cluster, one of the service nodes other than the first node is set as the second node; wherein, one of the service nodes other than the first node is a service node having an idle position in a task queue and sending a response message; the task queue is a linear table for arranging at least one task information; the response message indicates that the other service node having the idle position requests to process the task information;

[0008] Control the second node to process the task information to obtain a task result, and obtain an execution log and a thread status generated in the process of processing the task information; wherein the execution log describes the process of processing the task information; the thread status reflects the running status in the process of processing the task information; the thread status includes: normal execution, abnormal execution and interrupted execution;

[0009] If it is confirmed that the thread state is an execution exception or an execution interruption, the second node is controlled to re-process the task information to obtain a task result, and an execution log corresponding to the thread state is obtained and sent to the control end.

[0010] In the above solution, setting one of the service nodes other than the first node in the multi-node cluster as the second node includes:

[0011] Generating task parameters according to the task information, and saving the task parameters in a database of the multi-node cluster;

[0012] Generate a task notification according to the task information, and send the task notification to other service nodes in the multi-node cluster except the first node;

[0013] If a response message sent by at least one of the other service nodes is received, the preset distributed lock is called, and the other service node corresponding to the first response message received is set as the second node.

[0014] In the above solution, after sending the task notification to other service nodes in the multi-node cluster except the first node, the method further includes:

[0015] Control each of the other service nodes to view the task queue in each of the other service nodes;

[0016] If it is confirmed that the task queue of one of the other service nodes has a free position, feedback information is sent to the other service node; wherein the feedback information is used to instruct the other service node to send the response information.

[0017] In the above solution, controlling the second node to process the task information to obtain the task result, and obtaining the execution log and thread status generated in the process of processing the task information, includes:

[0018] Acquire task parameters corresponding to the task information from the database, and create a task thread for implementing the task information according to the task parameters;

[0019] Inserting a monitoring aspect into the task thread to obtain a proxy thread; wherein the monitoring aspect is a combination of a notification and an entry point for recording an execution log and a thread state of the task thread during execution; the execution log is log information generated when the task thread executes to a position corresponding to the notification and the entry point, and each log information corresponding to the notification and the entry point describes the process of processing the task information; the thread state is the running state of the task thread when the task thread executes to a position corresponding to the notification and the entry point, and each running state corresponding to the notification and the entry point reflects the running status during the processing of the task information;

[0020] Loading the proxy thread into a task queue in the second node;

[0021] The thread pool of the second node is controlled to execute the task thread in the proxy thread in the task queue, obtain the task result corresponding to the task information, and execute the monitoring aspect in the proxy thread to obtain the execution log and thread status of the task information.

[0022] In the above solution, after sending the task notification to other service nodes in the multi-node cluster except the first node, the method further includes:

[0023] Setting the status information of the task information to a new status, and saving the status information to the database, and associating it with the task parameters corresponding to the task information;

[0024] After setting the other service node corresponding to the received first response information as the second node, the method further includes:

[0025] Setting the status information of the task information to a waiting state, and saving the status information to the database, and associating the status information with the task parameters corresponding to the task information;

[0026] After loading the proxy thread into the task queue in the second node, the method further includes:

[0027] Setting the status information of the task information as an execution status, and saving the status information into the database, and associating it with the task parameters corresponding to the task information;

[0028] After obtaining the task result corresponding to the task information, the method further includes:

[0029] The status information of the task information is set to a completed state, and the status information is saved in the database and associated with the task parameters corresponding to the task information.

[0030] In the above solution, after setting one of the service nodes other than the first node in the multi-node cluster as the second node, the method further includes:

[0031] Responding to a restart signal; wherein the restart signal is instruction information for restarting the multi-node cluster;

[0032] According to the restart signal, the task parameter of the task state in the database is set as the first parameter, and the distributed lock is called to set the other service node corresponding to the first received response information corresponding to the first parameter as the second node; and / or

[0033] According to the restart signal, the task parameter of the task state in the database is set to a second parameter, and the second parameter is obtained from the database, and a task thread for implementing the task information is created according to the second parameter; and / or

[0034] According to the restart signal, the task parameter of the task status in the database is set to the third parameter, and the thread pool of the second node is controlled to execute the proxy thread corresponding to the third parameter in the task queue to obtain the task result corresponding to the task information.

[0035] In the above solution, before receiving the task information sent by any service node in the multi-node cluster, the method further includes:

[0036] Acquire at least one task type; wherein the task type represents a type of service task;

[0037] Registering each of the service nodes respectively, so that each of the service nodes is associated with the multi-threaded component;

[0038] Constructing a thread creator according to each of the task types;

[0039] A thread pool and a task queue are created in each of the service nodes according to each of the task types.

[0040] In a second aspect, the present application provides a task scheduling processing device, wherein the device is installed in a multi-threaded component in a multi-node cluster, wherein the multi-node cluster includes: the multi-threaded component and a plurality of service nodes, and the device includes:

[0041] An input module, used for receiving task information sent by any service node in the multi-node cluster, and setting the service node sending the task information as the first node;

[0042] A processing module, used for setting one of the service nodes other than the first node in the multi-node cluster as a second node; wherein one of the service nodes other than the first node is a service node having an idle position in a task queue and sending a response message; the task queue is a linear table for arranging at least one task information; the response message indicates that the other service node having the idle position requests to process the task information;

[0043] A control module, used to control the second node to process the task information to obtain a task result, and to obtain an execution log and a thread status generated in the process of processing the task information; wherein the execution log describes the process of processing the task information; the thread status reflects the running status in the process of processing the task information; the thread status includes: normal execution, abnormal execution and interrupted execution;

[0044] The retry module is used to control the second node to reprocess the task information to obtain the task result if it is confirmed that the thread state is an execution exception or an execution interruption, and to obtain the execution log corresponding to the thread state and send the execution log to the control end.

[0045] In a third aspect, the present application provides a computer device, comprising: a processor and a memory communicatively connected to the processor;

[0046] The memory stores computer-executable instructions;

[0047] The processor executes the computer-executable instructions stored in the memory to implement the task scheduling processing method as claimed in claim 1.

[0048] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the above-mentioned task scheduling processing method.

[0049] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned task scheduling processing method when executed by a processor.

[0050] The present application provides a task scheduling processing method, apparatus, computer equipment and readable storage medium, which receive task information sent by any service node in the multi-node cluster, and set the service node that sends the task information as the first node; set one of the service nodes other than the first node in the multi-node cluster as the second node; control the second node to process the task information to obtain the task result, so that the task information sent by the first node is processed by the second node to balance the task amount of each service node in the multi-node cluster, avoid the situation that some service nodes need to process too much task information, while other service nodes need to process too little task information, thereby improving the overall processing efficiency of the multi-node cluster for task information.

[0051] At the same time, by acquiring the execution log and thread status generated in the process of processing the task information, the process of processing the task information and the running status in the process of processing the task information are obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0053] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0054] Figure 2 A flowchart of Example 1 of a task scheduling processing method provided in an embodiment of the present application;

[0055] Figure 3 A flowchart of Example 2 of a task scheduling processing method provided in an embodiment of the present application;

[0056] Figure 4 A schematic diagram of a program module of a task scheduling processing device provided by the present invention;

[0057] Figure 5 The figure is a schematic diagram of the hardware structure of the computer device in the computer device of the present invention.

[0058] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0059] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0060] See also Figure 1 , the specific application scenarios of this application are:

[0061] The multi-node cluster 3 includes: a multi-threaded component 31 and multiple service nodes 32. The task scheduling processing method runs in the control module 2 in the multi-threaded component 31, wherein the service node 32 is a server for processing task information and obtaining task results; the control module 2 receives task information sent by any service node 32 in the multi-node cluster 3, and sets the service node 32 that sends the task information as the first node; the control module 2 sets one of the service nodes 32 in the multi-node cluster 3 except the first node as the second node; the control module 2 controls the second node to process the task information and obtain the task result.

[0062] The technical solution of the present application and how the technical solution of the present application solves the problems of the prior art are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0063] Embodiment 1:

[0064] See also Figure 2 The present application provides a task scheduling processing method, which is applied to a multi-threaded component in a multi-node cluster. The multi-node cluster includes: a multi-threaded component and multiple service nodes; the method includes:

[0065] S101: Receive task information sent by any service node in a multi-node cluster, and set the service node that sends the task information as the first node.

[0066] In this step, if the amount of task information of a service node in the multi-node cluster is too large, it can be sent to the multi-threaded component, and the multi-threaded component will hand over the task information sent by the first node to other service nodes for processing.

[0067] In this embodiment, each service node in the multi-node cluster receives task information through the load balancing module, generates a task thread based on the task information, and inserts the task thread into the task queue, so that the thread pool can obtain the task thread from the task queue and implement the task information; when the first node receives the task information, the number of task threads in the task queue of the first node exceeds the preset threshold number, and the first node will send the task information to the multi-threaded component to seek help from other service nodes to process the task information.

[0068] In a preferred embodiment, before receiving the task information sent by any service node in the multi-node cluster, the method further includes:

[0069] Obtain at least one task type; wherein the task type represents a type of service task;

[0070] Register each service node separately, so that each service node is associated with the multi-threaded component;

[0071] Constructing a thread creator for each task type; and

[0072] A thread pool and a task queue are created in each service node according to each task type.

[0073] In this example, the task type represents a type of service task that can be implemented by a multi-node service cluster. A thread creator is constructed based on the acquired task type to generate a task thread with task information corresponding to the task type. The thread creator is a creation class for business processing entity threads. By calling the creation class based on the task parameters of the task information, a task thread for implementing the task information and obtaining the task results can be generated.

[0074] At the same time, according to the acquired task type, a thread pool for implementing the service task corresponding to the task type is created for each registered service node in the multi-node cluster, as well as a task queue corresponding to the thread pool for sending task threads to the thread pool, thereby achieving the technical effect that each node can realize task information.

[0075] Finally, by registering each service node in the multi-node cluster, the multi-threaded component can be managed for each service node to ensure the controllability of each service node.

[0076] In this embodiment, the thread pool configuration parameters and thread creation parameters corresponding to the task type are obtained;

[0077] In the multithreaded component, a thread creator is constructed according to thread creation parameters, and the thread creator is used to generate a task thread of task information corresponding to the task type; specifically, according to the interface specification preset in the multithreaded component, the abstract interface of the thread creator is implemented.

[0078] The task registration interface preset in the multithreaded component is called according to the task type. The task registration interface is used to register the task name of the task information corresponding to the task type, the creator name of the thread creator corresponding to the task type, and the thread pool configuration parameters corresponding to the task type in each service node. Specifically, the service node can determine the task type of the task information according to the task name of the task information, call the thread creator according to the creator name corresponding to the task type, and save the task thread generated by the thread creator to the task queue corresponding to the task type, and send the task thread to the thread pool corresponding to the task type through the task queue to run the task thread and obtain the task result.

[0079] According to the thread configuration parameters registered in the service node, create a task thread for running the task information corresponding to the task type and a task queue corresponding to the thread pool; wherein the task queue is used to save the task thread of the task information corresponding to the task type. Specifically, the thread pool configuration parameters include: pool type and pool parameters; pool types include cache thread pool, fixed-length thread pool, and single-thread thread pool; pool parameters refer to the maximum concurrency of the thread pool. Optionally, if no configuration is performed, the default thread pool is used. The default thread pool is a fixed-length thread pool, and the maximum concurrency of the fixed-length thread pool is the number of CPU cores. Threads exceeding the limit will wait in the queue for execution.

[0080] Furthermore, a distributed lock and a message queue are created in the multi-threaded component; wherein the distributed lock is a computer rule used to ensure that among each service node in the multi-node cluster, one and only one service node can process the task information; and the message queue is a communication method for sending task notifications to the service nodes in the multi-node cluster.

[0081] S102: Set one of the service nodes other than the first node in the multi-node cluster as the second node; wherein, one of the service nodes other than the first node is a service node having an idle position in the task queue and sending a response message; the task queue is a linear list for arranging at least one task information; the response message represents that other service nodes with idle positions request to process the task information.

[0082] In this step, one of the service nodes other than the first node in the multi-node cluster is set as the second node, so that the second node is called to process the task information to obtain the task result.

[0083] In a preferred embodiment, setting one of the service nodes other than the first node in the multi-node cluster as the second node includes:

[0084] Generate task parameters according to the task information, and save the task parameters in the database of the multi-node cluster; wherein the task parameters are the calling parameters and task types of the task threads generated in the task information; the calling parameters are used to describe the content of the task information; the task type is used to describe the type of the task information;

[0085] Generate a task notification according to the task information, and send the task notification to other service nodes in the multi-node cluster except the first node;

[0086] If a response message sent by at least one other service node is received, the preset distributed lock is called, and the other service node corresponding to the first response message received is set as the second node.

[0087] In this example, when a node needs to initiate a multi-threaded task, the task parameters in the task information are extracted as parameter information for generating the task thread; the task parameters are saved in the database of the multi-node cluster by calling the task conversion interface in the multi-threaded component.

[0088] A task notification is generated according to the task information, wherein the task notification has a task number that uniquely identifies the task information, and the task notification is sent to a message queue preset in the multi-threaded component through a task push interface preset in the multi-threaded component; wherein the task notification is a notification message that records the task number.

[0089] All service nodes in a multi-node cluster subscribe to the message queue in the multi-threaded component to receive task notifications broadcast by the message queue.

[0090] When the message queue broadcasts the task notification therein, each subscribed service node can receive the task notification.

[0091] The multi-threaded component serializes the task type and at least one calling parameter of the task information to obtain a task string, and saves the task string to the database. The serialization conversion method is specified when the thread task is registered, and the serialization process can be customized; in this embodiment, the JSON format is used by default for serialization conversion of calling parameters and task types.

[0092] If the thread task is configured with public information processing when it is registered, the multi-threaded component will also serialize the public information parameters and store them in the database after the task parameters are saved.

[0093] The publish-subscribe model is a messaging paradigm in which the sender of a message (called a publisher) does not send the message directly to a specific receiver (called a subscriber), but broadcasts it through a message channel so that subscribers who subscribe to the message topic can consume it. The message channels supported by the multithreaded component include: Kafka, RocketMQ, and Redis.

[0094] In this embodiment, the implementation methods of distributed locks include: task locking based on Redis, task locking based on Zookeeper, and task locking based on a database.

[0095] The task is locked in Redis, and the Redlock algorithm is used to implement distributed locks. The steps are as follows:

[0096] Get the system time in milliseconds.

[0097] Try to obtain locks from N Redis instances in sequence using the task ID and random UUID in the task information, and set the lock validity period to 60 seconds.

[0098] After the lock is acquired successfully, the current time is subtracted from the acquired system time. If the time is less than the lock expiration time, it is determined that the lock is acquired successfully from the instance.

[0099] When the lock is successfully acquired from most Redis instances, it is determined that the service node has successfully locked the task, and the service node that has successfully locked the task is set as the second node.

[0100] Based on Zookeeper, we lock the tasks and use the feature of Zookeeper to create temporary nodes with serial numbers to implement distributed locks. The steps are as follows:

[0101] In Zookeeper, create a task directory based on the task ID in the task information.

[0102] Creating temporary sequence nodes corresponding to other service nodes in the task directory according to the response time of the response information sent by other service nodes, wherein the temporary sequence nodes are arranged in ascending order of response time in the task directory;

[0103] The temporary sequence node with the smallest order in the task directory is obtained, and it is determined that the task of the temporary sequence node is locked successfully; and other service nodes corresponding to the temporary sequence node with the successfully locked task are set as the second node.

[0104] Task locking is performed based on the database, and distributed locking is implemented using the unique index feature of the database. The steps are as follows:

[0105] Initialize a Lock table with a unique index in the database.

[0106] When locking, the task ID in the task information is inserted into the database table. There is a unique constraint on the task ID in the task information in the data table. The database will ensure that only one operation can succeed. Therefore, the first response information received by the database can be inserted into the data table.

[0107] The other service nodes corresponding to the response information successfully inserted into the data table are set as the second nodes.

[0108] Further, after sending the task notification to other service nodes in the multi-node cluster except the first node, the method further includes:

[0109] Control each other service node to view the task queue in each other service node;

[0110] If it is confirmed that the task queue of another service node has a free position, feedback information is sent to the other service node; wherein the feedback information is used to instruct the other service node to send response information.

[0111] In this example, after listening to the task message, each service node will call the task receiving interface in the multi-threaded component to obtain the task type corresponding to the task notification from the database, and each other service node obtains the task queue corresponding to the task type. If the number of task threads in the task queue does not reach the number of positions configured for the task queue, it is confirmed that the task queue of the other service node has free positions; if the number of task threads in the task queue reaches the number of positions configured for the task queue, it is confirmed that the task queue of the other service node does not have free positions.

[0112] Preferably, after sending the task notification to other service nodes in the multi-node cluster except the first node, the method further includes:

[0113] The status information of the task information is set to a new status, and the status information is saved in a database and associated with the task parameters corresponding to the task information.

[0114] In this step, the newly created state representation has generated a task notification corresponding to the task information, which has been broadcast to other service nodes. The state information is stored in the database at the location of the task parameter corresponding to the task information, so as to associate it with the task parameter. For example, the state information is used as the attribute value or configuration value of the task parameter, or the state information and the task parameter are stored in the database in the form of a K-value (key-value pair), where the state information is the primary key (key value) of the task parameter and the task parameter is the key value (value value) of the state information.

[0115] After setting the other service node corresponding to the received first response information as the second node, the method further includes:

[0116] The status information of the task information is set to a waiting state, and the status information is saved in a database and associated with the task parameters corresponding to the task information.

[0117] In this step, the waiting state indicates that the multi-threaded component has received at least one response message, and sets the other service node corresponding to the first response message as the second node, and associates the state information with the task parameter by saving it in the database at the position of the task parameter corresponding to the task information. For example: using the state information as the attribute value or configuration value of the task parameter, or saving the state information and the task parameter in the database in the form of a K-value (key-value pair), the state information is the primary key (key value) of the task parameter, and the task parameter is the key value (value value) of the state information.

[0118] S103: Control the second node to process the task information to obtain the task result, and obtain the execution log and thread status generated in the process of processing the task information; wherein, the execution log describes the process of processing the task information; the thread status reflects the running status in the process of processing the task information; the thread status includes: normal execution, abnormal execution and interrupted execution.

[0119] In this step, the second node is controlled to process the task information to obtain the task result, so that the second node helps the first node to share the task information. Therefore, the task information sent by the first node is processed by the second node to balance the task volume of each service node in the multi-node cluster, avoiding the situation where some service nodes need to process too much task information while other service nodes need to process too little task information, thereby improving the overall processing efficiency of the multi-node cluster for task information.

[0120] In a preferred embodiment, controlling the second node to process the task information to obtain the task result, and obtaining the execution log and thread status generated in the process of processing the task information, includes:

[0121] Obtaining task parameters corresponding to the task information from the database, and creating a task thread for implementing the task information according to the task parameters;

[0122] Insert a monitoring aspect into the task thread to obtain a proxy thread; wherein the monitoring aspect is a combination of notifications and entry points used to record the execution log and thread status of the task thread during execution; the execution log is the log information generated when the task thread executes to the position corresponding to the notification and entry point, and the log information corresponding to each notification and entry point describes the process of processing task information; the thread status is the running state of the task thread when the task thread executes to the position corresponding to the notification and entry point, and the running state corresponding to each notification and entry point reflects the running status during the processing of task information;

[0123] Loading the proxy thread into the task queue in the second node;

[0124] The thread pool of the second node is controlled to execute the task thread in the proxy thread in the task queue, obtain the task result corresponding to the task information, and execute the monitoring aspect in the proxy thread to obtain the execution log and thread status of the task information.

[0125] In this example, after receiving the lock success information, the task executor preset in the second node obtains the task parameters, task type and status information from the database through the task ID; after obtaining the task parameters, it will first determine whether the current status information of the task parameters is a newly created state. If the status information is a newly created state, the state of the task is changed to waiting for execution. If the status information is not a newly created state, it indicates that the task information has been processed by a determined second node, and then the task parameters, task type and status information are abandoned, and the task parameters, task type and status information are directly obtained from the second node.

[0126] After updating the status information, the task executor will deserialize the call parameters into entity objects through the task parameter conversion interface of the multi-threaded component. The deserialization method is specified when the thread task is registered. The task executor calls the thread creator of the multi-threaded component to create a task thread of the task information according to the entity object of the call parameter. The monitoring aspect is encapsulated into the task thread to obtain the proxy thread; wherein, the monitoring aspect (Aspect) is a combination of notification and entry point for recording the execution log and thread status of the task thread; wherein, the notification defines what the monitoring aspect is and when to use it, the notification describes the work to be done by the monitoring aspect, and the notification and when to perform this work; the pointcut defines where the monitoring aspect performs the above work in the proxy thread, that is: it defines at which connection points the notification needs to be executed.

[0127] Add the proxy thread to the task thread queue and wait for the thread pool to execute it. The thread pool will execute the proxy thread in the task queue corresponding to it according to its own thread pool type; the thread pool is a form of multi-threaded processing, which adds tasks to the queue during processing and then automatically starts these tasks after the thread is created. Thread pool threads are all background threads. Each thread uses the default stack size, runs at the default priority, and is in a multi-threaded unit.

[0128] When the proxy thread starts to be executed through the thread pool of the second node, it will first access the database through the independent state information changer in the multi-threaded component to modify the state information of the task parameters corresponding to the proxy thread and change the state information to the execution state.

[0129] After the execution of the proxy thread is completed through the thread pool of the second node, the database is accessed through the state information changer to modify the state information of the task parameters corresponding to the proxy thread, change the state information to the completion state, and release the computing resources of the thread pool.

[0130] When all tasks in the task queue of this type of task are processed, the multi-threaded component will re-obtain the task parameters in the newly created state from the database.

[0131] If there are task parameters in the new state, try to lock and execute them.

[0132] If there are no task parameters in the newly created state, continue to wait for the service node in the multi-node cluster to send task information.

[0133] In this embodiment, the status information is adjusted by a task recorder preset in the database; the status information in the database also records the time point when the status information is changed, the task ID of the task information of the task parameters corresponding to the status information, and the node ID of the second node corresponding to the task information; the node ID is the unique identifier of the second node.

[0134] The database also has a thread expansion monitoring interface;

[0135] Receive the monitoring request sent by the control end; obtain the task parameters and / or the task ID of the task information corresponding to the task parameters and / or the status information of the task parameters from the database according to the monitoring request. This is used to achieve the technical effect of providing the control end with the status information query of the task information, so as to facilitate the subsequent statistics of the efficiency of the second node in processing the task information.

[0136] Furthermore, in the process of executing the proxy thread by the thread pool, the monitoring aspect in the proxy thread is also called to make the monitoring aspect perform its preset work and obtain the current execution log and thread status of the proxy thread. Among them, the execution log represents the log information generated when the thread pool executes the monitoring aspect; the thread status represents the status of the thread pool when it executes the monitoring aspect, and the thread status includes: normal execution, abnormal execution, and interrupted execution. The execution log and thread status of each proxy thread are monitored through the monitoring aspect to ensure the controllability of the execution of each proxy thread.

[0137] Preferably, after loading the proxy thread into the task queue in the second node, the method further comprises:

[0138] The status information of the task information is set as the execution status, and the status information is saved in the database and associated with the task parameters corresponding to the task information.

[0139] In this step, the execution state represents that the proxy thread is loaded into the task queue of the second node, is being executed or is waiting to be executed, and is associated with the task parameter by saving the state information in the position of the task parameter corresponding to the task information in the database. For example, the state information is used as the attribute value or configuration value of the task parameter, or the state information and the task parameter are saved in the database in the form of a K-value (key-value pair), where the state information is the primary key (key value) of the task parameter, and the task parameter is the key value (value value) of the state information.

[0140] After obtaining the task result corresponding to the task information, the method further includes:

[0141] The status information of the task information is set to a completed state, and the status information is saved in a database and associated with the task parameters corresponding to the task information.

[0142] In this step, the completion status characterizes that the thread pool has completed the execution of the proxy thread corresponding to the task information, and generates the task result corresponding to the task information, and associates the state information with the task parameter by saving it in the database at the location of the task parameter corresponding to the task information. For example: the state information is used as the attribute value or configuration value of the task parameter, or the state information and the task parameter are saved in the database in the form of a K-value (key-value pair), where the state information is the primary key (key value) of the task parameter, and the task parameter is the key value (value value) of the state information.

[0143] S104: If it is confirmed that the thread state is an execution exception or an execution interruption, the second node is controlled to re-process the task information to obtain the task result, and obtain the execution log corresponding to the thread state and send the execution log to the control end.

[0144] In this instance, if it is found that the thread status of the proxy thread has an execution exception or execution interruption during the execution process, it means that the task thread in the proxy thread has an exception, failure or error during the operation. Therefore, the proxy thread is re-run by calling the thread pool in the second node, so that the second node can reprocess the task information and obtain the task result of the task information.

[0145] At the same time, the execution log corresponding to the thread state of the execution exception or execution interruption is sent to the control end, so that the control end can analyze the problems in the proxy thread according to the execution log, which is conducive to the control end personnel to adjust or repair the second node.

[0146] Embodiment 2:

[0147] See also Figure 3The present application provides a task scheduling processing method, which is applied to a multi-threaded component in a multi-node cluster. The multi-node cluster includes: a multi-threaded component and multiple service nodes; the method includes:

[0148] S201: Receive task information sent by any service node in a multi-node cluster, and set the service node that sends the task information as the first node.

[0149] This step is consistent with S101 in Example 1, so it will not be described here in detail.

[0150] S202: Set one of the service nodes other than the first node in the multi-node cluster as the second node; wherein, one of the service nodes other than the first node is a service node having an idle position in the task queue and sending response information; the task queue is a linear list for arranging at least one task information; the response information represents that other service nodes with idle positions request to process task information.

[0151] This step is consistent with S102 in Example 1, so it will not be described here in detail.

[0152] S203: Respond to the restart signal; set the task parameter with status information of the newly created state in the database as the first parameter according to the restart signal, and call the distributed lock, set the other corresponding service nodes of the first received response information corresponding to the first parameter as the second node; wherein the restart signal is the instruction information for restarting the multi-node cluster.

[0153] S204: Respond to a restart signal; set the task parameter whose status information in the database is in a waiting state to a second parameter according to the restart signal, obtain the second parameter from the database, and create a task thread for implementing the task information according to the second parameter; wherein the restart signal is instruction information for restarting the multi-node cluster.

[0154] S205: Control the second node to process the task information to obtain the task result, and obtain the execution log and thread status generated in the process of processing the task information; wherein, the execution log describes the process of processing the task information; the thread status reflects the running status in the process of processing the task information; the thread status includes: normal execution, abnormal execution and interrupted execution.

[0155] This step is consistent with S103 in Example 1, so it will not be described here in detail.

[0156] S206: Respond to a restart signal; set the task parameter whose status information in the database is the execution status to a third parameter according to the restart signal, and control the proxy thread corresponding to the third parameter in the thread pool execution task queue of the second node to obtain the task result corresponding to the task information; wherein the restart signal is the instruction information for restarting the multi-node cluster.

[0157] In this example, if the multi-node cluster freezes or crashes due to an exception or failure, the processing of each task information in the multi-node cluster will be interrupted. Once restarted, it is easy to cause all task information in the multi-node cluster to be processed step by step from the new state, resulting in a waste of computing power resources of the multi-node cluster.

[0158] In this regard, this instance sets the first parameter, second parameter and third parameter respectively according to the task parameters corresponding to the new state, waiting state and execution state in the database, and calls the distributed lock to set the other corresponding service nodes of the first response information received corresponding to the first parameter as the second node; and / or obtains the second parameter from the database, and creates a task thread for implementing the task information according to the second parameter; and / or controls the thread pool of the second node to execute the proxy thread corresponding to the third parameter in the task queue to obtain the task result corresponding to the task information; realizes the status information corresponding to the task parameters of each task information in the database, and executes each task information, which greatly saves the computing power consumption of the multi-node cluster.

[0159] S207: If it is confirmed that the thread state is an execution exception or an execution interruption, the second node is controlled to re-process the task information to obtain the task result, and obtain the execution log corresponding to the thread state and send the execution log to the control end.

[0160] This step is consistent with S104 in Example 1, so it will not be described here in detail.

[0161] Embodiment 3:

[0162] See also Figure 4 The present application provides a task scheduling processing device 1, which is installed in a multi-threaded component in a multi-node cluster. The multi-node cluster includes: a multi-threaded component and multiple service nodes. The device includes:

[0163] An input module 11 is used to receive task information sent by any service node in the multi-node cluster, and set the service node sending the task information as the first node;

[0164] A processing module 12, configured to set one of the service nodes other than the first node in the multi-node cluster as a second node;

[0165] The control module 15 is used to control the second node to process the task information to obtain the task result, and obtain the execution log and thread status generated in the process of processing the task information; wherein the execution log describes the process of processing the task information; the thread status reflects the running status in the process of processing the task information.

[0166] The retry module 17 is used to control the second node to reprocess the task information to obtain the task result if it is confirmed that the thread state is an execution exception or an execution interruption, and to obtain the execution log corresponding to the thread state and send the execution log to the control end.

[0167] Optionally, the task scheduling processing device 1 further includes:

[0168] The first restart module 13 is used to respond to the restart signal; according to the restart signal, the task parameter with status information of the newly created state in the database is set as the first parameter, and the distributed lock is called to set the other service nodes corresponding to the first received response information corresponding to the first parameter as the second node; wherein the restart signal is the instruction information for restarting the multi-node cluster.

[0169] The second restart module 14 is used to respond to the restart signal; set the task parameter whose status information in the database is in the waiting state to the second parameter according to the restart signal, obtain the second parameter from the database, and create a task thread for implementing the task information according to the second parameter; wherein the restart signal is the instruction information for restarting the multi-node cluster.

[0170] The third restart module 16 is used to respond to the restart signal; set the task parameter with status information in the database as the execution status to the third parameter according to the restart signal, and control the thread pool of the second node to execute the proxy thread corresponding to the third parameter in the task queue to obtain the task result corresponding to the task information; wherein the restart signal is the instruction information for restarting the multi-node cluster.

[0171] Embodiment 4:

[0172] To achieve the above-mentioned purpose, the present application also provides a computer device 4, comprising: a processor 42 and a memory 41 in communication with the processor 42; the memory stores computer execution instructions;

[0173] The processor executes the computer execution instructions stored in the memory 41 to implement the above-mentioned task scheduling processing method, wherein the components of the task scheduling processing device can be dispersed in different computer devices, and the computer device 4 can be a smart phone, tablet computer, laptop computer, desktop computer, rack server, blade server, tower server or cabinet server (including independent servers, or server clusters composed of multiple application servers) that executes the program. The computer device of this embodiment includes at least but is not limited to: a memory 41 and a processor 42 that can be interconnected through a system bus, such as Figure 5 It should be pointed out that Figure 5 Only a computer device with component - is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. In this embodiment, the memory 41 (i.e., readable storage medium) includes flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. In other embodiments, the memory 41 can also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device, such as the program code of the task scheduling processing device of the third embodiment. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are about to be output. The processor 42 can be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device. In this embodiment, the processor 42 is used to run the program code stored in the memory 41 or process data, such as running the task scheduling processing device to implement the task scheduling processing method of the above embodiment.

[0174] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of each embodiment of the present application. It should be understood that the above-mentioned processor can be a central processing unit (Central Processing Unit, referred to as CPU), or other general-purpose processors, digital signal processors (Digital Signal Processor, referred to as DSP), application-specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as a hardware processor to be executed, or the hardware and software modules in the processor are combined and executed. The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and can also be a U disk, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0175] To achieve the above purpose, the present application also provides a computer-readable storage medium, such as flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, App application store, etc., on which computer execution instructions are stored, and the corresponding functions are realized when the program is executed by the processor 42. The computer-readable storage medium of this embodiment is used to store computer execution instructions for implementing the task scheduling processing method, and when executed by the processor 42, the task scheduling processing method of the above embodiment is realized.

[0176] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0177] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic device or a main control device.

[0178] The present application provides a computer program product, including a computer program, which implements the above-mentioned task scheduling processing method when executed by a processor.

[0179] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0180] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0181] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A task scheduling processing method, characterized in that: The method is applied to a multi-threaded component in a multi-node cluster, wherein the multi-node cluster includes: the multi-threaded component and a plurality of service nodes; the method includes: Receiving task information sent by any service node in the multi-node cluster, and setting the service node that sends the task information as the first node; In the multi-node cluster, one of the service nodes other than the first node is set as the second node; wherein, one of the service nodes other than the first node is a service node having an idle position in a task queue and sending a response message; the task queue is a linear table for arranging at least one task information; the response message indicates that the other service node having the idle position requests to process the task information; Control the second node to process the task information to obtain a task result, and obtain an execution log and a thread status generated in the process of processing the task information; wherein the execution log describes the process of processing the task information; the thread status reflects the running status in the process of processing the task information; the thread status includes: normal execution, abnormal execution and interrupted execution; If it is confirmed that the thread state is an execution exception or an execution interruption, control the second node to re-process the task information to obtain a task result, and obtain an execution log corresponding to the thread state and send the execution log to the control end; The step of setting one of the service nodes other than the first node in the multi-node cluster as the second node includes: Generating task parameters according to the task information, and saving the task parameters in a database of the multi-node cluster; Generate a task notification according to the task information, and send the task notification to other service nodes in the multi-node cluster except the first node; If a response message sent by at least one of the other service nodes is received, the preset distributed lock is called, and the other service node corresponding to the first response message received is set as the second node.

2. The task scheduling processing method according to claim 1, characterized in that: After sending the task notification to other service nodes in the multi-node cluster except the first node, the method further includes: Control each of the other service nodes to view the task queue in each of the other service nodes; If it is confirmed that the task queue of one of the other service nodes has a free position, feedback information is sent to one of the other service nodes; wherein the feedback information is used to instruct one of the other service nodes to send the response information.

3. The task scheduling processing method according to claim 1, characterized in that: The controlling the second node to process the task information to obtain a task result, and obtaining an execution log and a thread status generated in the process of processing the task information, includes: Acquire task parameters corresponding to the task information from the database, and create a task thread for implementing the task information according to the task parameters; Inserting a monitoring aspect into the task thread to obtain a proxy thread; wherein the monitoring aspect is a combination of a notification and an entry point, and the monitoring aspect is used to record the execution log and thread status of the task thread during execution; the execution log is the log information generated when the task thread executes to a position corresponding to the notification and the entry point, and each log information corresponding to the notification and the entry point describes the process of processing the task information; the thread status is the running state of the task thread when the task thread executes to a position corresponding to the notification and the entry point, and each running state corresponding to the notification and the entry point reflects the running status during the processing of the task information; Loading the proxy thread into a task queue in the second node; The thread pool of the second node is controlled to execute the task thread in the proxy thread in the task queue, obtain the task result corresponding to the task information, and execute the monitoring aspect in the proxy thread to obtain the execution log and thread status of the task information.

4. The task scheduling processing method according to claim 3, characterized in that: After sending the task notification to other service nodes in the multi-node cluster except the first node, the method further includes: Setting the state information of the task information to a newly created state, and saving the state information to the database, and associating it with the task parameters corresponding to the task information; wherein the newly created state indicates that a task notification corresponding to the task information has been generated and broadcast to other service nodes; After setting the other service node corresponding to the received first response information as the second node, the method further includes: Setting the state information of the task information to a waiting state, and saving the state information to the database, and associating it with the task parameters corresponding to the task information; wherein the waiting state indicates that the multi-threaded component has received at least one response information, and setting the other service node corresponding to the first response information to the second node; After loading the proxy thread into the task queue in the second node, the method further includes: Setting the status information of the task information as an execution status, and saving the status information in the database, and associating it with the task parameters corresponding to the task information; wherein the execution status represents that the proxy thread is loaded into the task queue of the second node, and is being executed or waiting to be executed; After obtaining the task result corresponding to the task information, the method further includes: The status information of the task information is set to a completed status, and the status information is saved in the database and associated with the task parameters corresponding to the task information; wherein the completed status indicates that the thread pool has completed the execution of the proxy thread corresponding to the task information and generated a task result corresponding to the task information.

5. The task scheduling processing method according to claim 4, characterized in that: After setting one of the service nodes other than the first node in the multi-node cluster as the second node, the method further includes: Responding to a restart signal; wherein the restart signal is instruction information for restarting the multi-node cluster; According to the restart signal, the task parameter of the task state in the database is set as the first parameter, and the distributed lock is called to set the other service node corresponding to the first received response information corresponding to the first parameter as the second node; and / or According to the restart signal, the task parameter of the task state in the database is set to a second parameter, and the second parameter is obtained from the database, and a task thread for implementing the task information is created according to the second parameter; and / or According to the restart signal, the task parameter of the task status in the database is set to the third parameter, and the thread pool of the second node is controlled to execute the proxy thread corresponding to the third parameter in the task queue to obtain the task result corresponding to the task information.

6. The task scheduling processing method according to any one of claims 1 to 5, characterized in that: Before receiving the task information sent by any service node in the multi-node cluster, the method further includes: Acquire at least one task type; wherein the task type represents a type of service task; Registering each of the service nodes respectively, so that each of the service nodes is associated with the multi-threaded component; Constructing a thread creator according to each of the task types; A thread pool and a task queue are created in each of the service nodes according to each of the task types.

7. A task scheduling processing device, characterized in that: The device is installed in a multi-threaded component in a multi-node cluster, the multi-node cluster includes: the multi-threaded component and multiple service nodes, the device includes: An input module, used for receiving task information sent by any service node in the multi-node cluster, and setting the service node sending the task information as the first node; A processing module, used for setting one of the service nodes other than the first node in the multi-node cluster as a second node; wherein one of the service nodes other than the first node is a service node having an idle position in a task queue and sending a response message; the task queue is a linear table for arranging at least one task information; the response message indicates that the other service node having the idle position requests to process the task information; The processing module is specifically used to generate task parameters according to the task information, and save the task parameters in the database of the multi-node cluster; generate a task notification according to the task information, and send the task notification to other service nodes in the multi-node cluster except the first node; if a response message sent by at least one of the other service nodes is received, call the preset distributed lock and set the other service node corresponding to the first response message received as the second node; A control module, used to control the second node to process the task information to obtain a task result, and to obtain an execution log and a thread status generated in the process of processing the task information; wherein the execution log describes the process of processing the task information; the thread status reflects the running status in the process of processing the task information; the thread status includes: normal execution, abnormal execution and interrupted execution; The retry module is used to control the second node to reprocess the task information to obtain the task result if it is confirmed that the thread state is an execution exception or an execution interruption, and to obtain the execution log corresponding to the thread state and send the execution log to the control end.

8. A computer device, characterized in that: include: A processor and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the task scheduling processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the task scheduling processing method according to any one of claims 1 to 6.

Images