Timing Task Data Processing Method, Device, Electronic Device and Storage Medium

Abstract

The present application provides a method, apparatus, electronic device, and storage medium for processing scheduled task data. The method for processing scheduled task data includes: sending a request for obtaining shard data of a target scheduled task to a database node, receiving a first current value of the counter returned by the database node, and persisting the first current value of the counter; determining an acquisition range for the shard data based on the first current value of the counter; determining a second current value of the counter based on the acquisition range, and sending the second current value of the counter to the database node; processing the shard data pointed to by the acquisition range, and other steps. The present application can process a large amount of scheduled task data, and at the same time, the present application has advantages such as high availability and high scalability.

Description

Technical Field

[0001] The present application relates to the field of computers, and in particular, to a method, apparatus, electronic device, and storage medium for processing timed task data. Background Art

[0002] At present, the volume of timed task data is increasing. For a large amount of timed task data, if a single machine device is used for processing, the performance of the single machine device needs to be improved by expanding the physical hardware of the single machine device so that the performance of the single machine device can meet the processing requirements for the large amount of timed task data, for example, meet requirements such as execution speed. However, the method of improving the performance of a single machine device by expanding the physical hardware of the single machine device has limited data processing speed due to physical expansion bottlenecks. On the other hand, when data processing anomalies occur in a single machine device, the abnormal data will affect the next data processing, that is, it has the disadvantage of low availability. On the other hand, due to physical expansion bottlenecks, this method has the disadvantage of being difficult to expand. Furthermore, due to being difficult to expand, this method still cannot meet the processing requirements in the case of particularly large data. Summary of the Invention

[0003] The purpose of the embodiments of the present application is to provide a method, apparatus, electronic device, and storage medium for processing timed task data to process a large amount of timed task data. Further, the present application can overcome at least one of the above technical problems.

[0004] In a first aspect, the present invention provides a method for processing timed task data, which is applied to an executor node in a distributed system. The distributed system includes two or more of the executor nodes. The method includes:

[0005] Sending a request for obtaining shard data of a target timed task to a database node, so that after receiving the obtaining request, the server determines whether the executor node has the permission to obtain the shard data based on the obtaining request, and when the executor node has the permission to obtain the shard data, returns a first current value of a counter to the executor node. The database node divides the data of the target timed task into M shard data, where M is a natural number greater than 2, and the first current value of the counter points to the first shard data among the remaining unprocessed data in the M shard data;

[0006] Receiving the first current value of the counter returned by the database node and persisting the first current value of the counter;

[0007] Determining an obtaining range for the shard data based on the first current value of the counter;

[0008] Determine a second current value of the counter based on the obtained range, and send the second current value of the counter to the database node, so that other executor nodes in the distributed system process the remaining unprocessed data among the M shard data starting from the shard data pointed to by the second current value of the counter based on the second current value of the counter returned by the database node;

[0009] Process the shard data pointed to by the obtained range.

[0010] The first aspect of this application can utilize the data of timed tasks executed by two or more executor nodes in a distributed system. Therefore, compared with the method of processing the data of timed tasks by a single-machine device in the prior art, it has a higher data processing speed and thus higher processing performance. On the other hand, by using the data of timed tasks executed by two or more executor nodes in a distributed system, it is possible to dynamically match the execution requirements for the data of timed tasks. Among them, when the execution requirements for the data of timed tasks increase, the number of executor nodes can be increased to match the execution requirements. Moreover, since the increase in the number of executor nodes is not limited by the physical hardware bottleneck, it has better scalability.

[0011] In an optional embodiment, the distributed system further includes a distributed lock node, and the method further includes:

[0012] When sending a request to obtain shard data of a target timed task to the database node, send a lock request to the distributed lock node, so that after receiving the lock request, the distributed lock node prevents the database node from responding to the obtain requests sent by other executor nodes in the distributed system.

[0013] In this optional embodiment, by sending a lock request to the distributed lock node when sending a request to obtain shard data of a target timed task to the database node, it is possible to make the distributed lock node prevent the database node from responding to the obtain requests sent by other executor nodes in the distributed system after receiving the lock request, thereby avoiding two executor nodes obtaining the same counter value from the database node.

[0014] In an optional embodiment, the method further includes:

[0015] When sending the second current value of the counter to the database node, send an unlock request to the distributed lock node, so that after receiving the unlock request, the distributed lock node allows the database node to respond to the obtain requests sent by other executor nodes in the distributed system.

[0016] In this alternative embodiment, by allowing the database node to respond to an acquisition request sent by other executor nodes in the distributed system, the distributed lock node can then complete the unlocking operation.

[0017] In an alternative embodiment, the method further includes:

[0018] When it is detected that an exception occurs in processing the shard data pointed to by the acquisition range, determine the exception shard data;

[0019] Send the exception shard data to the database node, so that the database node inserts the exception shard data at the bottom of the M shard data and serves as the (M + 1)-th shard data.

[0020] In this alternative embodiment, by sending the exception shard data to the database node, the database node can insert the exception shard data at the bottom of the M shard data and serve as the (M + 1)-th shard data. In this way, the executor node can reprocess the exception shard data again, ultimately increasing the probability that all allocated data is correctly processed. Compared with the prior art, since this application inserts the exception shard data at the bottom of the M shard data, the occurrence of the exception shard data will not affect the processing of the next data, so it has higher availability.

[0021] In an alternative embodiment, determining the acquisition range for the shard data based on the first current value of the counter includes:

[0022] Determine the local performance data of the executor node;

[0023] Determine the acquisition range for the shard data based on the local performance data and the first current value of the counter.

[0024] In this alternative embodiment, based on the local performance data and the first current value of the counter, the acquisition range for the shard data can be determined. Furthermore, based on the real-time performance of the executor node, an acquisition range that matches the real-time information of the executor node can be matched, thereby avoiding overloading the executor node or avoiding underutilizing the processing capacity of the executor node.

[0025] In an alternative embodiment, determining the acquisition range for the shard data based on the local performance data and the first current value of the counter includes:

[0026] Determine the acquisition quantity of the shard data based on the local performance data;

[0027] Add the first current value of the counter to the number of acquired shard data to obtain the upper limit of the acquisition range, where the lower limit of the acquisition range is the first current value of the counter.

[0028] In an alternative embodiment, the local performance data includes the CPU usage rate of the actuator node.

[0029] In this alternative embodiment, the acquisition range can be determined based on the CPU usage rate of the actuator node.

[0030] In a second aspect, the present invention provides a timing task data processing device, which is applied to an actuator node in a distributed system, where the distributed system includes two or more of the actuator nodes, and the device includes:

[0031] A first sending module, configured to send a request for acquiring shard data of a target timing task to a database node, so that after receiving the acquisition request, the server determines whether the actuator node has the permission to acquire the shard data based on the acquisition request, and when the actuator node has the permission to acquire the shard data, returns the first current value of a counter to the actuator node, where the database node divides the data of the target timing task into M shard data, M is a natural number greater than 2, and the first current value of the counter points to the first shard data of the remaining unprocessed data among the M shard data;

[0032] A receiving module, configured to receive the first current value of the counter returned by the database node and persist the first current value of the counter;

[0033] A first determining module, configured to determine an acquisition range for the shard data based on the first current value of the counter;

[0034] A second determining module, configured to determine a second current value of the counter based on the acquisition range;

[0035] A second sending module, configured to send the second current value of the counter to the database node, so that other actuator nodes in the distributed system start processing the remaining unprocessed data among the M shard data from the shard data pointed to by the second current value of the counter based on the second current value of the counter returned by the database node;

[0036] A data execution module, configured to process the shard data pointed to by the acquisition range.

[0037] The device according to the second aspect of the present application can utilize data of timing tasks executed by two or more actuator nodes in a distributed system. Therefore, compared with the way of processing data of timing tasks by a single-device in the prior art, it has a higher data processing speed and thus higher processing performance. On the other hand, by utilizing data of timing tasks executed by two or more actuator nodes in a distributed system, the execution requirements for data of timing tasks can be dynamically matched. Among them, when the execution requirements for data of timing tasks increase, the number of actuator nodes can be increased to match the execution requirements, and since the increase in the number of actuator nodes is not limited by the physical hardware bottleneck, it has better scalability.

[0038] In a third aspect, the present invention provides an electronic device, comprising:

[0039] a processor; and

[0040] a memory configured to store machine-readable instructions, which, when executed by the processor, execute the timing task data processing method according to any one of the foregoing embodiments.

[0041] The electronic device according to the third aspect of the present application can utilize data of timing tasks executed by two or more actuator nodes in a distributed system by executing the timing task data processing method. Therefore, compared with the way of processing data of timing tasks by a single-device in the prior art, it has a higher data processing speed and thus higher processing performance. On the other hand, by utilizing data of timing tasks executed by two or more actuator nodes in a distributed system, the execution requirements for data of timing tasks can be dynamically matched. Among them, when the execution requirements for data of timing tasks increase, the number of actuator nodes can be increased to match the execution requirements, and since the increase in the number of actuator nodes is not limited by the physical hardware bottleneck, it has better scalability.

[0042] In a fourth aspect, the present invention provides a storage medium storing a computer program, and the computer program is executed by a processor to perform the timing task data processing method according to any one of the foregoing embodiments.

[0043] By executing the timed task data processing method, the storage medium according to the fourth aspect of the present application can utilize two or more executor nodes in the distributed system to execute the data of the timed task. Therefore, compared with the method of processing the data of the timed task by a single device in the prior art, it has a higher data processing speed and thus higher processing performance. On the other hand, by using two or more executor nodes in the distributed system to execute the data of the timed task, the execution requirements for the data of the timed task can be dynamically matched. Among them, when the execution requirements for the data of the timed task increase, the number of executor nodes can be increased to match the execution requirements. And since the increase in the number of executor nodes is not limited by the physical hardware bottleneck, it has better scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments of the present application. It should be understood that the following drawings only show some embodiments of the present application, and thus should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can also be obtained based on these drawings without creative efforts.

[0045] Figure 1 is a schematic flowchart of a method for processing timed task data disclosed in an embodiment of the present application;

[0046] Figure 2 is a schematic structural diagram of a device for processing timed task data disclosed in an embodiment of the present application;

[0047] Figure 3 is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] The following will describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application.

[0049] Embodiment 1

[0050] Please refer to Figure 1 , Figure 1 which is a schematic flowchart of a method for processing timed task data disclosed in an embodiment of the present application. Among them, the method of the embodiment of the present application is applied to an executor node in a distributed system, and the distributed system includes two or more executor nodes. As Figure 1 shown, the method of the embodiment of the present application includes the following steps:

[0051] 101. Send a request to the database node to obtain sharded data for a target scheduled task, so that after the server receives the acquisition request, it determines whether the executor node has the permission to obtain the sharded data based on the acquisition request, and when the executor node has the permission to obtain the sharded data, returns the first current value of the counter to the executor node. Among them, the database node divides the data of the target scheduled task into M sharded data, where M is a natural number greater than 2, and the first current value of the counter points to the first sharded data of the remaining unprocessed data among the M sharded data;

[0052] 102. Receive the first current value of the counter returned by the database node and persist the first current value of the counter;

[0053] 103. Determine the acquisition range for the sharded data based on the first current value of the counter;

[0054] 104. Determine the second current value of the counter based on the acquisition range and send the second current value of the counter to the database node, so that other executor nodes in the distributed system start processing the remaining unprocessed data among the M sharded data from the sharded data pointed to by the second current value of the counter returned by the database node;

[0055] 105. Process the sharded data pointed to by the acquisition range.

[0056] In the embodiment of the present application, the database node is communicatively connected to each executor node, and thus each executor node can perform data interaction with the database node.

[0057] In the embodiment of the present application, the database node refers to a node that stores data of scheduled tasks. Among them, this node is installed with database tools to store data of scheduled tasks through the database tools. Further, the database node divides the data of the scheduled task into M sharded data based on the data volume of the data of the scheduled task and other configuration parameters, where M is a natural number greater than or equal to 2, that is, the database node divides the data of the scheduled task into at least 2 sharded data. Further, the subscripts of the M sharded data are represented by the sequence (1, 2, 3... M). For example, the subscript of the first sharded data is 1, and the subscript of the Mth sharded data is M.

[0058] In an embodiment of the present application, for step 101, the acquisition request sent by the actuator node to the database node may include the identification information of the actuator node. This identification information may be the certificate of the actuator node, the MAC address, etc., which can indicate the legitimacy of the actuator node. Then, the database node verifies the actuator node based on this identification information. After the actuator node passes the verification, it is determined that the actuator node has the permission to obtain sharded data. Further, by verifying the actuator node, it is possible to prevent illegal devices from accessing the database node.

[0059] In an embodiment of the present application, for step 101, the acquisition request may further include the status information of the actuator node. Then, it is possible to determine whether the execution node has the permission to obtain the allocated data based on the status information of the actuator node. For example, when the status information of the actuator node indicates that an execution exception occurs during the processing of the previous sharded data, it can be determined that the actuator node does not have the permission to obtain sharded data until the actuator node completes troubleshooting and modifies the status information to indicate that the fault has been eliminated.

[0060] In an embodiment of the present application, for step 101, the first current value of the counter is used to point to the first data of the unprocessed sharded data. For example, when the first current value of the counter is 3, it points to the third sharded data among the M sharded data, where the first sharded data and the second sharded data have both been processed previously.

[0061] In an embodiment of the present application, for step 101, the counter starts counting from 1. When all M sharded data have not been processed, the first current value of the counter points to the first sharded data. It should be noted that in an embodiment of the present application, both the first current value and the second current value of the counter refer to the current value of the counter. The use of the first and second descriptions here is only for the convenience of describing the values of the counter at different times.

[0062] In an embodiment of the present application, for step 102, a specific way to persist the first current value of the counter is to transfer the first current value of the counter from the memory space to the hard disk storage space.

[0063] In an embodiment of the present application, for step 103, by way of example, when the first current value of the counter obtained by the actuator node is 1 and the actuator node needs to process 3 sharded data, the acquisition range is expressed as 1 - 4, that is, the actuator node processes the first sharded data, the second sharded data, and the third sharded data within the acquisition range.

[0064] In an embodiment of the present application, for step 104, exemplarily, assuming that the acquisition range of an actuator node is 1 - 4, it sends the upper limit value 4 to the database node, and then another actuator node processes the data from the 4th shard data.

[0065] In an embodiment of the present application, for step 105, when the actuator node processes the shard data pointed to by the acquisition range, it can obtain the shard data from the database node based on the acquisition range, or all the shard data can be pre - stored in the database node and the actuator node in advance, and then the actuator node obtains the local data based on the acquisition range.

[0066] Based on the above, it can be seen that the embodiments of the present application can use two or more actuator nodes in a distributed system to execute the data of the timing task. Therefore, compared with the prior art method of using a single - machine device to process the data of the timing task, it has a higher data - processing speed and thus higher processing performance. On the other hand, by using two or more actuator nodes in a distributed system to execute the data of the timing task, it is possible to dynamically match the execution requirements for the data of the timing task. Among them, when the execution requirements for the data of the timing task increase, the number of actuator nodes can be increased to match the execution requirements, and since the increase in the number of actuator nodes is not limited by the physical hardware bottleneck, it has better scalability.

[0067] In an embodiment of the present application, as an alternative embodiment, the distributed system of the present application further includes a distributed lock node. Correspondingly, the method of the present application further includes the following steps:

[0068] When sending a request to obtain shard data for a target timing task to the database node, send a lock - request to the distributed lock node, so that after receiving the lock - request, the distributed lock node prevents the database node from responding to the acquisition requests sent by other actuator nodes in the distributed system.

[0069] In this alternative embodiment, by sending a lock - request to the distributed lock node when sending a request to obtain shard data for a target timing task to the database node, it can make the distributed lock node prevent the database node from responding to the acquisition requests sent by other actuator nodes in the distributed system after receiving the lock - request, thereby avoiding two actuator nodes obtaining the same counter value from the database node.

[0070] In an embodiment of the present application, as an alternative embodiment, the method of the present application further includes the following steps:

[0071] When sending the second current value of the counter to the database node, an unlock request is sent to the distributed lock node, so that after the distributed lock node receives the unlock request, it allows the database node to respond to the acquisition requests sent by other executor nodes in the distributed system.

[0072] In this alternative embodiment, by allowing the database node to respond to the acquisition requests sent by other executor nodes in the distributed system, the distributed lock node can then complete the unlocking operation.

[0073] In the embodiment of the present application, as an alternative embodiment, the method of the embodiment of the present application further includes the following steps:

[0074] When it is detected that an exception occurs in processing the shard data pointed to by the acquisition range, determine the abnormal shard data;

[0075] Send the abnormal shard data to the database node, so that the database node inserts the abnormal shard data at the bottom of the M shard data and serves as the (M + 1)-th shard data.

[0076] In this alternative embodiment, by sending the abnormal shard data to the database node, the database node can insert the abnormal shard data at the bottom of the M shard data and serve as the (M + 1)-th shard data. In this way, the executor node can reprocess the abnormal shard data again, ultimately increasing the probability that all allocated data is correctly processed. Compared with the prior art, since the present application inserts the abnormal shard data at the bottom of the M shard data, the occurrence of the abnormal shard data will not affect the processing of the next data, so it has higher availability.

[0077] In the embodiment of the present application, as an alternative embodiment, determining the acquisition range for the shard data based on the first current value of the counter includes the following sub-steps:

[0078] Determine the local performance data of the executor node;

[0079] Determine the acquisition range for the shard data based on the local performance data and the first current value of the counter.

[0080] In this alternative embodiment, based on the local performance data and the first current value of the counter, the acquisition range for the shard data can be determined. Furthermore, based on the real-time performance of the executor node, an acquisition range that matches the real-time information of the executor node can be matched, thus avoiding overloading the executor node or underutilizing the processing capacity of the executor node.

[0081] In the embodiment of the present application, as an alternative embodiment, determining the acquisition range for the shard data based on the local performance data and the first current value of the counter includes the following sub-steps:

[0082] Determine the acquisition quantity of sharded data based on the local performance data;

[0083] Add the first current value of the counter to the acquisition quantity of the sharded data to obtain the upper limit of the acquisition range, where the lower limit of the acquisition range is the first current value of the counter.

[0084] In the embodiment of the present application, as an optional implementation manner, the local performance data includes the CPU usage rate of the executor node. Furthermore, based on the CPU usage rate of the executor node, the acquisition range can be determined. Further optionally, the local performance data may also include the memory usage rate of the executor node, network performance data, etc.

[0085] Embodiment Two

[0086] Please refer to Figure 2 , Figure 2 which is a schematic structural diagram of a timing task data processing device disclosed in the embodiment of the present application. Among them, the timing task data processing device is applied to the executor node in a distributed system, and the distributed system includes two or more executor nodes. As Figure 2 shown, the device in the embodiment of the present application includes the following functional modules:

[0087] The first sending module 201 is used to send a request for acquiring sharded data of a target timing task to the database node, so that after receiving the acquisition request, the server determines whether the executor node has the permission to acquire the sharded data based on the acquisition request, and when the executor node has the permission to acquire the sharded data, returns the first current value of the counter to the executor node. Among them, the database node divides the data of the target timing task into M sharded data, M is a natural number greater than 2, and the first current value of the counter points to the first sharded data of the remaining unprocessed data among the M sharded data;

[0088] The receiving module 202 is used to receive the first current value of the counter returned by the database node and persist the first current value of the counter;

[0089] The first determination module 203 is used to determine the acquisition range for the sharded data based on the first current value of the counter;

[0090] The second determination module 204 is used to determine the second current value of the counter based on the acquisition range;

[0091] The second sending module 205 is used to send the second current value of the counter to the database node, so that other executor nodes in the distributed system start processing the remaining unprocessed data among the M sharded data from the sharded data pointed to by the second current value of the counter based on the second current value of the counter returned by the database node;

[0092] The data execution module 206 is used to process the shard data pointed to by the acquisition range.

[0093] The device according to the embodiment of the present application can use two or more actuator nodes in the distributed system to execute the data of the timing task. Therefore, compared with the method of processing the data of the timing task by a single device in the prior art, it has a higher data processing speed and thus higher processing performance. On the other hand, by using two or more actuator nodes in the distributed system to execute the data of the timing task, the execution requirements for the data of the timing task can be dynamically matched. Among them, when the execution requirements for the data of the timing task increase, the number of actuator nodes can be increased to match the execution requirements, and since the increase in the number of actuator nodes is not limited by the physical hardware bottleneck, it has better scalability.

[0094] It should be noted that for other detailed descriptions of the device according to the embodiment of the present application, please refer to the relevant descriptions in Embodiment 1 of the present application, and the embodiment of the present application will not elaborate on this.

[0095] Embodiment 3

[0096] Please refer to Figure 3 , Figure 3 which is a schematic structural diagram of an electronic device disclosed in the embodiment of the present application. As Figure 3 shown, the electronic device according to the embodiment of the present application includes:

[0097] A processor 301; and

[0098] A memory 302 configured to store machine-readable instructions, which, when executed by the processor 301, execute the timing task data processing method according to any one of the foregoing embodiments.

[0099] The electronic device according to the embodiment of the present application can use two or more actuator nodes in the distributed system to execute the data of the timing task by executing the timing task data processing method. Therefore, compared with the method of processing the data of the timing task by a single device in the prior art, it has a higher data processing speed and thus higher processing performance. On the other hand, by using two or more actuator nodes in the distributed system to execute the data of the timing task, the execution requirements for the data of the timing task can be dynamically matched. Among them, when the execution requirements for the data of the timing task increase, the number of actuator nodes can be increased to match the execution requirements, and since the increase in the number of actuator nodes is not limited by the physical hardware bottleneck, it has better scalability.

[0100] Embodiment 4

[0101] An embodiment of the present application provides a storage medium storing a computer program, and the computer program is executed by a processor to perform the timing task data processing method according to any one of the foregoing embodiments.

[0102] By executing the timing task data processing method, the storage medium according to the embodiment of the present application can utilize two or more executor nodes in a distributed system to execute the data of the timing task. Therefore, compared with the prior art method of processing the data of the timing task by a single device, it has a higher data processing speed and thus higher processing performance. On the other hand, by using two or more executor nodes in a distributed system to execute the data of the timing task, the execution requirements for the data of the timing task can be dynamically matched. Among them, when the execution requirements for the data of the timing task increase, the number of executor nodes can be increased to match the execution requirements. And since the increase in the number of executor nodes is not limited by the physical hardware bottleneck, it has better scalability.

[0103] In the embodiments provided by the present application, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some communication interfaces, and the indirect couplings or communication connections of devices or units can be in electrical, mechanical or other forms.

[0104] In addition, the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] Furthermore, in each embodiment of the present application, the various functional modules can be integrated together to form an independent part, or each module can exist alone, or two or more modules can be integrated to form an independent part.

[0106] It should be noted that if a function is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0107] In this text, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0108] The above are only the embodiments of this application and are not used to limit the protection scope of this application. For those skilled in the art, this application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. A method for processing scheduled task data, characterized in that, The method is applied to an executor node in a distributed system, where the distributed system includes two or more of the executor nodes, and the method includes: Sending a request to obtain shard data for a target timed task to a database node, so that after the server corresponding to the database node receives the obtain request, it determines whether the executor node has the permission to obtain the shard data based on the obtain request, and when the executor node has the permission to obtain the shard data, returns a first current value of a counter to the executor node, where the database node divides the data of the target timed task into M shard data, M is a natural number greater than 2, and the first current value of the counter points to the first shard data of the remaining unprocessed data among the M shard data; Receiving the first current value of the counter returned by the database node and persisting the first current value of the counter; Determining an obtain range for the shard data based on the first current value of the counter; Determining a second current value of the counter based on the obtain range and sending the second current value of the counter to the database node, so that other executor nodes in the distributed system start processing the remaining unprocessed data among the M shard data from the shard data pointed to by the second current value of the counter returned by the database node; Processing the shard data pointed to by the obtain range.

2. The method according to claim 1, wherein The distributed system further includes a distributed lock node, and the method further includes: When sending a request to obtain shard data for a target timed task to the database node, sending a lock request to the distributed lock node, so that after the distributed lock node receives the lock request, it prevents the database node from responding to obtain requests sent by other executor nodes in the distributed system.

3. The method according to claim 2, characterized in that, The method further includes: When sending the second current value of the counter to the database node, sending an unlock request to the distributed lock node, so that after the distributed lock node receives the unlock request, it allows the database node to respond to obtain requests sent by other executor nodes in the distributed system.

4. The method according to claim 1, wherein The method further includes: When it is detected that an exception occurs in processing the shard data pointed to by the obtain range, determining the exception shard data; Sending the exception shard data to the database node, so that the database node inserts the exception shard data at the bottom of the M shard data and serves as the (M + 1)-th shard data.

5. The method according to claim 1, wherein The determining an obtain range for the shard data based on the first current value of the counter includes: Determining the local performance data of the executor node; Determining the obtain range for the shard data based on the local performance data and the first current value of the counter.

6. The method according to claim 5, characterized in that The determining the obtain range for the shard data based on the local performance data and the first current value of the counter includes: Determining the number of shard data to be obtained based on the local performance data; Add the first current value of the counter to the number of shard data obtained, to obtain the upper limit of the acquisition range, where the lower limit of the acquisition range is the first current value of the counter.

7. The method according to claim 5, characterized in that, The local performance data includes the CPU usage rate of the actuator node.

8. A timed task data processing device, characterized in that The device is applied to an actuator node in a distributed system, where the distributed system includes two or more of the actuator nodes, and the device includes: A first sending module, configured to send a request for obtaining shard data for a target timing task to a database node, so that after the server corresponding to the database node receives the obtaining request, it determines whether the actuator node has the permission to obtain the shard data based on the obtaining request, and when the actuator node has the permission to obtain the shard data, returns the first current value of the counter to the actuator node, where the database node divides the data of the target timing task into M shard data, M is a natural number greater than 2, and the first current value of the counter points to the first shard data of the remaining unprocessed data among the M shard data; A receiving module, configured to receive the first current value of the counter returned by the database node and persist the first current value of the counter; A first determining module, configured to determine an acquisition range for the shard data based on the first current value of the counter; A second determining module, configured to determine a second current value of the counter based on the acquisition range; A second sending module, configured to send the second current value of the counter to the database node, so that other actuator nodes in the distributed system start processing the remaining unprocessed data among the M shard data from the shard data pointed to by the second current value of the counter based on the second current value of the counter returned by the database node; A data execution module, configured to process the shard data pointed to by the acquisition range.

9. An electronic device, characterized in that, Comprising: A processor; And A memory, configured to store machine-readable instructions, which when executed by the processor, execute the timing task data processing method according to any one of claims 1-7.

10. A storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by a processor to perform the timing task data processing method according to any one of claims 1-7.

Images