Task processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a task processing method and device, electronic equipment and a computer readable storage medium, and can be applied to the technical field of computer technology and data processing. The task processing method comprises the steps that in response to a received task processing request, current machine configuration information is determined, the task processing request comprises to-be-processed task identifiers corresponding to M to-be-processed tasks respectively, and the current machine configuration information comprises execution instance identifiers corresponding to Q execution instances respectively; according to the current machine configuration information, first-level splitting processing is carried out on the M to-be-processed tasks, N first task sets are obtained, and each of the N first task sets comprises a first number of to-be-processed tasks; performing second-level splitting processing on the first number of to-be-processed tasks to obtain P second task sets; and respectively sending the P second task sets to a target execution instance in the Q execution instances, so that the target execution instance processes the second task sets.

Description

Technical Field

[0001] The present disclosure relates to the fields of computer technology and data processing technology, and more specifically, to a task processing method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of computer technology, massive data has emerged. The Push method based on message queues and the Pull method based on timers can be used to monitor the changes of some data in massive data source systems.

[0003] The Push method based on the message queue can refer to a method of implementing data monitoring by subscribing to the topic of the message queue provided by the source data system through the business system and listening to the data change messages corresponding to the topic. The Pull method based on the timer can refer to a method of implementing data monitoring by actively querying data from the source data system using the timer.

[0004] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following problems in the related art: the real-time performance of data change monitoring and the stability of system operation cannot be effectively guaranteed. Summary of the invention

[0005] In view of this, the present disclosure provides a task processing method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0006] According to one aspect of the present disclosure, a task processing method is provided, comprising: in response to receiving a task processing request, determining current machine configuration information, wherein the task processing request includes a to-be-processed task identifier corresponding to each of M to-be-processed tasks, and the current machine configuration information includes an execution instance identifier corresponding to each of Q execution instances; performing a first-level splitting process on the M to-be-processed tasks according to the current machine configuration information to obtain N first task sets, wherein each of the N first task sets includes a first number of the to-be-processed tasks; performing a second-level splitting process on the first number of to-be-processed tasks to obtain P second task sets; and respectively sending the P second task sets to target execution instances among the Q execution instances, so that the target execution instances process the second task sets; wherein M, N, P, and Q are all positive integers, and P is greater than N.

[0007] According to an embodiment of the present disclosure, it also includes, before performing second-level splitting processing on the above-mentioned first number of tasks to be processed to obtain P second task sets: in response to the first scheduled task being triggered, determining the task interval time; and in response to monitoring changes in the amount of data corresponding to the above-mentioned tasks to be processed, determining the target splitting ratio and the target execution time corresponding to the above-mentioned tasks to be processed identifiers.

[0008] According to an embodiment of the present disclosure, the current machine configuration information further includes a processing capability evaluation value.

[0009] According to an embodiment of the present disclosure, in response to the first scheduled task being triggered, determining the task interval time includes: in response to the first scheduled task being triggered, determining the current number of tasks to be processed; determining a first product value based on the processing capability evaluation value and the number of execution instances; determining a first ratio based on the current number of tasks to be processed and the first product value; and determining the task interval time based on the first ratio.

[0010] According to an embodiment of the present disclosure, in response to monitoring a change in the amount of data corresponding to the above-mentioned task to be processed, determining the target split ratio and the target execution time corresponding to the above-mentioned task to be processed identifier includes: in response to monitoring a change in the amount of data corresponding to the above-mentioned task to be processed, determining a first current system moment and a data volume trend evaluation value; determining the target split ratio based on the above-mentioned data volume trend evaluation value; and determining the above-mentioned target execution time based on the above-mentioned first current system moment and the above-mentioned task interval time.

[0011] According to an embodiment of the present disclosure, the above-mentioned second-level splitting processing of the above-mentioned first number of tasks to be processed to obtain P second task sets includes: determining S target interval moments according to the above-mentioned task interval time, where S is a positive integer; and, based on the above-mentioned S target interval moments and the above-mentioned target splitting ratio, performing second-level splitting processing on the above-mentioned first number of tasks to be processed to obtain the above-mentioned P second task sets.

[0012] According to an embodiment of the present disclosure, determining S target interval moments based on the above-mentioned task interval time includes: when s=1, determining the first target interval moment based on the above-mentioned task interval time and the above-mentioned first current system moment; when 1<s≤S, determining the sth interval time cumulative value based on the above-mentioned task interval time; and determining the sth target interval moment based on the above-mentioned first target interval moment and the above-mentioned sth interval time cumulative value.

[0013] According to an embodiment of the present disclosure, the above-mentioned first number of tasks to be processed are subjected to second-level splitting processing according to the above-mentioned S target interval moments and the above-mentioned target splitting ratio to obtain the above-mentioned P second task sets, including: grouping the above-mentioned S target interval moments according to the above-mentioned target splitting ratio to obtain T groups of target interval moments, wherein T is a positive integer; and, according to the above-mentioned T groups of target interval moments and the above-mentioned target execution time corresponding to each of the above-mentioned tasks to be processed, the above-mentioned first number of tasks to be processed are subjected to second-level splitting processing to obtain a second number of tasks to be processed corresponding to each of the above-mentioned T groups of target interval moments.

[0014] According to an embodiment of the present disclosure, the above-mentioned sending the above-mentioned P second task sets to the target execution instances in the above-mentioned Q execution instances respectively includes: in response to the second scheduled task being triggered, determining the second current system moment; according to the above-mentioned second current system moment, determining the above-mentioned target execution instance in an idle state in the above-mentioned Q execution instances; according to the above-mentioned second current system moment and the above-mentioned T group of target interval moments, determining the target second task set in the above-mentioned P second task sets; and, sending the above-mentioned target second task set to the above-mentioned target execution instance.

[0015] According to an embodiment of the present disclosure, the current machine configuration information includes the number of running execution instances.

[0016] According to an embodiment of the present disclosure, the above-mentioned M tasks to be processed are split at the first level according to the above-mentioned current machine configuration information to obtain N first task sets, including: according to the above-mentioned number of execution instances, the above-mentioned M tasks to be processed are split at the first level to obtain the first number of the above-mentioned tasks to be processed corresponding to each of the above-mentioned Q execution instance identifiers.

[0017] According to another aspect of the present disclosure, a task processing device is provided, including: a first determination module, for determining current machine configuration information in response to receiving a task processing request, wherein the task processing request includes a to-be-processed task identifier corresponding to each of the M to-be-processed tasks, and the current machine configuration information includes an execution instance identifier corresponding to each of the Q execution instances; a first-level splitting processing module, for performing a first-level splitting processing on the M to-be-processed tasks according to the current machine configuration information to obtain N first task sets, wherein the N first task sets each include a first number of the to-be-processed tasks; a second-level splitting processing module, for performing a second-level splitting processing on the first number of to-be-processed tasks to obtain P second task sets; and a sending module, for sending the P second task sets to target execution instances in the Q execution instances, respectively, so that the target execution instances process the second task sets; wherein M, N, P, and Q are all positive integers, and P is greater than N.

[0018] According to another aspect of the present disclosure, an electronic device is provided, comprising: one or more processors; and a memory for storing one or more instructions, wherein when the one or more instructions are executed by the one or more processors, the one or more processors implement the method described in the present disclosure.

[0019] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which executable instructions are stored. When the executable instructions are executed by a processor, the processor implements the method described in the present disclosure.

[0020] According to another aspect of the present disclosure, a computer program product is provided. The computer program product includes computer executable instructions. When the computer executable instructions are executed, they are used to implement the method described in the present disclosure.

[0021] According to the embodiments of the present disclosure, by performing the first-level splitting processing on M pending tasks according to the current machine configuration information, N first task sets are obtained. Since the first-level task splitting can be automatically realized according to the received task processing request, the technical problem that the real-time monitoring of data changes cannot be effectively guaranteed in the related technology is avoided, and the real-time performance of task processing is improved. By performing the second-level splitting processing on the first number of pending tasks, P second task sets are obtained. Since the multi-level splitting processing is performed on the pending tasks, the task processing efficiency and the throughput of the system can be better improved, and the technical problem that the stability of the system operation cannot be effectively guaranteed in the related technology is avoided, and the machine resources can be used more effectively. On this basis, by sending the P second task sets to the target execution instances in the Q execution instances respectively, due to the use of a distributed processing method, the computing power of multiple execution instances can be fully utilized, and the efficiency of task processing and the stability of system operation are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0023] Figure 1 The system architecture to which the task processing method according to the embodiment of the present disclosure can be applied is schematically shown;

[0024] Figure 2 A flowchart schematically shows a task processing method according to an embodiment of the present disclosure;

[0025] Figure 3A An example schematic diagram of a process for determining a task interval time according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 3B An example schematic diagram of a process for determining task interval time according to another embodiment of the present disclosure is schematically shown;

[0027] Figure 4A An example schematic diagram schematically shows a process of determining a target split ratio and a target execution time corresponding to a task identifier to be processed according to an embodiment of the present disclosure;

[0028] Figure 4B An example schematic diagram schematically shows a process of determining a target execution time corresponding to a task identifier to be processed according to an embodiment of the present disclosure;

[0029] Figure 5A An example schematic diagram of a process of determining S target interval moments according to a task interval time according to an embodiment of the present disclosure is schematically shown;

[0030] Figure 5BAn example schematic diagram of a process of performing second-level splitting processing on a first number of tasks to be processed according to S target intervals and target splitting ratios to obtain P second task sets according to an embodiment of the present disclosure is schematically shown;

[0031] Figure 6 An example schematic diagram schematically shows a task processing process according to an embodiment of the present disclosure;

[0032] Figure 7 A block diagram schematically shows a task processing device according to an embodiment of the present disclosure; and

[0033] Figure 8 A block diagram of an electronic device suitable for implementing a task processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0034] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0035] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0036] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0037] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0038] In the embodiments of the present disclosure, the collection, updating, analysis, processing, use, transmission, provision, disclosure, storage, etc. of the data involved (for example, including but not limited to user personal information) are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. In particular, necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security, and national security.

[0039] In the embodiments of the present disclosure, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0040] For example, after collecting data corresponding to the task to be processed, your information can be desensitized by using methods including de-identification or anonymization to protect the security of your information.

[0041] The Push method based on message queues can refer to the active push mode, that is, the server actively sends data to the client, which has more immediate responsiveness. The Push method based on message queues has obvious advantages when the source data system is small in size, with high real-time performance and low pressure on the system. However, if the source data is massive, a large number of irrelevant data change messages will enter the business system, and the business system will find it difficult to adapt to this pressure. Even if the database table is queried in real time when the message comes in and then discarded, the business side database may still crash.

[0042] The timer-based Pull method can refer to the polling mode, that is, the client pulls data from the server at a fixed time, and the server needs to provide a query interface. The timer-based Pull method is more applicable in scenarios with massive source data. In the case where a large amount of monitoring data has not changed and has accumulated, the amount of data run by the timer at a single time may be very large, resulting in the need to set the timer frequency to a very low level in order to run all the currently monitored data and avoid multiple scheduled tasks from happening concurrently, which puts more pressure on the system and causes complex concurrency problems. However, due to the limitation of resources such as machine memory, it is difficult to process such a large amount of data at a single time; in addition, too low a monitoring frequency is contrary to the demands of the business. The low execution frequency compromised to meet the limit of data volume leads to slow response changes, which will have a greater impact on user experience, and in scenarios where the data volume is not large most of the time, it causes a waste of system resources. Although the above problems can be improved by adding machine instances, frequent calls to the upstream interface for query at the same time point will bring greater pressure to the upstream system, resulting in interface timeouts, and the problem of system resource waste cannot be solved either.

[0043] In summary, neither the Push method based on message queues nor the Pull method based on timers can effectively guarantee the real-time nature of data change monitoring and the stability of system operation.

[0044] In order to at least partially solve the technical problems existing in the related art, the present disclosure provides a task processing method and device, an electronic device and a computer-readable storage medium, which can be applied to the fields of computer technology and data processing technology. The task processing method includes: in response to receiving a task processing request, determining the current machine configuration information, wherein the task processing request includes the to-be-processed task identifiers corresponding to each of the M to-be-processed tasks, and the current machine configuration information includes the execution instance identifiers corresponding to each of the Q execution instances; according to the current machine configuration information, performing a first-level splitting process on the M to-be-processed tasks to obtain N first task sets, wherein the N first task sets each include a first number of to-be-processed tasks; performing a second-level splitting process on the first number of to-be-processed tasks to obtain P second task sets; and sending the P second task sets to the target execution instances in the Q execution instances respectively, so that the target execution instances process the second task sets.

[0045] Figure 1 The system architecture to which the task processing method according to the embodiment of the present disclosure can be applied is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0046] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0047] The user may use at least one of the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0048] The first terminal device 101, the second terminal device 102, and the third terminal device 103 may be various electronic devices having display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and the like.

[0049] The server 105 may be a server that provides various services, such as a background management server (only as an example) that provides support for websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0050] It should be noted that the task processing method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the task processing device provided in the embodiment of the present disclosure can generally be set in the server 105. The task processing method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105. Accordingly, the task processing device provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the first terminal device 101, the second terminal device 102, the third terminal device 103 and / or the server 105.

[0051] Alternatively, the task processing method provided in the embodiment of the present disclosure may also be executed by the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may also be executed by other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103. Accordingly, the task processing apparatus provided in the embodiment of the present disclosure may also be provided in the first terminal device 101, the second terminal device 102, or the third terminal device 103, or may be provided in other terminal devices different from the first terminal device 101, the second terminal device 102, or the third terminal device 103.

[0052] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0053] It should be noted that the sequence numbers of the operations in the following method are only used as representations of the operations for the purpose of description, and should not be regarded as representing the execution order of the operations. Unless explicitly stated, the method does not need to be executed completely in the order shown.

[0054] Figure 2 The flowchart of the task processing method according to the embodiment of the present disclosure is schematically shown.

[0055] like Figure 2 As shown, the task processing method 200 includes operations S210 to S240.

[0056] In operation S210, in response to receiving a task processing request, current machine configuration information is determined, wherein the task processing request includes a to-be-processed task identifier corresponding to each of the M to-be-processed tasks, and the current machine configuration information includes an execution instance identifier corresponding to each of the Q execution instances.

[0057] In operation S220, the M tasks to be processed are split at a first level according to the current machine configuration information to obtain N first task sets, wherein each of the N first task sets includes a first number of tasks to be processed.

[0058] In operation S230, a second-level splitting process is performed on the first number of tasks to be processed to obtain P second task sets.

[0059] In operation S240 , the P second task sets are respectively sent to target execution instances among the Q execution instances, so that the target execution instances process the second task sets.

[0060] According to an embodiment of the present disclosure, M, N, P, and Q are all positive integers, and P is greater than N.

[0061] According to an embodiment of the present disclosure, the code of the task processing request can be pre-written into the script, and in response to detecting a task processing operation initiated by a user, at least one pending task identifier corresponding to the task processing operation can be obtained. The pending task identifier can refer to a unique identifier used to identify different pending tasks. The client can run the script and generate a task processing request message based on at least one task processing identifier. On this basis, the client can send the task processing request message to the server, so that the server can process the pending tasks corresponding to each of the at least one task processing identifier according to the task processing request message. The pending task may refer to a data monitoring task, such as a waybill status monitoring task in the order reminder.

[0062] According to an embodiment of the present disclosure, after receiving a task processing request, the current machine configuration information can be determined. The current machine configuration information may include at least one of the following: a processing capability evaluation value, the number of execution instances in operation, and an execution instance identifier corresponding to each of the Q execution instances. The processing capability evaluation value may be used to measure the task processing efficiency of different execution instances. For example, the processing capability evaluation value may use the processing volume per minute as an evaluation indicator. An execution instance may refer to a machine instance used to process a task. An execution instance identifier may refer to a unique identifier used to identify different execution instances.

[0063] According to an embodiment of the present disclosure, after obtaining the current machine configuration information, the M tasks to be processed can be split at the first level according to the number of running execution instances included in the current machine configuration information to obtain N first task sets. The specific method of the first level splitting process can be configured according to actual business needs and is not limited here. For example, the first level splitting process can include average splitting and random splitting.

[0064] According to an embodiment of the present disclosure, taking M as 6 and the number of running execution instances as 2 as an example, for example, when the first-level splitting process is an average splitting process, the 6 pending tasks can be split into 2 first task sets, and the 1st first task set and the 2nd first task set can each include 3 pending tasks. Alternatively, when the first-level splitting process is a random splitting process, the 6 pending tasks can be further split into 2 first task sets in combination with the processing capacity evaluation value, the 3rd first task set can include 2 pending tasks, and the 4th first task set can include 4 pending tasks.

[0065] According to an embodiment of the present disclosure, after obtaining N first task sets, the first number of tasks to be processed included in each first task set can be respectively subjected to second-level splitting processing to obtain P second task sets. The specific method of the second-level splitting processing can be configured according to actual business needs and is not limited here. For example, the second-level splitting processing may include random splitting and time-slice-based splitting.

[0066] According to an embodiment of the present disclosure, for each first task set, taking the first number of 6 as an example, for example, when the second-level splitting process is random splitting, the 6 tasks to be processed can be split into 2 second task sets, the first second task set can include 2 tasks to be processed, and the second second task set can include 4 tasks to be processed. Alternatively, when the second-level splitting process is splitting based on time slices, the 6 tasks to be processed can be split according to their respective task execution times based on the current system moment to obtain 2 second task sets, and the third second task set and the fourth second task set can each include 3 tasks to be processed.

[0067] According to an embodiment of the present disclosure, after obtaining P second task sets, for each second task set in the P second task sets, a target execution instance corresponding to the second task set can be determined among Q execution instances, and the second task set can be sent to the target execution instance so that the target execution instance processes at least one to-be-processed task included in the second task set to obtain a task processing result.

[0068] According to the embodiments of the present disclosure, by performing the first-level splitting processing on M pending tasks according to the current machine configuration information, N first task sets are obtained. Since the first-level task splitting can be automatically realized according to the received task processing request, the technical problem that the real-time monitoring of data changes cannot be effectively guaranteed in the related technology is avoided, and the real-time performance of task processing is improved. By performing the second-level splitting processing on the first number of pending tasks, P second task sets are obtained. Since the multi-level splitting processing is performed on the pending tasks, the task processing efficiency and the throughput of the system can be better improved, and the technical problem that the stability of the system operation cannot be effectively guaranteed in the related technology is avoided, and the machine resources can be used more effectively. On this basis, by sending the P second task sets to the target execution instances in the Q execution instances respectively, due to the use of a distributed processing method, the computing power of multiple execution instances can be fully utilized, and the efficiency of task processing and the stability of system operation are improved.

[0069] Reference below Figure 3A , Figure 3B , Figure 4A , Figure 4B , Figure 5A , Figure 5B and Figure 6 , the task processing method 200 according to an embodiment of the present invention is further explained.

[0070] According to an embodiment of the present disclosure, the current machine configuration information includes the number of running execution instances.

[0071] According to an embodiment of the present disclosure, operation S220 may include the following operations.

[0072] According to the number of execution instances, the M tasks to be processed are split at the first level to obtain a first number of tasks to be processed corresponding to each of the Q execution instance identifiers.

[0073] According to an embodiment of the present disclosure, at least one machine instance currently running in the system can be determined, at least one machine instance in an idle state can be determined in the at least one machine instance, and the number of running execution instances can be determined based on the at least one machine instance in an idle state.

[0074] According to an embodiment of the present disclosure, after obtaining the number of execution instances, the M pending tasks can be split at the first level according to the number of execution instances to obtain a first number of pending tasks corresponding to at least one execution instance identifier. For example, the running execution instances may include execution instance 1 and execution instance 2, that is, the number of execution instances is 2. On this basis, the M pending tasks can be split into M / 2 and M / 2, with execution instance 1 responsible for processing the first M / 2 pending tasks and execution instance 2 responsible for processing the last M / 2 pending tasks.

[0075] According to the embodiments of the present disclosure, since the first task set is obtained by performing a first-level splitting process on M tasks to be processed according to the number of execution instances, machine resources can be better utilized. By allocating the tasks to be processed to different execution instances, the number of tasks processed by each execution instance is more balanced. Since multiple execution instances can process M tasks to be processed at the same time, the task processing efficiency is improved.

[0076] According to an embodiment of the present disclosure, before operation S230, the following operations may be further included.

[0077] In response to the first scheduled task being triggered, the task interval time is determined. In response to monitoring a change in the amount of data corresponding to the task to be processed, a target split ratio and a target execution time corresponding to the identifier of the task to be processed are determined.

[0078] According to an embodiment of the present disclosure, the first scheduled task may include triggering the execution of a first action at a first predetermined time. The first scheduled task may be associated with a pending task. The first action may include determining the number of current pending tasks, in which case the first scheduled task may be used to trigger counting the number of current pending tasks at a first predetermined time. When the first scheduled task is triggered, the task interval time may be further determined based on the number of current pending tasks. The task interval time may be used to characterize a reference time period for splitting pending tasks.

[0079] According to an embodiment of the present disclosure, the first scheduled time may include a first scheduled time point and a first scheduled time period. The first scheduled time may be set according to actual business needs and is not limited here. For example, in the case where the first scheduled time is the first scheduled time point, the first scheduled time may be set to 12:00. In this case, in response to the current system time reaching 12:00, the first scheduled task is triggered. Alternatively, in the case where the first scheduled time is the first scheduled time period, the first scheduled time may be set to 12 hours. In this case, in response to the current system time interval reaching 12 hours, the first scheduled task is triggered.

[0080] According to an embodiment of the present disclosure, the first scheduled task may be set based on a periodic timer or a one-time timer. The periodic timer may be used to trigger the scheduled task, that is, trigger the scheduled task at a fixed interval or in conjunction with a Cron expression.

[0081] According to an embodiment of the present disclosure, for each of the M pending tasks, the amount of data corresponding to the pending task can be monitored in real time. When it is monitored that the amount of data corresponding to the pending task changes, the target split ratio of the pending task and the target execution time corresponding to each pending task identifier can be determined. The target split ratio can be used to characterize the number of splits of the M pending tasks. The target execution time can be used to characterize the expected task execution time corresponding to each pending task identifier.

[0082] According to an embodiment of the present disclosure, the current machine configuration information further includes a processing capability evaluation value.

[0083] According to an embodiment of the present disclosure, in response to the first scheduled task being triggered, determining the task interval time may include the following operations.

[0084] In response to the first scheduled task being triggered, the number of tasks to be processed is determined. A first product value is determined according to the processing capacity evaluation value and the number of execution instances. A first ratio is determined according to the number of tasks to be processed and the first product value. A task interval is determined according to the first ratio.

[0085] According to an embodiment of the present disclosure, the configuration parameters may include a task execution frequency and a task interval time. The task execution frequency may be a fixed value, for example, the task execution frequency may be set to 5 minutes. The task interval time may change dynamically according to the amount of data corresponding to the task to be processed. For example, when the amount of data corresponding to the task to be processed is small, the task interval time may be set to the task execution frequency, which is equivalent to not performing the second-level splitting processing on the first task set, thereby maintaining a higher real-time performance. Alternatively, when the amount of data corresponding to the task to be processed is large, the task interval time may be set to several times the task execution frequency, which is equivalent to splitting the first task set according to time slices, thereby maintaining a higher throughput and stability.

[0086] According to an embodiment of the present disclosure, when the first scheduled task is triggered, an operation of determining the current number of tasks to be processed can be performed. After obtaining the sum of the current number of tasks to be processed, the current number of tasks to be processed and the number of execution instances can be directly multiplied to obtain a first product value. Alternatively, the first first product value can be determined according to the first preset weight and the processing capacity evaluation value, and the second first product value can be determined according to the second preset weight and the number of execution instances. On this basis, the first first product value and the second first product value are accumulated to obtain the first product value.

[0087] According to an embodiment of the present disclosure, after obtaining the first product value, the current number of tasks to be processed may be divided by the first product value to obtain a first ratio. Alternatively, the number of currently processed tasks to be processed may be determined based on a third preset weight and the current number of tasks to be processed, and on this basis, the first ratio may be obtained by dividing the number of currently processed tasks to be processed by the first product value.

[0088] According to an embodiment of the present disclosure, a method for determining the task interval time may be as shown in the following formula (1).

[0089]

[0090] Among them, t represents the task interval time, n represents the number of tasks currently to be processed, W represents the processing capacity evaluation value, and m represents the number of execution instances.

[0091] According to the embodiment of the present disclosure, taking the processing capacity evaluation value as 100 per minute, the task execution frequency as 5 minutes, and the current number of tasks to be processed as 3000 as an example, the task interval time can be 3000 / 100, that is, 30 minutes. Alternatively, the task interval time can also be manually modified to achieve more precise control.

[0092] According to the embodiments of the present disclosure, since the current number of tasks to be processed is determined in response to the triggering of the first scheduled task, the first product value is determined based on the processing capacity evaluation value and the number of execution instances, thereby being able to comprehensively consider the current number of tasks to be processed, the processing capacity evaluation value and the number of execution instances. On this basis, since the first ratio is determined based on the current number of tasks to be processed and the first product value, by determining the task interval time based on the first ratio, it is possible to achieve dynamic adjustment of the task interval time, thereby being able to more fully utilize system resources and ensure the stable operation of the task processing system.

[0093] Figure 3A An example schematic diagram of a process for determining task interval time according to an embodiment of the present disclosure is schematically shown.

[0094] like Figure 3AAs shown, in 300A, in response to the first scheduled task 301 being triggered, the current number of tasks to be processed 302 can be determined. The current machine configuration information can include a processing capacity evaluation value 303_1 and an execution instance number 303_2. According to the processing capacity evaluation value 303_1 and the execution instance number 303_2, a first product value 304 is determined. According to the current number of tasks to be processed and the first product value 304, a first ratio 305 is determined. On this basis, according to the first ratio 305, a task interval time 306 can be determined.

[0095] Figure 3B An example schematic diagram of a process for determining task interval time according to another embodiment of the present disclosure is schematically shown.

[0096] like Figure 3B As shown, in 300B, a single scheduled task 311 can be opened to monitor the current number of pending tasks 307 (i.e., the total number of tasks being executed), and determine the optimal task interval 308 in combination with the processing capacity evaluation value. On this basis, the configuration personnel can update the configuration value 309 (i.e., the configuration center configuration value) according to the task interval 308. When the scheduled task 311 detects that the configuration value 309 has changed, the actual task interval 310 can be determined, that is, the updated configuration value is used when determining the next execution time of the task.

[0097] According to an embodiment of the present disclosure, in response to monitoring a change in the amount of data corresponding to the task to be processed, determining a target split ratio and a target execution time corresponding to the identifier of the task to be processed may include the following operations.

[0098] In response to monitoring a change in the amount of data corresponding to the task to be processed, determining a first current system time and a data amount trend evaluation value, determining a target split ratio based on the data amount trend evaluation value, and determining a target execution time based on the first current system time and the task interval time.

[0099] According to an embodiment of the present disclosure, when a change in the amount of data related to the task to be processed is monitored, a first current system moment and a data amount trend evaluation value may be determined. The first current system moment may be used to characterize the moment when the amount of data changes. The data amount trend evaluation value may be determined based on the change in the amount of data. The data amount trend evaluation value may be used to evaluate the change trend of the amount of data.

[0100] For example, a first data volume corresponding to a historical system moment and a second data volume corresponding to a first current system moment may be obtained. Based on the first data volume and the second data volume, a difference between the first data volume and the second data volume is determined. On this basis, the difference may be directly determined as a data volume trend evaluation value. Alternatively, the data volume trend evaluation value may be determined based on a fourth preset weight and the difference. Alternatively, the data volume may be segmented in advance, and the data volume trend evaluation value may be determined based on the difference and the comparison of different data volume segments.

[0101] According to an embodiment of the present disclosure, by monitoring the change in the amount of data corresponding to the task to be processed, the rule engine is used to identify and trigger the modification of the time slice division rule, that is, the parameter of the target split ratio. For example, in the case where the data volume trend evaluation value represents an increase in the data volume, the target split ratio can be increased, thereby increasing the task throughput, thereby ensuring smooth processing of the task. Alternatively, in the case where the data volume trend evaluation value represents a decrease in the data volume, the target split ratio can be reduced, thereby improving the real-time processing, thereby ensuring a rapid response to data changes. In this way, it can be ensured that machine resources are fully utilized so that the tasks to be processed run smoothly.

[0102] According to an embodiment of the present disclosure, when executing the pending task, the value of the next execution time field can be maintained, that is, the target execution time is determined according to the first current system time and the task interval time. By adding the target execution time field in the task monitoring table, the pending tasks can be divided into different time slices. In addition, when the task timer loads the data to be executed, the pending tasks can also be screened according to the target execution time.

[0103] According to the embodiments of the present disclosure, since the target split ratio is determined based on the data volume trend evaluation value, and the data volume trend evaluation value is determined in response to detecting a change in the data volume corresponding to the task to be processed, the split ratio of the task to be processed can be dynamically adjusted to achieve effective use of system resources. Since the target execution time is determined based on the first current system moment and the task interval time, and the first current system moment is determined in response to monitoring a change in the data volume corresponding to the task to be processed, it can better adapt to different workloads and task changes, thereby improving the adaptability and flexibility of the system.

[0104] Figure 4A An example schematic diagram of a process of determining a target split ratio and a target execution time corresponding to a task identifier to be processed according to an embodiment of the present disclosure is schematically shown.

[0105] like Figure 4AAs shown, in 400A, in response to monitoring a change in the amount of data 401 corresponding to the task to be processed, a first current system moment 402 and a data amount trend evaluation value 403 can be determined. On this basis, a target split ratio 406 can be determined according to the data amount trend evaluation value 403. According to the first current system moment 402 and the task interval time 404, a target execution time 405 is determined.

[0106] Figure 4B An example schematic diagram of a process of determining a target execution time corresponding to a task identifier to be processed according to an embodiment of the present disclosure is schematically shown.

[0107] like Figure 4B As shown, in 400B, during the task processing, a task to be processed 407 may be loaded. After the loading of the task to be processed 407 is completed, operation S410 may be performed.

[0108] In operation S410, has the amount of data corresponding to the pending task changed? If so, the target execution time 408 can be updated and the task processing flow can be terminated. If not, the task status corresponding to the pending task 407 can be modified to the end status 409.

[0109] According to an embodiment of the present disclosure, operation S230 may include the following operations.

[0110] According to the task interval time, S target interval moments are determined, where S is a positive integer. According to the S target interval moments and the target splitting ratio, the first number of tasks to be processed are split at a second level to obtain P second task sets.

[0111] According to an embodiment of the present disclosure, after obtaining the task interval time, the first target interval time can be determined according to the first current system time and the task interval time. The sth task interval time accumulation value can be determined in sequence. On this basis, the sth target interval time can be determined in sequence by adding the first target interval time and the sth task interval time accumulation value.

[0112] According to an embodiment of the present disclosure, after obtaining S target intervals, at least one to-be-processed task corresponding to each of the S target intervals can be determined. On this basis, at least one to-be-processed task corresponding to any target interval can be split according to the target splitting ratio to obtain P second task sets.

[0113] According to an embodiment of the present disclosure, determining S target interval moments according to the task interval time may include the following operations.

[0114] In the case of s=1, the first target interval time is determined according to the task interval time and the first current system time. In the case of 1<s≤S, the sth interval time accumulation value is determined according to the task interval time. The sth target interval time is determined according to the first target interval time and the sth interval time accumulation value.

[0115] According to an embodiment of the present disclosure, when s=1, the first target interval time can be determined according to the first current system time and the task interval time. On this basis, when 1<s≤S, the sth task interval time cumulative value can be determined in sequence. For example, the first task interval time cumulative value can be the task interval time, the second task interval time cumulative value can be 2*task interval time, and so on, the sth task interval time cumulative value can be s*task interval time.

[0116] According to an embodiment of the present disclosure, after obtaining the sth interval time accumulation value, the second target interval time can be determined according to the first target interval time and the first task interval time accumulation value. The third target interval time can be determined according to the second target interval time and the second task interval time accumulation value. Similarly, the sth target interval time can be determined according to the s-1th target interval time and the s-1th task interval time accumulation value.

[0117] According to the embodiment of the present disclosure, since the first target interval moment is determined according to the task interval time and the first current system moment when s=1, the first target interval moment can be used as the basis for determining S target interval moments. On this basis, since the sth target interval moment is determined according to the first target interval moment and the sth interval time cumulative value, the sth interval time cumulative value is determined according to the task interval time, the time interval between tasks can be adjusted more flexibly, thereby improving the processing efficiency of tasks.

[0118] Figure 5A An example schematic diagram of a process of determining S target interval moments according to task interval time according to an embodiment of the present disclosure is schematically shown.

[0119] like Figure 5A As shown, in 500A, when s=1, the first target interval time 503 can be determined according to the task interval time 502 and the first current system time 501 .

[0120] In the case of 1<s≤S, the sth interval time accumulation value 504 can be determined according to the task interval time 502. The sth target interval time 505 can be determined according to the first target interval time 503 and the sth interval time accumulation value 504.

[0121] According to an embodiment of the present disclosure, performing second-level splitting processing on a first number of tasks to be processed according to S target interval times and target splitting ratios to obtain P second task sets may include the following operations.

[0122] According to the target splitting ratio, the S target intervals are grouped to obtain T groups of target intervals, where T is a positive integer. According to the T groups of target intervals and the target execution time corresponding to each task to be processed, the first number of tasks to be processed are split at the second level to obtain the second number of tasks to be processed corresponding to the T groups of target intervals.

[0123] According to an embodiment of the present disclosure, after obtaining the target split ratio, the S target interval moments can be grouped according to the target split ratio to obtain T groups of target interval moments. For example, taking S as 6 and the target split ratio as 2 as an example, the S target interval moments can be divided into 2 groups to obtain the first group of target interval moments and the second group of target interval moments, and each group of target interval moments can correspond to 3 target interval moments.

[0124] According to an embodiment of the present disclosure, after obtaining T groups of target interval moments, for each group of target interval moments in the T groups of target moments and for each target interval moment in each group of target interval moments, a first number of tasks to be processed can be split into a second level according to the target execution time corresponding to each task to be processed, so as to obtain a second number of tasks to be processed corresponding to any target interval moment in the T groups of target interval moments.

[0125] According to an embodiment of the present disclosure, the second-level split processing can be implemented based on the divided time slices through the target split ratio. For example, the running execution instance may include execution instance 3 and execution instance 4, that is, the number of execution instances is 2. M pending tasks can be split into M / 2 and M / 2, and execution instance 3 is responsible for processing the first M / 2 pending tasks, and execution instance 4 is responsible for processing the last M / 2 pending tasks.

[0126] According to the embodiments of the present disclosure, on this basis, the target split ratio can be 2, that is, for the first M / 2 pending tasks that execution instance 3 is responsible for, the second-level splitting process can be performed to obtain the first M / 4 pending tasks and the second M / 4 pending tasks. For the last M / 2 pending tasks that execution instance 4 is responsible for, the second-level splitting process can be performed to obtain the third M / 4 pending tasks and the fourth M / 4 pending tasks. In this way, the number of tasks for each execution instance can be reduced to M / 4, thereby reducing the data level and making more effective use of limited machine resources.

[0127] According to the embodiments of the present disclosure, since the T groups of target intervals are obtained by grouping S target intervals according to the target splitting ratio, the subsequent accurate splitting of the tasks to be processed can be achieved. On this basis, since the second number of tasks to be processed corresponding to each of the T groups of target intervals are obtained by performing a second-level splitting process on the first number of tasks to be processed according to the T groups of target intervals and the target execution time corresponding to each task to be processed, the parallel processing of multiple tasks to be processed can be achieved, thereby improving the utilization of system resources and the efficiency of task processing.

[0128] Figure 5B An example schematic diagram of a process of performing second-level splitting processing on a first number of tasks to be processed according to S target interval times and target splitting ratios to obtain P second task sets according to an embodiment of the present disclosure is schematically shown.

[0129] like Figure 5B As shown, in 500B, S target interval moments 507 can be grouped according to the target split ratio 506 to obtain T groups of target interval moments 508. On this basis, the first number of tasks to be processed can be split at a second level according to the T groups of target interval moments 508 and the target execution time 509 corresponding to each task to be processed, to obtain the second number of tasks to be processed 510 corresponding to the T groups of target interval moments.

[0130] According to an embodiment of the present disclosure, operation S240 may include the following operations.

[0131] In response to the second scheduled task being triggered, a second current system time is determined. According to the second current system time, a target execution instance in an idle state is determined among the Q execution instances. According to the second current system time and the T group of target interval times, a target second task set is determined among the P second task sets. The target second task set is sent to the target execution instance.

[0132] According to an embodiment of the present disclosure, the second scheduled task may include triggering the execution of a second action at a second predetermined time. The second scheduled task may be associated with task processing. The second action may include processing a pending task, in which case the second scheduled task may be used to trigger the execution of the processing of the pending task at the second predetermined time. When the second scheduled task is triggered, the second current system moment may be determined, and a target execution instance that is in an idle state at the second current system moment may be determined. Based on the second current system moment and T groups of target interval moments, a target second task set is determined from P second task sets. On this basis, the target second task set may be sent to the target execution instance so that the target execution instance processes the second number of pending tasks in the second pending task set.

[0133] According to an embodiment of the present disclosure, the second scheduled time may include a second scheduled time point and a second scheduled time period. The second scheduled time may be set according to actual business needs and is not limited here. For example, in the case where the second scheduled time is the second scheduled time point, the second scheduled time may be set to 12:00. In this case, in response to the current system time reaching 12:00, the second scheduled task is triggered. Alternatively, in the case where the second scheduled time is the second scheduled time period, the second scheduled time may be set to 12 hours. In this case, in response to the current system time interval reaching 12 hours, the second scheduled task is triggered.

[0134] According to an embodiment of the present disclosure, since the second current system moment is determined in response to the second scheduled task being triggered, by determining the target execution instance in an idle state among Q execution instances based on the second current system moment and T groups of target interval moments, determining the target second task set among P second task sets, and sending the target second task set to the target execution instance, it is possible to dynamically allocate execution instances for tasks to be processed, thereby improving the system's resource utilization and task processing efficiency.

[0135] Figure 6 An example schematic diagram of a task processing process according to an embodiment of the present disclosure is schematically shown.

[0136] like Figure 6 As shown, in 600 , after the first-level splitting process is performed on the M pending tasks, pending tasks 601 to 610 to be processed which are processed by the execution instance 611 can be obtained.

[0137] The first target split ratio may be, for example, 1 / 2. The S target interval moments may include T1, T2, T3, and T4, and according to the first target split ratio, T1, T2, T3, and T4 may be grouped to obtain the first group of target interval moments (T1 and T2) and the second group of target interval moments (T3 and T4).

[0138] On this basis, the first number of pending tasks can be split into a second level according to the first group of target interval moments, the second group of target interval moments, and the target execution time corresponding to each pending task, to obtain the second number of pending tasks corresponding to each of the T groups of target interval moments. Specifically, for the first group of target interval moments, pending tasks 601 to 604 corresponding to T1 and pending tasks 605 to 608 corresponding to T2 can be obtained. For the second group of target interval moments, pending tasks 601 to 604 and 609 corresponding to T3 can be obtained, and pending tasks 605 to 608 and 610 corresponding to T4 can be obtained.

[0139] When the data volume growth is detected, the target split ratio can be automatically increased, that is, the first target split ratio is adjusted to the second target split ratio. The second target split ratio can be, for example, 1 / 3. The S target interval times can include T5, T6, and T7. According to the second target split ratio, T5, T6, and T7 can be grouped and processed to obtain a third group of target interval times (T5, T6, and T7).

[0140] On this basis, the first number of pending tasks can be split into a second level according to the third group of target intervals and the target execution time corresponding to each pending task, to obtain a second number of pending tasks corresponding to each of the T groups of target intervals. Specifically, for the third group of target intervals, pending tasks 601 to 603 corresponding to T5, pending tasks 605 to 607 and 610 corresponding to T6, and pending tasks 604, 609, and 608 corresponding to T7 can be obtained.

[0141] The above are only exemplary embodiments, but are not limited thereto, and may also include other task processing methods known in the art, as long as they can improve the efficiency of task processing and the stability of system operation.

[0142] Figure 7 The block diagram of a task processing device according to an embodiment of the present disclosure is schematically shown.

[0143] like Figure 7 As shown, the task processing device 700 may include a first determination module 710 , a first-level splitting processing module 720 , a second-level splitting processing module 730 and a sending module 740 .

[0144] The first determination module 710 is used to determine the current machine configuration information in response to receiving a task processing request, wherein the task processing request includes a pending task identifier corresponding to each of the M pending tasks, and the current machine configuration information includes an execution instance identifier corresponding to each of the Q execution instances.

[0145] The first-level splitting processing module 720 is used to perform first-level splitting processing on the M tasks to be processed according to the current machine configuration information to obtain N first task sets, wherein the N first task sets each include a first number of tasks to be processed.

[0146] The second-level splitting processing module 730 is used to perform second-level splitting processing on the first number of tasks to be processed to obtain P second task sets.

[0147] The sending module 740 is used to send the P second task sets to target execution instances among the Q execution instances respectively, so that the target execution instances process the second task sets.

[0148] According to an embodiment of the present disclosure, M, N, P, and Q are all positive integers, and P is greater than N.

[0149] According to an embodiment of the present disclosure, the task processing device 700 may further include a second determining module and a third determining module.

[0150] The second determining module is used to determine the task interval time in response to the first scheduled task being triggered.

[0151] The third determination module is used to determine the target split ratio and the target execution time corresponding to the task identifier to be processed in response to monitoring the change in the data volume corresponding to the task to be processed.

[0152] According to an embodiment of the present disclosure, the current machine configuration information further includes a processing capability evaluation value.

[0153] According to an embodiment of the present disclosure, the second determining module may include a first determining unit, a second determining unit, a third determining unit, and a fourth determining unit.

[0154] The first determining unit is used to determine the number of tasks currently to be processed in response to the first scheduled task being triggered.

[0155] The second determining unit is used to determine a first product value according to the processing capability evaluation value and the number of execution instances.

[0156] The third determining unit is used to determine the first ratio according to the current number of tasks to be processed and the first product value.

[0157] The fourth determining unit is used to determine the task interval time according to the first ratio.

[0158] According to an embodiment of the present disclosure, the third determining module may include a fifth determining unit, a sixth determining unit, and a seventh determining unit.

[0159] The fifth determining unit is used to determine a first current system time and a data volume trend evaluation value in response to monitoring a change in the data volume corresponding to the task to be processed.

[0160] The sixth determination unit is used to determine the target split ratio according to the data volume trend evaluation value.

[0161] The seventh determining unit is used to determine the target execution time according to the first current system time and the task interval time.

[0162] According to an embodiment of the present disclosure, the second-level splitting processing module 730 may include an eighth determining unit and a second-level splitting processing unit.

[0163] The eighth determination unit is used to determine S target interval moments according to the task interval time, where S is a positive integer.

[0164] The second-level splitting processing unit performs second-level splitting processing on the first number of tasks to be processed according to the S target interval times and the target splitting ratio to obtain P second task sets.

[0165] According to an embodiment of the present disclosure, the eighth determining unit may include a first determining subunit, a second determining subunit, and a third determining subunit.

[0166] The first determination subunit is used to determine the first target interval time according to the task interval time and the first current system time when s=1.

[0167] The second determining subunit is used to determine the sth interval time accumulation value according to the task interval time when 1<s≤S.

[0168] The third determination subunit is used to determine the sth target interval time according to the first target interval time and the accumulated value of the sth interval time.

[0169] According to an embodiment of the present disclosure, the second-level splitting processing unit may include a grouping processing subunit and a second-level splitting processing subunit.

[0170] The group processing subunit is used to group the S target intervals according to the target splitting ratio to obtain T groups of target intervals, where T is a positive integer.

[0171] The second-level splitting processing subunit is used to perform second-level splitting processing on the first number of tasks to be processed according to T groups of target interval moments and the target execution time corresponding to each task to be processed, so as to obtain a second number of tasks to be processed corresponding to each of the T groups of target interval moments.

[0172] According to an embodiment of the present disclosure, the sending module 740 may include a ninth determining unit, a tenth determining unit, an eleventh determining unit, and a generating unit.

[0173] The ninth determining unit is configured to determine a second current system time in response to the second scheduled task being triggered.

[0174] The tenth determining unit is configured to determine, according to the second current system time, a target execution instance in an idle state among the Q execution instances.

[0175] An eleventh determining unit is used to determine a target second task set from P second task sets according to a second current system time and T groups of target interval times.

[0176] The generating unit is used to send the target second task set to the target execution instance.

[0177] According to an embodiment of the present disclosure, the current machine configuration information includes the number of running execution instances.

[0178] According to an embodiment of the present disclosure, the first-level splitting processing module 720 may include a first-level splitting processing unit.

[0179] The first-level splitting processing unit performs first-level splitting processing on the M tasks to be processed according to the number of execution instances, and obtains a first number of tasks to be processed corresponding to each of the Q execution instance identifiers.

[0180] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0181] For example, any of the first determination module 710, the first level splitting processing module 720, the second level splitting processing module 730 and the sending module 740 can be combined in one module / unit / sub-unit for implementation, or any one of the modules / units / sub-units can be split into multiple modules / units / sub-units. Alternatively, at least part of the functions of one or more modules / units / sub-units in these modules / units / sub-units can be combined with at least part of the functions of other modules / units / sub-units and implemented in one module / unit / sub-unit. According to an embodiment of the present disclosure, at least one of the first determination module 710, the first level splitting processing module 720, the second level splitting processing module 730 and the sending module 740 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware and firmware or by a suitable combination of any of them. Alternatively, at least one of the first determination module 710, the first-level splitting processing module 720, the second-level splitting processing module 730 and the sending module 740 may be at least partially implemented as a computer program module, which may perform corresponding functions when the computer program module is executed.

[0182] It should be noted that the task processing device part in the embodiment of the present disclosure corresponds to the task processing method part in the embodiment of the present disclosure. The description of the task processing device part specifically refers to the task processing method part, which will not be repeated here.

[0183] Figure 8 A block diagram of an electronic device suitable for implementing a task processing method according to an embodiment of the present disclosure is schematically shown. Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0184] like Figure 8 As shown, the computer electronic device 800 according to the embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 809 to the random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0185] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0186] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The electronic device 800 may further include one or more of the following components connected to the input / output (I / O) interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 808 including a hard disk, etc.; and a communication portion 809 including a network interface card such as a LAN card, a modem, etc. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the input / output (I / O) interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage portion 808 as needed.

[0187] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0188] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0189] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0190] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 802 and / or the RAM 803 described above and / or one or more memories other than the ROM 802 and the RAM 803 .

[0191] An embodiment of the present disclosure also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method provided by the embodiment of the present disclosure. When the computer program product runs on an electronic device, the program code is used to enable the electronic device to implement the task processing method provided by the embodiment of the present disclosure.

[0192] When the computer program is executed by the processor 801, the above functions defined in the system / device of the embodiment of the present disclosure are executed. According to the embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0193] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 809, and / or installed from a removable medium 811. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0194] According to an embodiment of the present disclosure, the program code for executing the computer program provided by the embodiment of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect through the Internet).

[0195] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It can be understood by those skilled in the art that the features recorded in the various embodiments and / or claims of the present disclosure can be combined and / or combined in a variety of ways, even if such a combination or combination is not explicitly recorded in the present disclosure. In particular, without departing from the spirit and teaching of the present disclosure, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0196] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A task processing method, comprising: In response to receiving a task processing request, determining current machine configuration information, wherein the task processing request includes a to-be-processed task identifier corresponding to each of the M to-be-processed tasks, and the current machine configuration information includes an execution instance identifier corresponding to each of the Q execution instances; According to the current machine configuration information, the M tasks to be processed are split into a first level to obtain N first task sets, wherein each of the N first task sets includes a first number of the tasks to be processed; Performing second-level splitting processing on the first number of tasks to be processed to obtain P second task sets; and Sending the P second task sets to target execution instances among the Q execution instances respectively, so that the target execution instances process the second task sets; Among them, M, N, P, and Q are all positive integers, and P is greater than N.

2. The method according to claim 1 further comprises, before performing second-level splitting processing on the first number of tasks to be processed to obtain P second task sets: In response to the first scheduled task being triggered, determining a task interval time; and In response to monitoring a change in the amount of data corresponding to the task to be processed, a target split ratio and a target execution time corresponding to the identifier of the task to be processed are determined.

3. The method according to claim 2, wherein: The current machine configuration information also includes a processing capability evaluation value; In response to the first scheduled task being triggered, determining the task interval time includes: In response to the first scheduled task being triggered, determining the number of tasks currently to be processed; Determining a first product value according to the processing capability evaluation value and the number of execution instances; Determining a first ratio according to the current number of tasks to be processed and the first product value; and The task interval time is determined according to the first ratio.

4. The method according to claim 2, wherein: In response to monitoring a change in the amount of data corresponding to the task to be processed, determining a target split ratio and a target execution time corresponding to the identifier of the task to be processed includes: In response to monitoring a change in the amount of data corresponding to the task to be processed, determining a first current system time and a data amount trend assessment value; Determining a target split ratio according to the data volume trend evaluation value; and The target execution time is determined according to the first current system time and the task interval time.

5. The method according to any one of claims 2 to 4, wherein: The performing second-level splitting processing on the first number of tasks to be processed to obtain P second task sets includes: Determine S target intervals according to the task interval time, where S is a positive integer; and According to the S target interval times and the target splitting ratio, the first number of tasks to be processed are split at a second level to obtain the P second task sets.

6. The method according to claim 5, wherein: Determining S target intervals according to the task interval time includes: When s=1, Determining a first target interval time according to the task interval time and the first current system time; In the case of 1<s≤S, Determine the sth interval time accumulation value according to the task interval time; and The sth target interval time is determined according to the first target interval time and the sth interval time accumulated value.

7. The method according to claim 5, wherein: The performing second-level splitting processing on the first number of tasks to be processed according to the S target intervals and the target splitting ratio to obtain the P second task sets includes: According to the target splitting ratio, the S target intervals are grouped to obtain T groups of target intervals, where T is a positive integer; and According to the T groups of target interval moments and the target execution time corresponding to each of the tasks to be processed, the first number of tasks to be processed are split into a second level to obtain a second number of tasks to be processed corresponding to each of the T groups of target interval moments.

8. The method according to any one of claims 1 to 4, wherein: The sending the P second task sets to target execution instances among the Q execution instances respectively includes: In response to the second scheduled task being triggered, determining a second current system time; Determine, according to the second current system time, the target execution instance in an idle state among the Q execution instances; Determining a target second task set from the P second task sets according to the second current system time and the T groups of target interval times; and The target second task set is sent to the target execution instance.

9. The method according to any one of claims 1 to 4, wherein: The current machine configuration information includes the number of running execution instances; The first-level splitting process is performed on the M tasks to be processed according to the current machine configuration information to obtain N first task sets including: According to the number of execution instances, the M tasks to be processed are split at a first level to obtain a first number of the tasks to be processed corresponding to each of the Q execution instance identifiers.

10. A task processing device, comprising: a first determining module, configured to determine current machine configuration information in response to receiving a task processing request, wherein the task processing request includes a to-be-processed task identifier corresponding to each of the M to-be-processed tasks, and the current machine configuration information includes an execution instance identifier corresponding to each of the Q execution instances; A first-level splitting processing module is used to perform first-level splitting processing on the M tasks to be processed according to the current machine configuration information to obtain N first task sets, wherein each of the N first task sets includes a first number of the tasks to be processed; A second-level splitting processing module, configured to perform second-level splitting processing on the first number of tasks to be processed to obtain P second task sets; and A sending module, used for sending the P second task sets to target execution instances among the Q execution instances respectively, so that the target execution instances process the second task sets; Among them, M, N, P, and Q are all positive integers, and P is greater than N.

11. An electronic device, comprising: one or more processors; a memory for storing one or more instructions, Wherein, when the one or more instructions are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having executable instructions stored thereon, wherein when the executable instructions are executed by a processor, the processor is enabled to implement the method according to any one of claims 1 to 9.

13. A computer program product, comprising computer executable instructions, which are used to implement the method of any one of claims 1 to 9 when executed.

Images