Method and apparatus for timing execution of metrics computation tasks

Abstract

The application discloses a kind of methods and devices for timing execution index calculation task, it is related to computer technical field.The specific embodiment of the method includes: in response to the arrival of the set task trigger time, trigger execution this time index calculation task;Obtain the characteristic data of this time index calculation task, and the characteristic data is input into pre-trained duration estimation model, to determine the execution duration of this time index calculation task;According to the execution duration of this time index calculation task, determine the trigger time of next time index calculation task.The embodiment is through the dynamic adjustment of task execution trigger time, when system judges that there is more idle time, reduce task trigger interval, effectively improve system resource utilization, and improve the timeliness of data processing result;Through the estimation of task execution duration, effectively avoid the situation that target calculation task trigger fails because of task execution time is too long, guarantee the robustness of business.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a method and device for executing index calculation task at fixed time. BACKGROUND

[0002] An index is a concept and its value that indicates the overall quantitative characteristics, so it is also called a comprehensive index. The process of generating an index value is called index calculation. In the field of logistics business, indexes are mainly used to count the quantitative characteristics of corresponding logistics concepts within a corresponding time period, such as warehouse in-out quantity, full-process compliance rate, and timely pickup rate. The warehouse in-out quantity refers to the in-out quantity of the warehouse; the full-process compliance rate refers to the percentage of the order quantity of the full-process compliance in the total order quantity; and the timely pickup rate refers to the percentage of the timely pickup quantity in the total pickup quantity. The existing index calculation scheme mainly includes real-time index calculation and fixed-time index calculation. Real-time index calculation is suitable for scenarios where the business has a high requirement for real-time performance and is not sensitive to resource consumption. For scenarios where the requirement for real-time performance is not high, a fixed-time triggered index calculation scheme is usually adopted.

[0003] In the process of implementing the present application, the inventors have found that at least the following problems exist in the prior art:

[0004] During the idle time from the completion of a previous index calculation to the start of the next calculation, the computing resources are in a completely idle state, and the related resources are not fully utilized, resulting in waste of resources. If the calculation time is too long, the computing resources may still be occupied by the task of the previous calculation when the current calculation starts, which may cause the current calculation to fail. This makes the calculation task that should be executed unable to be executed on time, resulting in task failure and failure to meet the requirement of the business for timeliness. SUMMARY

[0005] Therefore, the embodiments of the present application provide a method and device for executing index calculation task at fixed time, which can dynamically adjust the task execution trigger time, reduce the task trigger interval when the system determines that there is more idle time, effectively improve the system resource utilization, and improve the timeliness of the data processing result. Through the estimation of the task execution time, the situation that the target calculation task fails to be triggered due to the long task execution time can be effectively avoided, and the robustness of the business is ensured.

[0006] To achieve the above object, according to an aspect of the embodiments of the present application, a method for executing index calculation task at fixed time is provided.

[0007] The method for executing index calculation task at fixed time comprises the following steps.

[0008] In response to the arrival of the set task trigger time, the current index calculation task is triggered to be executed.

[0009] obtain feature data of the current index calculation task, and input the feature data into a pre-trained duration estimation model to determine an execution duration of the current index calculation task;

[0010] determine a trigger time of a next index calculation task according to the execution duration of the current index calculation task.

[0011] Optionally, the method further comprises: checking whether a previous index calculation task is completed before executing the current index calculation task;

[0012] and in a case where the previous index calculation task is not completed, uploading a correction mark to the duration estimation model, the correction mark being used to prompt modification of data weights of the duration estimation model.

[0013] Optionally, in a case where the previous index calculation task is not completed, the current index calculation task is spin-waited for a certain time, and then the previous index calculation task is checked again whether it is completed, until the previous index calculation task is completed.

[0014] Optionally, the method further comprises: recording an average execution speed, a processed data quantity and an execution duration of the current index calculation task after the current index calculation task is completed.

[0015] Optionally, determining a trigger time of a next index calculation task according to an execution duration of the current index calculation task comprises:

[0016] setting an execution time interval between adjacent two index calculation tasks;

[0017] determining the trigger time of the next index calculation task according to the execution duration of the current index calculation task and the execution time interval.

[0018] Optionally, after a certain number of index calculation tasks are executed, feature data of the index calculation tasks is used as training data of the duration estimation model to improve the duration estimation model.

[0019] Optionally, the duration estimation model is trained in the following manner:

[0020] collecting feature data of a certain number of index calculation tasks, the feature data comprising a minimum resource quantity of each index calculation task, a current data quantity to be processed, a daily cumulative processed data quantity, an average processing speed of the current index calculation task, a processed data quantity of the current index calculation task and an execution time of the current index calculation task;

[0021] using the feature data as training data, and using a regression algorithm to train a model to obtain the duration estimation model.

[0022] According to another aspect of the embodiments of the present application, a device for performing an index calculation task at a timing is provided.

[0023] A device for performing an index calculation task at a timing comprises:

[0024] A task triggering module is configured to trigger the execution of the current index calculation task in response to the arrival of a set task triggering time.

[0025] An execution duration determining module is configured to obtain feature data of the current index calculation task, and input the feature data into a pre-trained duration estimation model to determine the execution duration of the current index calculation task.

[0026] A triggering time determining module is configured to determine the triggering time of the next index calculation task according to the execution duration of the current index calculation task.

[0027] According to still another aspect of the embodiments of the present application, an electronic device for performing an index calculation task at a timing is provided.

[0028] An electronic device for performing an index calculation task at a timing comprises one or more processors, and a storage device configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method for performing an index calculation task at a timing provided by the embodiments of the present application.

[0029] According to still another aspect of the embodiments of the present application, a computer readable medium is provided.

[0030] A computer readable medium has a computer program stored thereon, wherein the program, when executed by a processor, implements the method for performing an index calculation task at a timing provided by the embodiments of the present application.

[0031] An embodiment of the above application has the following advantages or beneficial effects: the technical scheme triggers the execution of the current index calculation task in response to the arrival of the set task trigger time; acquires the feature data of the current index calculation task, and inputs the feature data into the pre-trained duration estimation model to determine the execution duration of the current index calculation task; determines the trigger time of the next index calculation task according to the execution duration of the current index calculation task; calculates the execution duration of the current index calculation task according to the feature data of the current index calculation task and further determines the trigger time of the next index calculation task, which can dynamically adjust the execution time of the next index calculation task according to the feature data of the current index calculation task. Thus, through the dynamic adjustment of the task execution trigger time, when the system judges that there is more idle time, the task trigger interval is reduced, the system resource utilization is effectively improved, and the timeliness of the data processing result is improved. Through the estimation of the task execution duration, the situation that the target calculation task trigger fails due to too long task execution time is effectively avoided, and the robustness of the business is ensured.

[0032] The further effects of the above-mentioned non-conventional optional mode will be described in the following combined with the specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings are used to better understand the application and do not constitute undue limitations on the application. Among them:

[0034] Figure 1 is the main step schematic diagram of the method for timing execution of an index calculation task according to an embodiment of the application;

[0035] Figure 2 is the flow schematic diagram of the method for timing execution of an index calculation task according to an embodiment of the application;

[0036] Figure 3 is the main module schematic diagram of the device for timing execution of an index calculation task according to an embodiment of the application;

[0037] Figure 4 is an exemplary system architecture diagram to which the embodiment of the application can be applied;

[0038] Figure 5 is a structural schematic diagram of a computer system of a terminal device or a server suitable for implementing the embodiment of the application. DETAILED DESCRIPTION

[0039] Exemplary embodiments of the present application are described herein below with reference to the accompanying drawings, in which various specific details are set forth to facilitate explanation of the present application. However, it should be noted that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, for the purpose of clarity and the brevity of description, descriptions of well-known functions and constructions are omitted herein.

[0040] In order to solve the technical problems existing in the prior art, the present application provides a technical solution that can dynamically adjust the task triggering time. Based on the Lasso regression algorithm, the model is trained according to the task execution time, the amount of data to be processed and other related feature data, and the recommended next task execution time is obtained, so that the task can be dynamically adjusted according to the execution of the task. The time of the next task execution. Thus, the following two problems are solved:

[0041] 1. By dynamically adjusting the task execution triggering time, when the system determines that there is more idle time, the task triggering interval is reduced, which effectively improves the system resource utilization and improves the timeliness of the data processing result;

[0042] 2. By estimating the task execution time, the situation that the target computing task triggering fails due to too long task execution time is effectively avoided, and a failure retry mechanism is added, and the failure information is fed back to the trained model for model correction, which improves the accuracy of the model output result and ensures the robustness of the business.

[0043] Figure 1 is the main step schematic diagram of the method for calculating the task according to the embodiment of the present application. As shown in Figure 1 The method for calculating the task according to the embodiment of the present application mainly includes the following steps S101 to S103.

[0044] Step S101: in response to the arrival of the set task triggering time, triggering the execution of the current index calculation task;

[0045] Step S102: obtaining the feature data of the current index calculation task, and inputting the feature data into the pre-trained time length estimation model to determine the execution time of the current index calculation task. Wherein, the feature data of the current index calculation task includes, for example: the slot (slot is a term in flink, mainly refers to a set of minimum resources for each sub-task execution in big data processing) resource number of the current task, the current data amount to be processed (pieces), the daily cumulative processing data amount (pieces), the average processing speed of the previous task (KB / s), the data number of the previous task (pieces) and the previous task calculation time (s), etc.

[0046] Step S103: determining the triggering time of the next index calculation task according to the execution duration of the current index calculation task.

[0047] According to an embodiment of the present application, the method of timing execution of an index calculation task can further comprise: checking whether the last index calculation task is completed before executing the current index calculation task; and uploading a correction mark to the duration estimation model if the last index calculation task is not completed, the correction mark being used to prompt modification of the data weight of the duration estimation model. If the last index calculation task is not completed, it indicates that the triggering time of the last index calculation task set is not appropriate or the estimated execution duration of the last index calculation task is not enough, so the data weight of the duration estimation model needs to be modified at this time to make the result of subsequent estimation of task execution duration more accurate.

[0048] According to another embodiment of the present application, in the case that the last index calculation task is not completed, the current index calculation task is spin-waited for a certain time, and then it is checked again whether the last index calculation task is completed until the last index calculation task is completed. Before the current index calculation task is executed, it is first judged whether the last task is completed, which can avoid the situation that the system is deadlocked due to direct execution of the current index calculation task when the task is not completed. By adding a failure retry mechanism and feeding back the failure information to the trained model for model correction, the accuracy of the output result of the model is improved, and the robustness of the business is ensured.

[0049] According to still another embodiment of the present application, after the current index calculation task is completed, the average execution speed, the number of processed data and the execution duration of the current index calculation task are recorded. After each task execution is completed, the average execution speed, the number of processed data and the execution duration of the task execution are recorded, which can be used as feature data for estimation of the execution duration of the next task, so that the execution duration estimated by the duration estimation model is more accurate.

[0050] According to still another embodiment of the present application, in step S103, when the triggering time of the next index calculation task is determined according to the execution duration of the current index calculation task, it can specifically comprise: setting an execution time interval between adjacent two index calculation tasks; and determining the triggering time of the next index calculation task according to the execution duration of the current index calculation task and the execution time interval. In an embodiment of the present application, the set execution time interval is, for example, 5 min. By setting the execution time interval between two tasks and summing the execution time interval and the execution duration of the current index calculation task to obtain the triggering time of the next index calculation task, a margin can be left when the triggering time of the next index calculation task is determined, which better ensures the timing execution of the task.

[0051] According to another embodiment of the present application, the feature data of the index calculation tasks is collected as training data of the time length estimation model after each index calculation task is executed a certain number of times, so as to improve the time length estimation model. In this way, the time length estimation model can be corrected according to the recorded feature data after each index calculation task is executed a certain number of times, so that the estimated execution time length is more accurate, the task triggering time is better dynamically adjusted, the utilization of system resources is effectively improved, and the timeliness of the index calculation result is improved.

[0052] According to an embodiment of the present application, the time length estimation model is trained by collecting feature data of a certain number of index calculation tasks, the feature data including the minimum number of resources of each index calculation task, the amount of data to be processed at present, the amount of data processed at present, the average processing speed of the present index calculation task, the number of processed data of the present index calculation task, and the execution time of the present index calculation task; using the feature data as training data, and using a regression algorithm to train the model to obtain the time length estimation model. In the embodiment of the present application, the certain number of times is 100, for example, and the regression algorithm is Lasso regression algorithm. Lasso is an improved algorithm based on linear regression. It adds L1 regularization to the cost function of linear regression, and the cost function is as follows:

[0053]

[0054] When the number of sample features is large and the number of samples is relatively small, the model is prone to overfitting. Lasso can be used to solve the overfitting problem of standard linear regression. By adding L1 regularization, the prior probability distribution of the parameter w satisfies Laplace distribution, so that the weight is more likely to become 0. Therefore, for the above multiple index inputs, Lasso can play a role in feature selection and avoid invalid features.

[0055] In an embodiment of the present application, the training of the duration estimation model is performed in advance, for example, the initial task trigger time can be set to 25 minutes later, and the index calculation logic is executed. The index calculation task is executed 100 times, and the slot resource number (pieces), the current data amount to be processed (pieces), the daily cumulative data amount processed (pieces), the average processing speed of this task (KB / s), the data piece number of this task (pieces), and the task calculation time (s) of each task are recorded. After the execution is completed, the recorded related data is used as feature data, and the Lasso regression algorithm is used for model training to obtain a duration estimation model, which has the ability to predict the execution duration of the current index calculation task, and can be used for adaptive adjustment of the execution interval of the task.

[0056] Figure 2 is a flowchart of a timing execution index calculation task according to an embodiment of the present application. As shown in Figure 2 , in an embodiment of the present application, the overall execution flow of a one-time index calculation task can include two parts: setting the trigger time of the next index calculation task based on the feature data of the current index calculation task, and executing the current index calculation task. Among them, the specific implementation steps of setting the trigger time of the next index calculation task based on the feature data of the current index calculation task are as follows:

[0057] Step 1.1, based on the set task trigger time to trigger the execution of the current index calculation task, first, record the feature data of the current index calculation task, mainly including the slot resource number (pieces) of the current index calculation task, the current data amount to be processed (pieces), the daily cumulative data amount processed (pieces), the average processing speed of the previous task (KB / s), the data piece number of the previous task (pieces), and the previous task calculation time (s) as the feature data of the current index calculation task;

[0058] Step 1.2, taking the execution duration of the current index calculation task as the target value, inputting the feature data collected in step 1 into the duration estimation model to estimate the execution duration of the current index calculation task;

[0059] Step 1.3, output the predicted execution duration t of the current index calculation task through the duration estimation model, assuming that the execution time interval between two index calculation tasks is set to 5 minutes, then (t+5min) is the trigger time of the next index calculation task, and the trigger time of the next task is set based on the calculated trigger time.

[0060] When the set task trigger time is reached, the current index calculation task will be triggered and executed. The specific steps of executing the current index calculation task are as follows:

[0061] Step 2.1, check if the last index calculation task is completed, if completed, directly jump to the following step 2.4;

[0062] Step 2.2, if the last index calculation task is not completed, upload a correction mark to the duration estimation model to prompt the data weight of the model training;

[0063] Step 2.3, the current index calculation task is spin waiting for a certain time (for example, 5 minutes), and then repeat step 2.1;

[0064] Step 2.4, execute the current index calculation, and record the execution duration of the current index calculation. And, upload feature data to the duration estimation model as model training data every 100 times of calculation to improve the duration estimation model.

[0065] Figure 3 is the main module schematic diagram of the device for periodically performing index calculation task according to the embodiment of the application. As Figure 3 shown, the device 300 for periodically performing index calculation task according to the embodiment of the application mainly includes a task triggering module 301, an execution duration determination module 302 and a triggering time determination module 303.

[0066] The task triggering module 301 is used for triggering the execution of the current index calculation task in response to the arrival of the set task triggering time;

[0067] The execution duration determination module 302 is used for acquiring the feature data of the current index calculation task, and inputting the feature data into the pre-trained duration estimation model to determine the execution duration of the current index calculation task;

[0068] The triggering time determination module 303 is used for determining the triggering time of the next index calculation task according to the execution duration of the current index calculation task.

[0069] According to one embodiment of the application, the device 300 for periodically performing index calculation task further includes an execution before checking module (not shown in the figure), which is used for:

[0070] checking if the last index calculation task is completed before executing the current index calculation task;

[0071] And, in the case that the last index calculation task is not completed, uploading a correction mark to the duration estimation model, the correction mark is used for prompting the data weight of the duration estimation model to be modified.

[0072] According to another embodiment of the present application, the performing previous check module (not shown in the figure) is further configured to: in the case that the previous index calculation task is not completed, the current index calculation task is spun for a certain time, and then the previous index calculation task is checked again until the previous index calculation task is completed.

[0073] According to still another embodiment of the present application, the device 300 for performing index calculation tasks at regular intervals further comprises a performing result recording module (not shown in the figure) configured to:

[0074] After the current index calculation task is completed, the average execution speed, the number of processed data and the execution time of the current index calculation task are recorded.

[0075] According to still another embodiment of the present application, the trigger time determining module 303 is further configured to:

[0076] set the execution time interval between two adjacent index calculation tasks;

[0077] determine the trigger time of the next index calculation task according to the execution time of the current index calculation task and the execution time interval.

[0078] According to still another embodiment of the present application, the device 300 for performing index calculation tasks at regular intervals further comprises a model correction module (not shown in the figure) configured to:

[0079] after a certain number of index calculation tasks are performed, the feature data of the index calculation tasks are used as training data of the time length estimation model to improve the time length estimation model.

[0080] According to still another embodiment of the present application, the time length estimation model is obtained by the following method:

[0081] the feature data of a certain number of index calculation tasks are collected, the feature data including the minimum number of resources, the amount of data to be processed, the amount of data processed in the day, the average processing speed, the number of processed data and the execution time of the current index calculation task;

[0082] the feature data is used as training data, and a regression algorithm is used for model training to obtain the time length estimation model.

[0083] According to the technical scheme of the embodiment of the present application, the execution of the current index calculation task is triggered in response to the arrival of the set task trigger time; the feature data of the current index calculation task is obtained, and the feature data is input into the pre-trained duration estimation model to determine the execution duration of the current index calculation task; and the trigger time of the next index calculation task is determined according to the execution duration of the current index calculation task. According to the feature data of the current index calculation task, the execution duration of the current index calculation task is calculated and the trigger time of the next index calculation task is further determined, and the execution time of the next index calculation task can be dynamically adjusted according to the feature data of the current index calculation task. Therefore, through the dynamic adjustment of the task execution trigger time, when the system judges that there is more idle time, the task trigger interval is reduced, the system resource utilization is effectively improved, and the timeliness of the data processing result is improved. Through the estimation of the task execution duration, the situation that the target calculation task trigger fails due to the too long task execution time is effectively avoided, and the robustness of the business is ensured.

[0084] Figure 4 An exemplary system architecture 400 is shown, which can apply the method for performing an index calculation task at a timing or the apparatus for performing an index calculation task at a timing according to the embodiments of the present application.

[0085] As shown in Figure 4 The system architecture 400 can include terminal devices 401, 402, 403, a network 404 and a server 405. The network 404 is used to provide a communication link medium between the terminal devices 401, 402, 403 and the server 405. The network 404 can include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0086] A user can use the terminal devices 401, 402, 403 to interact with the server 405 through the network 404 to receive or send messages, etc. Various client applications can be installed on the terminal devices 401, 402, 403, such as clock type applications, timing reminder type applications, feature processing type applications, model training type applications, etc. (only as examples).

[0087] The terminal devices 401, 402, 403 can be various electronic devices with display screens and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers and desktop computers, etc.

[0088] The server 405 can be a server providing various services, for example, a background management server (only for example) providing support for a user using the terminal device 401, 402, 403 to send a task trigger time setting request. The background management server can trigger execution of the current index calculation task in response to arrival of the set task trigger time in response to the received task trigger time setting request and the like, obtain characteristic data of the current index calculation task, and input the characteristic data into a pre-trained duration estimation model to determine an execution duration of the current index calculation task, determine a trigger time of a next index calculation task according to the execution duration of the current index calculation task, and the like, and feed back the processing result (for example, the task trigger time - only for example) to the terminal device.

[0089] It should be noted that the method for timing execution of an index calculation task provided by the embodiment of the application is generally executed by the server 405, and accordingly, the device for timing execution of an index calculation task is generally arranged in the server 405.

[0090] It should be understood that Figure 4 The number of terminal devices, networks and servers in the system is only illustrative. According to the needs of implementation, there can be any number of terminal devices, networks and servers.

[0091] Reference will be made to Figure 5 which shows a structural schematic diagram of a computer system 500 suitable for implementing a terminal device or a server of the embodiment of the application. Figure 5 The terminal device or server shown is only an example and should not bring any limitation to the function and use range of the embodiment of the application.

[0092] As shown in Figure 5 , the computer system 500 includes a central processing unit (CPU) 501 which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 502 or programs loaded from a storage portion 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for operation of the system 500 are also stored. The CPU 501, the ROM 502 and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0093] The following components are connected to the I / O interface 505: an input part 506 including a keyboard, a mouse, etc.; an output part 507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 508 including a hard disk, etc.; and a communication part 509 including a network interface card such as a LAN card, a modem, etc. The communication part 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as necessary. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 510 as necessary, so that a computer program read out therefrom is installed in the storage part 508 as necessary.

[0094] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-described functions defined in the system of the present disclosure are executed.

[0095] It should be noted that the computer-readable medium shown in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or component. In the present application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or component. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0096] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than that shown in the figures. For example, two blocks that are shown in succession can actually be executed substantially concurrently, or they can sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams or flowcharts, and combinations of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0097] The units or modules described in the embodiments of the present application can be implemented by software or hardware. The units or modules can also be arranged in a processor, for example, a processor can be described as including a task trigger module, an execution duration determination module and a trigger time determination module. In some cases, the names of the units or modules do not constitute a limitation on the units or modules themselves, for example, the task trigger module can also be described as a module for triggering execution of the current index calculation task in response to arrival of a set task trigger time.

[0098] As another aspect, the present application also provides a computer readable medium, which can be included in the device described in the above embodiments or exist separately without being assembled into the device. The computer readable medium carries one or more programs, which, when executed by the device, cause the device to include: triggering execution of the current index calculation task in response to arrival of a set task trigger time; obtaining feature data of the current index calculation task and inputting the feature data into a pre-trained duration estimation model to determine an execution duration of the current index calculation task; and determining a trigger time of the next index calculation task according to the execution duration of the current index calculation task.

[0099] According to the technical solution of the embodiments of the present application, the execution duration of the current index calculation task is calculated according to the feature data of the current index calculation task, and the trigger time of the next index calculation task is further determined, which can dynamically adjust the execution time of the next index calculation task according to the feature data of the current index calculation task. Thus, through dynamic adjustment of the task execution trigger time, the task trigger interval is reduced when the system judges that there is more idle time, which effectively improves the system resource utilization and improves the timeliness of the data processing result. Through estimation of the task execution duration, the situation that the target calculation task fails to be triggered due to too long execution time is effectively avoided, and the robustness of the business is ensured.

[0100] The above specific embodiments do not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made depending on design requirements and other factors. Any modification, equivalent replacement and improvement made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method of timing the execution of an indicator calculation task, characterized by, The method comprises the following steps: triggering execution of the current index calculation task in response to arrival of a set task trigger time; obtaining feature data of the current index calculation task, and inputting the feature data into a pre-trained duration estimation model to determine an execution duration of the current index calculation task; the feature data comprises a minimum resource quantity of the current index calculation task, a current data quantity to be processed, a daily cumulative processed data quantity, an average processing speed of a previous index calculation task, a processed data quantity of the previous index calculation task, and an execution duration of the previous index calculation task; and determining a trigger time of a next index calculation task according to the execution duration of the current index calculation task; recording the average processing speed, the processed data quantity, and the execution duration of the current index calculation task after the current index calculation task is executed; wherein, before the current index calculation task is executed, it is checked whether the previous index calculation task is executed; and in the case that the previous index calculation task is not executed, a correction mark is uploaded to the duration estimation model, the correction mark is used to prompt modification of data weight of the duration estimation model, and the current index calculation task is spun for a certain time, and then it is checked again whether the previous index calculation task is executed until the previous index calculation task is executed.

2. The method of claim 1, wherein, The method for determining the trigger time of the next index calculation task according to the execution duration of the current index calculation task comprises the following steps: setting an execution time interval between adjacent two index calculation tasks; determining the trigger time of the next index calculation task according to the execution duration of the current index calculation task and the execution time interval.

3. The method of claim 1, wherein, After a certain number of index calculation tasks are executed, the feature data of the index calculation tasks is used as training data of the duration estimation model to improve the duration estimation model.

4. The method of claim 1, wherein, The duration estimation model is obtained by the following method: collecting feature data of a certain number of index calculation tasks, the feature data comprising a minimum resource quantity of each index calculation task, a current data quantity to be processed, a daily cumulative processed data quantity, an average processing speed of the current index calculation task, a processed data quantity of the current index calculation task, and an execution time of the current index calculation task; using the feature data as training data, and using a regression algorithm to train the model to obtain the duration estimation model.

5. An apparatus for timing execution of an indicator calculation task, characterized by: The method comprises the following steps: a task trigger module, configured to trigger execution of the current index calculation task in response to arrival of a set task trigger time; an execution duration determination module, configured to obtain feature data of the current index calculation task, and input the feature data into a pre-trained duration estimation model to determine an execution duration of the current index calculation task; the feature data comprises a minimum resource quantity of the current index calculation task, a current data quantity to be processed, a daily cumulative processed data quantity, an average processing speed of a previous index calculation task, a processed data quantity of the previous index calculation task, and an execution duration of the previous index calculation task; a trigger time determination module, configured to determine a trigger time of a next index calculation task according to the execution duration of the current index calculation task. An execution result recording module, configured to record the average processing speed, the number of processed data and the execution time length of the current index calculation task after the execution of the current index calculation task is completed; An execution pre-checking module, configured to check whether the last index calculation task is completed before the execution of the current index calculation task; and in the case that the last index calculation task is not completed, upload a correction mark to the time length estimation model, the correction mark being used to prompt the modification of the data weight of the time length estimation model, and the current index calculation task is spun for a certain time, and then the last index calculation task is checked again whether it is completed, until the last index calculation task is completed.

6. An electronic device that performs a timing execution index calculation task, characterized by, Comprising: one or more processors; a memory device for storing one or more programs, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the method as claimed in any one of claims 1-4.

7. A computer readable medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the method as claimed in any one of claims 1-4. The program is executed by the processor to implement the method as claimed in any one of claims 1-4.

Images