A method for dynamically adjusting task memory and a terminal

Abstract

The application discloses a kind of dynamic adjustment method and terminal of task memory, monitor and calculate the average task processing rate of each subtask of pending task under historical condition;When the real-time execution data of pending task is obtained, if pending task has not been set fast task label, whether the current pending task exceeds the average task processing rate of preset proportion subtask, if yes, set fast task label for pending task;If pending task has been set fast task label, then increase the memory threshold of pending task according to preset proportion. In this way, the real-time processing rate of a task can be compared with the historical average processing rate to identify tasks that can improve processing speed by expanding memory, and dynamically configure appropriate memory threshold for the task. The dynamic adjustment of task memory can improve task processing speed and efficiency.

Description

Technical Field

[0001] This invention relates to the technical field of memory adjustment, and in particular to a method and terminal for dynamically adjusting task memory. Background Technology

[0002] In today's internet systems, there are various scheduled tasks. In order to support the processing of massive amounts of data, a distributed task processing approach must be adopted.

[0003] The typical distributed task processing workflow is as follows: a task that needs to process massive amounts of data is usually broken down into multiple subtasks, each with a fixed memory threshold for data processing. Typically, the amount of data processed by each task is different, and the internal processing logic of each task is also different, leading to varying processing efficiencies. However, the same memory threshold is set for all tasks.

[0004] Therefore, in current distributed tasks, the processing speed is limited due to the static memory threshold set for the task, thus restricting the overall task processing efficiency. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method and terminal for dynamically adjusting task memory, which can improve task processing speed and efficiency through dynamic adjustment of task memory.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] A method for dynamically adjusting task memory, comprising the following steps:

[0008] Monitor the historical execution data of the pending task, and calculate the average task processing rate of each subtask in the pending task based on the historical execution data;

[0009] Obtain the real-time execution data of the task to be processed. If the task to be processed does not have a fast task tag, then when more than a preset proportion of subtasks in the real-time execution data of the task to be processed reach the average task processing rate, a fast task tag is set for the task to be processed.

[0010] If the task to be processed has been tagged as a quick task, then the memory threshold of the task to be processed will be increased by a preset ratio.

[0011] To solve the above-mentioned technical problems, another technical solution adopted by the present invention is as follows:

[0012] A task memory dynamic adjustment terminal includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned task memory dynamic adjustment method.

[0013] The beneficial effects of this invention are as follows: It monitors and calculates the average task processing rate of each subtask of the task under historical conditions; when real-time execution data of the task is obtained, if the task has not yet been assigned a fast task tag, it determines whether the current task exceeds a preset proportion of subtasks reaching the average task processing rate; if so, it assigns a fast task tag to the task; if the task has already been assigned a fast task tag, it increases the memory threshold of the task by a preset proportion. In this way, by comparing the real-time processing rate of a task with its historical average processing rate, tasks whose processing rate can be improved by expanding memory can be identified, and appropriate memory thresholds can be dynamically configured for the tasks. This allows for dynamic adjustment of task memory to improve task processing speed and efficiency. Attached Figure Description

[0014] Figure 1 This is a flowchart of a method for dynamically adjusting task memory according to an embodiment of the present invention;

[0015] Figure 2 This is a schematic diagram of a terminal for dynamically adjusting task memory according to an embodiment of the present invention;

[0016] Label Explanation:

[0017] 1. A terminal for dynamically adjusting task memory; 2. A memory; 3. A processor. Detailed Implementation

[0018] To explain in detail the technical content, objectives, and effects of the present invention, the following description is provided in conjunction with the embodiments and accompanying drawings.

[0019] Please refer to Figure 1 This invention provides a method for dynamically adjusting task memory, comprising the following steps:

[0020] Monitor the historical execution data of the pending task, and calculate the average task processing rate of each subtask in the pending task based on the historical execution data;

[0021] Obtain the real-time execution data of the task to be processed. If the task to be processed does not have a fast task tag, then when more than a preset proportion of subtasks in the real-time execution data of the task to be processed reach the average task processing rate, a fast task tag is set for the task to be processed.

[0022] If the task to be processed has been tagged as a quick task, then the memory threshold of the task to be processed will be increased by a preset ratio.

[0023] As described above, the beneficial effects of this invention are as follows: It monitors and calculates the average task processing rate of each subtask of the task to be processed under historical conditions; when real-time execution data of the task to be processed is obtained, if the task to be processed has not yet been assigned a fast task tag, it determines whether the current task to be processed exceeds a preset proportion of subtasks reaching the average task processing rate; if so, it assigns a fast task tag to the task to be processed; if the task to be processed has already been assigned a fast task tag, it increases the memory threshold of the task to be processed by a preset proportion. In this way, by comparing the real-time processing rate of a task with its historical average processing rate, tasks whose processing rate can be improved by expanding memory can be identified, and appropriate memory thresholds can be dynamically configured for the tasks. This allows for dynamic adjustment of task memory to improve task processing speed and efficiency.

[0024] Furthermore, the historical execution data includes the number of subtasks of the task to be processed, the total amount of data processed for each subtask, and the processing time for each subtask.

[0025] The average task processing rate of each subtask in the task to be processed is calculated based on historical execution data, including:

[0026] Divide the total amount of data processed by each subtask of the task to be processed by the processing time to obtain the task processing rate of each subtask, and calculate the average task processing rate of each subtask.

[0027] As described above, by obtaining historical execution data of tasks, including the number of subtasks and the processing data of the subtasks, we can understand the historical situation of the tasks to be processed, so as to make subsequent judgments on fast tasks.

[0028] Furthermore, the memory threshold of the task to be processed is increased by a preset ratio, including:

[0029] Determine whether the memory threshold of the task to be processed has reached the preset maximum memory value. If not, increase the memory threshold of the task to be processed by a preset ratio. When the increased memory threshold exceeds the preset maximum memory value, modify the increased memory threshold to the preset maximum memory value.

[0030] As described above, when the task to be processed is a fast task, the optimal memory threshold for the task to be processed can be determined through automatic step-by-step verification, and the memory threshold can be prevented from increasing indefinitely, so as to ensure the rationality and reliability of memory adjustment.

[0031] Furthermore, if the memory threshold of the task to be processed reaches the preset maximum memory value, the number of subtasks of the task to be processed is increased.

[0032] As described above, once the optimal memory value is reached, the number of subtasks is increased to gradually and appropriately verify the number of subtasks, thereby further improving the task processing speed.

[0033] Further, the memory threshold of the task to be processed is increased by a preset ratio, followed by:

[0034] Determine whether the task processing rate of the task to be processed after raising the memory threshold is higher than the task processing rate of the task to be processed before raising the memory threshold. If not, remove the fast task tag of the task to be processed.

[0035] As described above, if the processing speed of a task does not improve after dynamically adjusting the memory threshold, it indicates that this task is not one whose execution efficiency can be improved by expanding memory. The fast task tag needs to be removed to avoid unnecessary memory expansion.

[0036] Please refer to Figure 2 Another embodiment of the present invention provides a terminal for dynamically adjusting task memory, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps of the above-described method for dynamically adjusting task memory.

[0037] The method and terminal for dynamically adjusting task memory described above are applicable to improving the task processing speed of distributed tasks. The following detailed implementation methods illustrate this:

[0038] Example 1

[0039] Please refer to Figure 1 A method for dynamically adjusting task memory, comprising the following steps:

[0040] S1. Monitor the historical execution data of the task to be processed, and calculate the average task processing rate of each subtask in the task to be processed based on the historical execution data.

[0041] The historical execution data includes the number of subtasks of the task to be processed, the total amount of data processed for each subtask, and the processing time for each subtask.

[0042] In this embodiment, for each task, it is necessary to monitor its historical execution data, which mainly includes the number of subtasks that the task is divided into, the total amount of data processed by each subtask, and the processing time of each subtask.

[0043] After monitoring the historical execution data of the task to be processed, the total amount of data processed for each subtask of the task to be processed is divided by the processing time to obtain the task processing rate of each subtask, and the average task processing rate of each subtask is calculated.

[0044] In this embodiment, the statistical data for each subtask within a task is calculated by dividing the amount of data processed by each subtask by its processing time to determine the task processing rate. The data volume refers to the amount of data transmitted to the subtask, including data transmitted from external systems or obtained from data sources, and is converted to kilobytes (KB). The processing time is measured in seconds, allowing for the precise calculation of the amount of data processed by each subtask per second.

[0045] The average task processing rate is calculated by averaging the processing rates of each subtask.

[0046] S2. Obtain the real-time execution data of the task to be processed. If the task to be processed does not have a fast task tag, then when more than a preset proportion of subtasks in the real-time execution data of the task to be processed reach the average task processing rate, a fast task tag is set for the task to be processed.

[0047] In this embodiment, assuming that the average task processing rate calculated in step S1 is 500kb / s, when at least half of the subtasks in a certain task reach this rate, the task is defaulted to a fast task and a fast task label is set for the task.

[0048] S3. If the task to be processed has been tagged with a fast task, then the memory threshold of the task to be processed is increased by a preset ratio.

[0049] Specifically, step S3 includes the following:

[0050] If the memory threshold of the task to be processed does not reach the preset maximum memory value, the memory threshold of the task to be processed is increased by a preset ratio. When the increased memory threshold exceeds the preset maximum memory value, the increased memory threshold is modified to the preset maximum memory value. Subsequently, if the task processing rate of the task to be processed after the memory threshold increase is lower than or equal to the task processing rate of the task to be processed before the memory threshold increase, the fast task tag of the task to be processed is removed.

[0051] If the memory threshold of the task to be processed reaches the preset maximum memory value, then the number of subtasks of the task to be processed is increased.

[0052] Specifically, when the task to be processed is executed again, the memory threshold of the task can be increased by a certain percentage. Then observe whether the task execution time is improved compared to before. If it is improved, continue to adjust the memory threshold of the task according to steps S1 to S3 until it is adjusted to a suitable threshold. If it is not improved, it means that this task is not one whose execution speed can be improved by expanding memory, and the fast task tag of the task needs to be removed.

[0053] At the same time, there is a maximum value for the memory threshold. When the memory threshold reaches the maximum value, it will not be adjusted further to prevent the memory threshold from increasing indefinitely. At this point, we will start trying to increase the number of subtasks and gradually verify how many subtasks are needed to achieve the optimal execution rate of the task.

[0054] In this embodiment, if the proportion of subtasks in the real-time execution data of the adjusted task to be processed is less than or equal to a preset ratio, the average task processing rate is reached, then the fast task tag is removed from the task to be processed.

[0055] Example 2

[0056] Please refer to Figure 2 A task memory dynamic adjustment terminal 1 includes a memory 2, a processor 3, and a computer program stored on the memory 2 and executable on the processor 3. When the processor 3 executes the computer program, it implements the various steps of the task memory dynamic adjustment method of Embodiment 1.

[0057] In summary, the present invention provides a method and terminal for dynamically adjusting task memory. It monitors and calculates the average task processing rate of each subtask of a task under historical conditions. When real-time execution data of the task is obtained, if the task has not yet been assigned a fast task label, it determines whether the current task exceeds a preset proportion of subtasks reaching the average task processing rate. If so, a fast task label is assigned to the task. If the task has already been assigned a fast task label, the memory threshold of the task is increased by a preset proportion. In this way, by comparing the real-time processing rate of a task with its historical average processing rate, tasks whose processing rate can be improved by expanding memory can be identified, and appropriate memory thresholds can be dynamically configured for these tasks. That is, by dynamically adjusting the memory thresholds of subtasks, the speed of subtasks can be increased, thereby improving the efficiency of the entire task.

[0058] The above description is merely an embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent modifications made based on the content of the present invention specification and drawings, or direct or indirect applications in related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for dynamically adjusting task memory, characterized in that, Including the following steps: Monitor the historical execution data of the pending task, and calculate the average task processing rate of each subtask in the pending task based on the historical execution data; Obtain the real-time execution data of the task to be processed. If the task to be processed does not have a fast task tag, then when more than a preset proportion of subtasks in the real-time execution data of the task to be processed reach the average task processing rate, a fast task tag is set for the task to be processed. If the task to be processed has been tagged as a fast task, then the memory threshold of the task to be processed is increased by a preset ratio, including: Determine whether the memory threshold of the task to be processed has reached the preset maximum memory value. If not, increase the memory threshold of the task to be processed by a preset ratio. When the increased memory threshold exceeds the preset maximum memory value, modify the increased memory threshold to the preset maximum memory value. Determine whether the task processing rate of the task to be processed after raising the memory threshold is higher than the task processing rate of the task to be processed before raising the memory threshold. If not, remove the fast task tag of the task to be processed.

2. The method for dynamically adjusting task memory according to claim 1, characterized in that, The historical execution data includes the number of subtasks of the task to be processed, the total amount of data processed for each subtask, and the processing time for each subtask. The average task processing rate of each subtask in the task to be processed is calculated based on historical execution data, including: Divide the total amount of data processed by each subtask of the task to be processed by the processing time to obtain the task processing rate of each subtask, and calculate the average task processing rate of each subtask.

3. The method for dynamically adjusting task memory according to claim 1, characterized in that, If the memory threshold of the task to be processed reaches the preset maximum memory value, then the number of subtasks of the task to be processed is increased.

4. A terminal for dynamically adjusting task memory, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it performs the following steps: Monitor the historical execution data of the pending task, and calculate the average task processing rate of each subtask in the pending task based on the historical execution data; Obtain the real-time execution data of the task to be processed. If the task to be processed does not have a fast task tag, then when more than a preset proportion of subtasks in the real-time execution data of the task to be processed reach the average task processing rate, a fast task tag is set for the task to be processed. If the task to be processed has been tagged as a fast task, then the memory threshold of the task to be processed is increased by a preset ratio, including: Determine whether the memory threshold of the task to be processed has reached the preset maximum memory value. If not, increase the memory threshold of the task to be processed by a preset ratio. When the increased memory threshold exceeds the preset maximum memory value, modify the increased memory threshold to the preset maximum memory value. Determine whether the task processing rate of the task to be processed after raising the memory threshold is higher than the task processing rate of the task to be processed before raising the memory threshold. If not, remove the fast task tag of the task to be processed.

5. A terminal for dynamically adjusting task memory according to claim 4, characterized in that, The historical execution data includes the number of subtasks of the task to be processed, the total amount of data processed for each subtask, and the processing time for each subtask. The average task processing rate of each subtask in the task to be processed is calculated based on historical execution data, including: Divide the total amount of data processed by each subtask of the task to be processed by the processing time to obtain the task processing rate of each subtask, and calculate the average task processing rate of each subtask.

6. A terminal for dynamically adjusting task memory according to claim 4, characterized in that, If the memory threshold of the task to be processed reaches the preset maximum memory value, then the number of subtasks of the task to be processed is increased.

Images