Task processing load analysis method and device

Abstract

The embodiment of the invention discloses a task processing load analysis method and device, and the method comprises the steps: determining a load analysis time period in response to a load analysis request of a computing resource node, and obtaining task information processed by the computing resource node; according to task event marking information in the task information, determining a task processing time period corresponding to each sub-task in the processing task in the load analysis time period; and determining the load condition of the computing resource node in the load analysis time period according to the load analysis time period and the task processing time period corresponding to each sub-task in the processing task in the load analysis time period. According to the technical scheme, the problem that the complex dynamic task processing load cannot be accurately analyzed through current load analysis is solved, the load can be calculated by analyzing the processing time period of the sub-task of the processing task, the actual load of the resource node can be calculated more reasonably, the effectiveness of load analysis is improved, and the resource utilization efficiency is improved.

Description

Technical Field

[0001] This case is a divisional application of an invention patent with the application number 202411374448.X, the application date of September 29, 2024, and the invention title of Task Processing Load Analysis Method and Device. The embodiments of the present invention relate to the technical field of computing resource management, and particularly to a task processing load analysis method and device. Background Art

[0002] Traditional resource scheduling methods mainly rely on the utilization rates of the CPU (Central Processing Unit) and GPU (Graphics Processing Unit) to estimate the load. Such a rough estimation method has obvious limitations when dealing with computing tasks such as large language models. The computing tasks of large language models are highly complex and dynamic. Complexity means that a task needs to be processed in multiple sub-tasks instead of the traditional processing of the entire task at once. Dynamics means that the computing power is always in a state of continuous computing and continuous receiving of new tasks, rather than the traditional state of processing one by one or a batch at the same time. Therefore, traditional load analysis often cannot accurately reflect the actual computing load, resulting in uneven resource allocation and reducing the utilization efficiency of computing resources. Summary of the Invention

[0003] The embodiments of the present invention provide a task processing load analysis method and device, which can calculate the load by analyzing the processing time periods of the sub-tasks of the processing tasks, calculate the actual load of the computing resource nodes more reasonably, improve the effectiveness of load analysis, and improve the resource utilization efficiency.

[0004] In a first aspect, the embodiments of the present invention provide a task processing load analysis method, which includes:

[0005] Respond to the load analysis request of the computing resource node to determine the load analysis time period;

[0006] Based on the load analysis time period, obtain the task information processed by the computing resource node;

[0007] According to the task event marking information in the task information, determine the task processing time periods corresponding to each sub-task in the processing task during the load analysis time period;

[0008] According to the load analysis time period and the task processing time periods corresponding to each sub-task in the processing task during the load analysis time period, determine the load situation of the computing resource node during the load analysis time period.

[0009] In a second aspect, the embodiments of the present invention provide a task processing load analysis device, which includes:

[0010] A load analysis time period determination module, configured to determine a load analysis time period in response to a load analysis request of a computing resource node;

[0011] A task information acquisition module, configured to acquire task information processed by a computing resource node based on the load analysis time period;

[0012] A task processing period determination module, configured to determine corresponding task processing periods of each subtask in a processing task within the load analysis time period according to task event marking information in the task information;

[0013] A load condition determination module, configured to determine the load condition of a computing resource node within the load analysis time period according to the load analysis time period and the corresponding task processing periods of each subtask in a processing task within the load analysis time period.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, which includes:

[0015] One or more processors;

[0016] A memory, configured to store one or more programs;

[0017] When the above one or more programs are executed by the one or more processors, the one or more processors implement the task processing load analysis method provided in any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the task processing load analysis method provided in any embodiment of the present invention.

[0019] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the task processing load analysis method provided in any embodiment of the present invention.

[0020] The embodiments in the above invention have the following advantages or beneficial effects:

[0021] In an embodiment of the present invention, in response to a load analysis request of a computing resource node, a load analysis time period is determined; based on the load analysis time period, task information processed by the computing resource node is obtained; according to the task event marking information in the task information, task processing time periods corresponding to each subtask in the processing task within the load analysis time period are determined; and according to the load analysis time period and the task processing time periods corresponding to each subtask in the processing task within the load analysis time period, the load condition of the computing resource node within the load analysis time period is determined. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze complex dynamic task processing loads. The load can be calculated by analyzing the processing time periods of the subtasks of the processing task, and the actual load of the computing resource node can be calculated more reasonably, improving the effectiveness of load analysis and the resource utilization efficiency. Description of the Drawings

[0022] Figure 1 is a flowchart of a method for analyzing task processing load provided by an embodiment of the present invention;

[0023] Figure 2 is a flowchart of a method for analyzing task processing load provided by an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of task processing provided by an embodiment of the present invention;

[0025] Figure 4 is a flowchart of a method for analyzing task processing load provided by an embodiment of the present invention;

[0026] Figure 5 is a flowchart of a method for analyzing task processing load provided by an embodiment of the present invention;

[0027] Figure 6 is a schematic structural diagram of a device for analyzing task processing load provided by an embodiment of the present invention;

[0028] Figure 7 is a schematic structural diagram of a computer device provided by an embodiment of the present invention. Detailed Embodiments

[0029] The present invention will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the present invention. Additionally, it should be noted that for the sake of description, only parts related to the present invention are shown in the drawings rather than all the structures.

[0030] Figure 1The flowchart of a task processing load analysis method provided by an embodiment of the present invention is applicable to scenarios where the load of task processing is analyzed. This method can be executed by a task processing load analysis device, which can be implemented in software and / or hardware and integrated into a computer device with application development functions.

[0031] As Figure 1 shown, the task processing load analysis method of this embodiment includes the following steps:

[0032] S110. In response to a load analysis request from a computing resource node, determine a load analysis time period.

[0033] The computing resource node can be any node in a distributed computing system used to execute computing tasks, specifically a computing node that includes at least one or a combination of several resources among hardware computing resources, software computing resources, and network computing resources. In response to a load analysis request from the computing resource node, determine the load analysis time period according to the request time of the load analysis request. The load analysis request can be triggered based on the acquisition of a new task. Since in this embodiment, the processing of computing tasks has the characteristic of dynamism, the computing resource node is always in a state of continuous computing and continuous reception of new tasks. Therefore, the preset duration before the request time, or the preset duration after the request time, or the preset duration before and after the request time can be taken as the load analysis time period. The preset time can be 30 seconds, one minute, five minutes, etc. This embodiment does not limit the length of the preset time.

[0034] S120. Based on the load analysis time period, obtain the task information processed by the computing resource node.

[0035] Based on the time interval corresponding to the load analysis time period, obtain the task information within this time interval. Among them, the task information can be obtained from the log information, and the task information can include task event marking information generated by means of task marking / reporting during the task execution process.

[0036] S130. According to the task event marking information in the task information, determine the task processing time periods corresponding to each subtask in the processing task during the load analysis time period.

[0037] The processing task can be composed of at least two subtasks. For example, the natural language processing task of a natural language model can include an input understanding and initialization subtask and a recursive inference and decoding output subtask. The input understanding and initialization subtask can first tokenize the user input, calculate the self-attention, and generate a KV (Key-Value) cache, and then sample and output, returning to the client; the recursive inference and decoding output subtask predicts words one by one, calculates the attention, calculates for each token, and then performs decoding conversion to obtain the model processing result.

[0038] The task event marking information may be marking information used to characterize the task processing status of each subtask in a processing task. The task time marking information may include time point information such as the start of task processing, completion of task processing, suspension of task processing, and restart of task processing after suspension. According to the above time point information, the intersection period between the load analysis time period and the processing time of each subtask is determined, and the intersection period is determined as the task processing period corresponding to each subtask in the processing task within the load analysis time period. It can be understood that the subtask processing time and the load analysis time period do not necessarily have an inclusion relationship. For example, if a subtask has started processing before the start time of the load analysis time period, or the subtask has completed processing after the end time of the load analysis time period, only the processing time within the load analysis time period is calculated.

[0039] S140. Determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing periods corresponding to each subtask in the processing task within the load analysis time period.

[0040] According to the load analysis time period and the task processing periods corresponding to each subtask in the processing task within the load analysis time period, according to the characteristics of the task processing methods of each subtask, determine the load duration of the task processing period and the relationship between the load duration and the load analysis time period. According to the relationship between the equivalent duration of the task processing period and the load analysis time period, determine the load condition of the computing resource node within the load analysis time period. The processing methods may include a serial processing method and a parallel processing method. The subtasks of the serial processing method may be subtasks that cannot be processed simultaneously with the subtasks of other processing tasks, that is, only one subtask of one processing task can be processed at a time. The parallel processing method may refer to subtasks that are processed simultaneously with the subtasks of other processing tasks, that is, multiple subtasks of multiple processing tasks can be processed at a time.

[0041] The technical solution of this embodiment determines the load analysis time period in response to a load analysis request of a computing resource node; obtains task information processed by the computing resource node based on the load analysis time period; determines the task processing periods corresponding to each subtask in the processing task within the load analysis time period according to the task event marking information in the task information; and determines the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing periods corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze complex dynamic task processing loads. It can calculate the load by analyzing the processing periods of the subtasks of the processing task, calculate the actual load of the computing resource node more reasonably, improve the effectiveness of load analysis, and improve resource utilization efficiency.

[0042] Figure 2 The figure is a flowchart of a task processing load analysis method provided by an embodiment of the present invention. This embodiment and the task processing load analysis method in the above embodiment belong to the same inventive concept, and further describes the process of determining the task processing period and determining the load situation according to the task processing period. This method can be executed by a task processing load analysis device, which can be implemented in a software and / or hardware manner and integrated into a computer device with application development functions.

[0043] As Figure 2 shown, the task processing load analysis method of this embodiment includes the following steps:

[0044] S210. In response to a load analysis request of a computing resource node, determine a load analysis time period.

[0045] S220. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0046] S230. According to the task event marking information in the task information, determine the start and end times of each subtask.

[0047] The task event marking information in the task information can be collected by means of logging / reporting. For example, the general process of logging / reporting is that when a behavior / status that needs to be collected occurs, it is recorded in the log. For example, the start and end times of each subtask of each processing task are all events to be collected. After collecting the task event marking information by logging, select an opportunity to report the log containing the task event marking information.

[0048] The specific method of logging can be any one of code logging, declarative logging, traceless logging, and agentless logging. Code logging can be adding logging code manually at specific business code to collect task event marking information. Declarative logging can be collecting task event marking information by adding event identifiers and business fields as attributes to response controls. Simplify the amount of code for code logging. Traceless logging can be obtaining all operations and then deciding which task event marking information needs to be reported. Agentless logging can be reporting all operations and having the server filter the task event marking information.

[0049] S240. Based on the start and end times of each subtask and the load analysis time period, determine the task processing period corresponding to each subtask within the load analysis time period.

[0050] As Figure 3 shown, Figure 3It includes 4 processing tasks. Taking the example that each processing task includes 2 subtasks, the 2 subtasks are the prefile and decoding subtasks. The prefile can be the input understanding and initialization subtask, and the decoding can be the recursive reasoning and decoding output subtask. Among them, the prefile is processed first, and the decoding is processed later. If a new task is obtained during the decoding process, the decoding needs to be paused, and the prefile of the new task is processed. When the prefile of the new task is processed, the paused decoding and the decoding of the new task are continued. The prefile is processed serially, while the decoding can be processed in parallel.

[0051] Taking the subtasks of processing task 1 as an example, the prefile of processing task 1 has been processed before the load analysis time period from 0 to 30 seconds. During the decoding process, it starts to be processed and enters the 0th second of the load analysis time period after about several seconds, pauses at the 4th second, resumes at the 10th second, pauses again at the 13th second, resumes at the 20th second, and is processed and completed at the 24th second. Then the task processing period of the decoding is that the intersection time of the processing time of the decoding subtask within the load analysis time period is from 0 to 4 seconds, from 10 to 13 seconds, and from 20 to 24 seconds.

[0052] S250. Determine the serial task load duration of the computing resource node within the load analysis time period according to the task processing periods corresponding to the serially processed subtasks in the task processing periods.

[0053] As Figure 3 shown, the task processing periods corresponding to the prefiles of the serially processed subtasks of all processing tasks are from 4 to 10 seconds, from 13 to 20 seconds, and from 28 to 30 seconds. Determine that the serial task load duration of the computing resource node within the load analysis time period is the duration of the task processing period, that is, the serial task load duration is 10 - 4 = 6s, 20 - 13 = 7s, and 30 - 28 = 2s.

[0054] S260. Determine the parallel task load equivalent duration of the computing resource node within the load analysis time period according to the task processing periods corresponding to the parallelly processed subtasks in the task processing periods and the preset maximum number of parallel processing tasks.

[0055] The equivalent duration of parallel task load is the duration that can characterize the actual load of the task processing period corresponding to the subtasks processed in parallel. In the task processing period corresponding to the subtasks processed in parallel, there is a situation where the number of subtasks does not reach the preset maximum number of parallel processing tasks. Therefore, the actual load cannot be measured by the actual processing duration, but by the numerical relationship between the number of subtasks processed in the task processing period and the preset maximum number of parallel processing tasks, and the equivalent duration of parallel task load of the computing resource node within the load analysis time period is calculated and determined. The actual load of processing the subtasks processed in parallel is measured by the equivalent duration of parallel task load. The preset maximum number of parallel processing tasks can be set according to actual needs, and this embodiment does not limit it.

[0056] In an alternative embodiment, determining the equivalent duration of parallel task load of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks may include the following steps A1 - A3:

[0057] Step A1: Determine the number of subtasks processed in parallel within the same period according to the task processing period corresponding to the subtasks processed in parallel in the task processing period.

[0058] As Figure 3 shown, the task processing periods corresponding to the subtasks processed in parallel include: period one, period two, period three, and period four, corresponding to the 0th to 4th seconds, the 10th to 13th seconds, the 20th to 24th seconds, and the 24th to 28th seconds respectively, and the numbers of subtasks processed in parallel corresponding to them are 1, 2, 3, and 2 respectively.

[0059] Step A2: Calculate the ratio of the number of subtasks processed in parallel within the same period to the preset maximum number of parallel processing tasks.

[0060] Exemplarily, the preset maximum number of parallel processing tasks is 3, and the ratio of the number of subtasks processed in parallel within the same period to the preset maximum number of parallel processing tasks, that is, the ratio values corresponding to period one to four are 1 / 3, 2 / 3, 1, and 2 / 3.

[0061] Step A3: Obtain the equivalent duration of parallel task load of the computing resource node within the load analysis time period according to the ratio value and the duration of the task processing period corresponding to the subtasks processed in parallel.

[0062] The equivalent duration of parallel task load of the computing resource node within the load analysis time period: period one: (4 - 0) * 1 / 3 = 4 / 3 s, period two: (13 - 10) * 2 / 3 = 2 s, period three: (24 - 20) * 3 / 3 = 4 s, and period four: (28 - 24) * 2 / 3 = 8 / 3 s.

[0063] S270. Determine the load condition of the computing resource node within the load analysis time period according to the serial task load duration and the parallel task load equivalent duration.

[0064] According to the calculated serial task load duration and parallel task load equivalent duration corresponding to the task processing period, calculate and determine the load condition of the computing resource node within the load analysis time period.

[0065] In an alternative embodiment, determining the load condition of the computing resource node within the load analysis time period according to the serial task load duration and the parallel task load equivalent duration may be to accumulate the serial task load duration and the parallel task load equivalent duration to obtain the total load time; calculate the ratio of the total load time to the duration of the load analysis time period, and use the ratio as the load condition of the computing resource node within the load analysis time period.

[0066] For example, the accumulated value of the serial task load duration is 6 + 7 + 2 = 15s. The accumulated value of the parallel task load equivalent duration is 4 / 3 + 2 + 4 + 8 / 3 = 10s. The total load time is the sum of the accumulated values of the load durations or load equivalent durations of the subtasks of the two processing methods, that is, the total load time = 15 + 10 = 25s. The load condition of the computing resource node within the load analysis time period is the ratio of the total load time to the duration of the load analysis time period, buzy = 25 / 30 = 83%. Optionally, use the difference idle between the ratio and 1 and the ratio as the load condition of the computing resource node within the load analysis time period at the same time. idle = 1 - 83% = 17%. When busy is 0%, it means the computing resource node is completely idle. When the busy value is 100%, it means the computing resource node cannot undertake more processing tasks. idle represents the idle condition of the computing resource node. When busy is 0%, idle is 100%. When busy is 20%, idle is 80%.

[0067] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; obtains the task information processed by the computing resource node based on the load analysis time period; determines the start and end times of each subtask according to the task event marking information in the task information; determines the task processing period corresponding to each subtask within the load analysis time period based on the start and end times of each subtask and the load analysis time period; determines the serial task load duration of the computing resource node within the load analysis time period according to the task processing periods corresponding to the serially processed subtasks in the task processing period; determines the parallel task load equivalent duration of the computing resource node within the load analysis time period according to the task processing periods corresponding to the parallelly processed subtasks in the task processing period and the preset maximum number of parallel processing tasks; determines the load condition of the computing resource node within the load analysis time period according to the serial task load duration and the parallel task load equivalent duration. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load. It can calculate the load by analyzing the processing periods of the subtasks of the processing task, and calculate the actual load of the computing resource node more reasonably according to the load durations or load equivalent durations corresponding to serial and parallel processing respectively, improving the effectiveness of load analysis and the resource utilization efficiency.

[0068] Figure 4 It is a flowchart of a task processing load analysis method provided by an embodiment of the present invention. This embodiment and the task processing load analysis method in the above embodiment belong to the same inventive concept, and further describes the process of determining the cluster load condition of the computing resource cluster corresponding to the computing resource node within the load analysis time period. This method can be executed by a task processing load analysis device, and this device can be implemented in a software and / or hardware manner and integrated in a computer device with application development functions.

[0069] As Figure 4 shown, the task processing load analysis method of this embodiment includes the following steps:

[0070] S310. Respond to the load analysis request of the computing resource node and determine the load analysis time period.

[0071] S320. Obtain the task information processed by the computing resource node based on the load analysis time period.

[0072] S330. Determine the task processing periods corresponding to each subtask in the processing task within the load analysis time period according to the task event marking information in the task information.

[0073] S340. Determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing periods corresponding to each subtask in the processing task within the load analysis time period.

[0074] S350. Determine the cluster load condition of the computing resource cluster corresponding to the computing resource node during the load analysis time period according to the load condition of the computing resource node during the load analysis time period.

[0075] According to the load conditions of multiple computing resource nodes during the load analysis time period, determine the cluster load condition of the computing resource cluster corresponding to the computing resource node during the load analysis time period by calculating the average value. For example, add the ratios corresponding to the load conditions of 2 computing resource nodes and then divide by 2 to obtain the ratio corresponding to the cluster load condition of the computing resource cluster composed of these two computing resource nodes during the load analysis time period. For example, if the ratio corresponding to the load condition of one computing resource node is 10% and the ratio corresponding to the load condition of another computing resource node is 20%, then the ratio corresponding to the cluster load condition of the computing resource cluster composed of these two computing resource nodes during the load analysis time period is (10% + 20%) / 2 = 15%.

[0076] S360. Send the cluster load condition to the task assignment center so that the task assignment center schedules and processes tasks according to the cluster load condition to achieve load balancing among computing resource clusters.

[0077] For each computing resource cluster, send the cluster load condition of the computing resource cluster to the task assignment center, so that the task assignment center schedules and processes tasks according to the cluster load condition to achieve load balancing and fine-grained scheduling among computing resource clusters.

[0078] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; obtains the task information processed by the computing resource node based on the load analysis time period; determines the corresponding task processing time periods of each subtask in the processing task within the load analysis time period according to the task event marking information in the task information; determines the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the corresponding task processing time periods of each subtask in the processing task; determines the cluster load condition of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load condition of the computing resource node within the load analysis time period; and sends the cluster load condition to the task allocation center so that the task allocation center schedules the processing task according to the cluster load condition to achieve load balancing among the computing resource clusters. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load, can calculate the load by analyzing the processing time periods of the subtasks of the processing task, more reasonably calculate the actual load of the computing resource node, and determine the actual load of the computing resource cluster through the load conditions of multiple computing resource nodes, improving the effectiveness of the load analysis and the resource utilization efficiency.

[0079] Figure 5 It is a flowchart of a method for analyzing the task processing load provided by an embodiment of the present invention. This embodiment and the method for analyzing the task processing load in the above embodiment belong to the same inventive concept, and further describe the process of determining the load condition of the computing resource node within the load analysis time period according to the task attributes of the subtasks. This method can be executed by a task processing load analysis device, which can be implemented in a software and / or hardware manner and integrated in a computer device with application development functions.

[0080] As Figure 5 shown, the method for analyzing the task processing load in this embodiment includes the following steps:

[0081] S410. Respond to the load analysis request of the computing resource node and determine the load analysis time period.

[0082] S420. Obtain the task information processed by the computing resource node based on the load analysis time period.

[0083] S430. Obtain the task attributes of at least one subtask according to the task information processed by the computing resource node.

[0084] The subtasks include at least one subtask corresponding to each processing stage of the task. The processing task includes various different processing stages, and for each stage, a corresponding model algorithm module runs to implement the corresponding model operation function. One processing stage corresponds to one subtask. If the processing task includes three processing stages, then the processing task includes three subtasks respectively corresponding to different processing stages.

[0085] The task attributes include serial / parallel processing attributes and / or processing priority attributes.

[0086] The serial / parallel processing attributes include serial processing attributes and parallel processing attributes. The serial processing attributes indicate that the subtask can only be processed alone, while the parallel processing attributes indicate that the subtask can be processed simultaneously with other subtasks that also have parallel processing attributes. The subtasks with parallel processing attributes can be subtasks of the same type or subtasks of different task types. The task type refers to the type of the model algorithm unit that the subtask runs or the functional attribute corresponding to the task implementation, such as implementing natural language text input understanding and initialization, or recursive reasoning and decoding output.

[0087] The processing priority attribute can be the processing priority among subtasks of multiple different task types.

[0088] S440. Determine the task processing time periods corresponding to each subtask in the processing task during the load analysis time period according to the task attributes of at least one subtask and the task event marking information in the task information.

[0089] When making dot reports, according to the task attributes of the subtasks, determine the impact of the start or completion of the subtask on the start and end times of other subtasks that are being processed. For example, the start of a subtask with a higher priority causes other subtasks with lower priorities to pause processing, or the end of a subtask causes multiple subtasks that start simultaneously to be processed in parallel. And according to the impact of the start of the subtask on other subtasks that are being processed, determine the start and end times of each subtask. Optionally, trigger corresponding events based on the start and end times of each subtask to generate task event marking information.

[0090] Determine the task processing time periods corresponding to each subtask in the processing task during the load analysis time period according to the task event marking information.

[0091] Exemplarily, the relationship between the task attributes of the subtasks and the start and end times is described through the following situations.

[0092] Example Scenario 1: The processing task includes two subtasks corresponding to two processing stages, namely subtask A and subtask B. Subtask A is processed serially, while subtask B can be processed in parallel with subtasks B of other processing tasks. The processing priority is that when subtask A of a new processing task is obtained, the processing of subtask B is paused. That is, the processing priority of subtask A is higher than that of subtask B.

[0093] Example Scenario 2: The processing task includes three subtasks corresponding to three processing stages, namely subtask A, subtask B, and subtask C. Subtask A is processed serially. Subtask B can be processed in parallel with at most one subtask B of another processing task, or in parallel with at most one subtask C of another processing task. Subtask C can be processed in parallel with at most two subtasks C of other processing tasks, or in parallel with at most one subtask B of another processing task.

[0094] When there is subtask A, subtask A must be processed first. When there is no subtask A, subtask B is processed first. When there is no subtask B, subtask C is processed.

[0095] The processing method can also be adjusted according to the number of subtasks. For example, when there are only 2 subtasks B, subtask B can be processed individually without serial processing. Similarly, the processing priority can also be adjusted according to the number of subtasks. For example, the processing priority can be 2 subtasks B > 1 subtask B + 1 subtask C > 1 subtask B > subtask C. That is, when two subtasks B and one subtask C are obtained, the two subtasks B are processed in parallel first, and then the 1 subtask C is processed.

[0096] S450. Determine the load condition of the computing resource node during the load analysis time period according to the load analysis time period, the task processing time periods corresponding to each subtask in the processing task during the load analysis time period, and the task attributes of at least one subtask.

[0097] According to the task processing time periods corresponding to each subtask with serial / parallel processing attributes during the load analysis time period, calculate the serial load duration and the parallel equivalent load duration of each subtask during the load analysis time period. Accumulate the serial load duration and the parallel equivalent load duration to obtain the total load time. The load condition of the computing resource node during the load analysis time period is the ratio of the total load time to the duration of the load analysis time period.

[0098] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; obtains the task information processed by the computing resource node based on the load analysis time period; obtains the task attributes of at least one subtask according to the task information processed by the computing resource node; determines the corresponding task processing time periods of each subtask in the processing task within the load analysis time period according to the task attributes of at least one subtask and the task event marking information in the task information; and determines the load condition of the computing resource node within the load analysis time period according to the load analysis time period, the corresponding task processing time periods of each subtask in the processing task within the load analysis time period, and the task attributes of at least one subtask. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex load analysis of the serial and parallel processing of complex subtasks and the processing according to priorities. It can determine the processing time periods of the subtasks of the processing task by analyzing the task attributes of the subtasks, calculate the load more reasonably, improve the effectiveness of the load analysis, and improve the resource utilization efficiency.

[0099] Figure 6 FIG. is a schematic structural diagram of a task processing load analysis device provided by an embodiment of the present invention. This embodiment is applicable to the scenario of analyzing the load of task processing. The task processing load analysis device can be implemented in a software and / or hardware manner and integrated into a computer terminal device with application development functions.

[0100] As Figure 6 shown, the task processing load analysis device includes: a load analysis time period determination module 510, a task information acquisition module 520, a task processing time period determination module 530, and a load condition determination module 540.

[0101] Among them, the load analysis time period determination module 510 is configured to determine the load analysis time period in response to the load analysis request of the computing resource node; the task information acquisition module 520 is configured to obtain the task information processed by the computing resource node based on the load analysis time period; the task processing time period determination module 530 is configured to determine the corresponding task processing time periods of each subtask in the processing task within the load analysis time period according to the task event marking information in the task information; and the load condition determination module 540 is configured to determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the corresponding task processing time periods of each subtask in the processing task within the load analysis time period.

[0102] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; obtains the task information processed by the computing resource node based on the load analysis time period; determines the task processing time periods corresponding to each subtask in the processing task within the load analysis time period according to the task event marking information in the task information; and determines the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time periods corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load, can calculate the load by analyzing the processing time periods of the subtasks of the processing task, calculate the actual load of the computing resource node more reasonably, improve the effectiveness of the load analysis, and improve the resource utilization efficiency.

[0103] In an alternative embodiment, the task processing time period determination module 530 is specifically configured to:

[0104] Determine the start and end times of each subtask according to the task event marking information in the task information; and determine the task processing time period corresponding to each subtask within the load analysis time period based on the start and end times of each subtask and the load analysis time period.

[0105] In an alternative embodiment, the load condition determination module 540 is specifically configured to:

[0106] Determine the serial task load duration of the computing resource node within the load analysis time period according to the task processing time periods corresponding to the subtasks processed serially in the task processing time periods; determine the parallel task load equivalent duration of the computing resource node within the load analysis time period according to the task processing time periods corresponding to the subtasks processed in parallel in the task processing time periods and the preset maximum number of parallel processing tasks; and determine the load condition of the computing resource node within the load analysis time period according to the serial task load duration and the parallel task load equivalent duration.

[0107] In an alternative embodiment, the load condition determination module 540 is further configured to:

[0108] Determine the number of subtasks processed in parallel within the same time period according to the task processing time periods corresponding to the subtasks processed in parallel in the task processing time periods; calculate the ratio value of the number of subtasks processed in parallel within the same time period to the preset maximum number of parallel processing tasks; and obtain the parallel task load equivalent duration of the computing resource node within the load analysis time period according to the ratio value and the duration of the task processing time periods corresponding to the corresponding subtasks processed in parallel.

[0109] In an alternative embodiment, the load condition determination module 540 is further configured to:

[0110] Accumulate the duration of the serial task load and the equivalent duration of the parallel task load to obtain the total load time; calculate the ratio of the total load time to the duration of the load analysis time period, and use the ratio as the load condition of the computing resource node within the load analysis time period.

[0111] In an alternative embodiment, the apparatus further includes:

[0112] A cluster load analysis module, configured to determine the cluster load condition of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load condition of the computing resource node within the load analysis time period; send the cluster load condition to the task distribution center, so that the task distribution center schedules and processes tasks according to the cluster load condition to achieve load balancing among computing resource clusters.

[0113] In an alternative embodiment, the subtasks include at least one subtask corresponding to each processing stage of the processing task, and the apparatus further includes:

[0114] A task attribute acquisition module, configured to acquire the task attributes of at least one subtask according to the task information processed by the computing resource node.

[0115] In an alternative embodiment, the load condition determination module 540 is further configured to:

[0116] Determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period, the task processing time periods corresponding to each subtask in the processing task within the load analysis time period, and the task attributes of at least one subtask.

[0117] In an alternative embodiment, the task attributes include serial / parallel processing attributes and / or processing priority attributes, and the task processing time period determination module 530 is further configured to:

[0118] Determine the task processing time periods corresponding to each subtask in the processing task within the load analysis time period according to the task attributes of at least one subtask and the task event marking information in the task information.

[0119] In an alternative embodiment, the apparatus further includes:

[0120] A marking information generation module, configured to trigger corresponding events based on the start and end times of each subtask, and generate task event marking information.

[0121] The task processing load analysis apparatus provided by the embodiments of the present invention can execute the task processing load analysis method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0122] Figure 7 It is a schematic structural diagram of a computer device provided by an embodiment of the present invention.Figure 7 FIG. shows a block diagram of an exemplary computer device 12 suitable for implementing embodiments of the present invention. Figure 7 The computer device 12 shown is merely an example and should not impose any limitation on the functions and usage scope of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as intelligent controllers, servers, mobile phones, and other terminal devices.

[0123] As Figure 7 shown, the computer device 12 is presented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0124] The bus 18 represents one or more of several types of bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. By way of example, these architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus.

[0125] The computer device 12 typically includes a variety of computer system-readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0126] The system memory 28 may include computer system-readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, a storage system 34 can be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, commonly referred to as a "hard disk drive"). Although Figure 7 not shown in, a disk drive for reading and writing removable non-volatile disks (such as "floppy disks") and an optical disk drive for reading and writing removable non-volatile optical disks (such as CD-ROM, DVD-ROM, or other optical media) can be provided. In these cases, each drive can be connected to the bus 18 through one or more data media interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0127] A program / utilities 40 having a set (at least one) of program modules 42 can be stored, for example, in the system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. The program modules 42 generally execute the functions and / or methods in the embodiments described in the present invention.

[0128] The computer device 12 can also communicate with one or more external devices 14 (such as a keyboard, a pointing device, a display 24, etc.), and can also communicate with one or more devices that enable a user to interact with the computer device 12, and / or communicate with any device that enables the computer device 12 to communicate with one or more other computing devices (such as a network card, a modem, etc.). Such communication can be carried out through the input / output (I / O) interface 22. Moreover, the computer device 12 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 20. As shown in the figure, the network adapter 20 communicates with other modules of the computer device 12 through the bus 18. It should be understood that although Figure 7 not shown in the figure, other hardware and / or software modules can be used in combination with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0129] The processing unit 16 executes various functional applications and data processing by running the programs stored in the system memory 28. For example, it implements the task processing load analysis method provided by the embodiments of the present invention. The method includes:

[0130] In response to a load analysis request of a computing resource node, determining a load analysis time period;

[0131] Based on the load analysis time period, obtaining task information processed by the computing resource node;

[0132] According to the task event marking information in the task information, determining the corresponding task processing time periods of each subtask in the processing task during the load analysis time period;

[0133] According to the load analysis time period and the corresponding task processing time periods of each subtask in the processing task during the load analysis time period, determining the load situation of the computing resource node during the load analysis time period.

[0134] An embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the task processing load analysis method provided in any embodiment of the present invention. The method includes:

[0135] Respond to the load analysis request of the computing resource node to determine the load analysis time period;

[0136] Based on the load analysis time period, obtain the task information processed by the computing resource node;

[0137] According to the task event marking information in the task information, determine the corresponding task processing time periods of each subtask in the processing task during the load analysis time period;

[0138] According to the load analysis time period and the corresponding task processing time periods of each subtask in the processing task during the load analysis time period, determine the load condition of the computing resource node during the load analysis time period.

[0139] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0140] The computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable medium may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device.

[0141] The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber cable, RF, etc., or any suitable combination of the above.

[0142] The computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, Python, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0143] An embodiment of the present invention also provides a computer program product, including a computer program which, when executed by a processor, implements the task processing load analysis method provided in any embodiment of the present application.

[0144] In the process of implementing the computer program product, the computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, Python, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0145] Those of ordinary skill in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. Optionally, they can be implemented with program codes executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0146] Note that the above is only the preferred embodiment of the present invention and the technical principles applied. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A task processing load analysis method, characterized in that: include: In response to a load analysis request from a computing resource node, determining a load analysis time period; Based on the load analysis time period, obtaining task information processed by the computing resource node; Determining, according to the task event marking information in the task information, the task processing period corresponding to each subtask in the processing task within the load analysis time period, comprises: Determine the start and end time of each subtask according to the task event marking information in the task information; Based on the start and end time of each subtask and the load analysis time period, determine the task processing time period corresponding to each subtask within the load analysis time period; Determining the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period includes: Determine the serial task load duration of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed serially in the task processing period; Determining the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks, comprising: Determine the number of subtasks to be processed in parallel within the same task processing period according to the task processing period corresponding to the subtasks to be processed in parallel within the task processing period; Calculate the ratio of the number of subtasks processed in parallel within the same period to the preset maximum number of tasks processed in parallel; Obtaining the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the ratio value and the duration of the task processing period of the corresponding subtask processed in parallel; The load condition of the computing resource node within the load analysis time period is determined according to the serial task load duration and the parallel task load equivalent duration.

2. The method according to claim 1, characterized in that The determining, according to the serial task load duration and the parallel task load equivalent duration, the load condition of the computing resource node within the load analysis time period comprises: The serial task load duration and the parallel task load equivalent duration are added together to obtain a total load time; The ratio of the total load time to the length of the load analysis time period is calculated, and the ratio is used as the load condition of the computing resource node within the load analysis time period.

3. The method according to claim 1, characterized in that The method also includes: Determine the cluster load condition of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load condition of the computing resource node within the load analysis time period; The cluster load condition is sent to a task allocation center so that the task allocation center schedules processing tasks according to the cluster load condition to achieve load balancing among computing resource clusters.

4. The method according to claim 1, characterized in that: The subtask includes at least one subtask corresponding to each processing stage of the processing task, and the method further includes: Acquire task attributes of at least one subtask according to task information processed by the computing resource node; The determining, according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period, the load condition of the computing resource node within the load analysis time period comprises: The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask.

5. The method according to claim 4, characterized in that The task attributes include serial-parallel processing attributes and / or processing priority attributes, and determining, based on the task event tag information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period includes: According to the task attribute of the at least one subtask and the task event marking information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period is determined.

6. The method according to claim 4, characterized in that The determining the load condition of the computing resource node within the load analysis time period according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask, comprises: According to the task processing period corresponding to each subtask with serial and parallel processing attributes in the load analysis time period, the serial load duration and parallel equivalent load duration of each subtask in the load analysis time period are calculated; Accumulate the serial load duration and parallel equivalent load duration of each subtask to get the total load time; The ratio of the total load time to the length of the load analysis time period is taken as the load condition of the resource node within the load analysis time period.

7. A task processing load analysis device, characterized in that: include: A load analysis time period determination module, used to determine a load analysis time period in response to a load analysis request of a computing resource node; A task information acquisition module, used to acquire task information processed by the computing resource node based on the load analysis time period; A task processing period determination module, used to determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event marking information in the task information; The task processing period determination module is specifically used for: Determine the start and end time of each subtask according to the task event tag information in the task information; determine the task processing period corresponding to each subtask within the load analysis time period based on the start and end time of each subtask and the load analysis time period; A load condition determination module, used to determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period; The load condition determination module is specifically used for: Determine the serial task load duration of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed serially in the task processing period; Determine the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks; determine the load of the computing resource node within the load analysis time period according to the serial task load duration and the equivalent duration of the parallel task load; The load condition determination module is also used for: According to the task processing time periods corresponding to the subtasks processed in parallel in the task processing time period, the number of subtasks processed in parallel in the same time period is determined; the ratio of the number of subtasks processed in parallel in the same time period to the preset maximum number of tasks processed in parallel is calculated; according to the ratio and the duration of the task processing time period of the corresponding subtasks processed in parallel, the equivalent duration of the parallel task load of the computing resource node in the load analysis time period is obtained.

8. A computer device comprising a memory, a processor or a processing unit, and a computer program stored in the memory and executable on the processor or the processing unit, characterized in that: When the processor or processing unit executes the program, the task processing load analysis method as described in any one of claims 1-6 is implemented.

9. A storage medium storing computer executable instructions, characterized in that: The computer executable instructions are used to perform the task processing load analysis method as described in any one of claims 1-6 when executed by a computer processor.

Images