Task processing load analysis method and device

Abstract

The embodiment of the present invention discloses a method and device for analyzing the load of task processing, wherein the method includes: responding to a load analysis request of a computing resource node, determining a load analysis time period, and obtaining task information processed by the computing resource node; determining the task processing time period corresponding to each subtask in the processing task within the load analysis time period based on the task event tag information in the task information; and determining the load of the computing resource node within the load analysis time period based on the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the load of complex dynamic task processing. The load can be calculated by analyzing the processing time period of the subtask of the processing task, and the actual load of the computing resource node can be more reasonably calculated, thereby improving the effectiveness of the load analysis and improving the efficiency of resource utilization.

Description

Technical Field

[0001] This case is a divisional application of the invention patent application number 202411374448.X, filed on September 29, 2024, entitled "Task Processing Load Analysis Method and Apparatus." The embodiments of the present invention relate to the field of computing resource management technology, and more particularly, to a task processing load analysis method and apparatus. Background Art

[0002] Traditional resource scheduling methods mainly rely on the utilization of the CPU (Central Processing Unit) and GPU (Graphics Processing Unit) to estimate the load. This rough estimation method has obvious limitations when dealing with tasks such as large-scale language model calculations. The computing tasks of large-scale language models are highly complex and dynamic. Complexity means that a task needs to be processed in multiple sub-tasks, rather than the traditional processing of the entire task. Dynamicity means that the computing power is always in a state of continuous computing and processing and constantly receiving new tasks, rather than the traditional state of processing one by one or processing a batch at the same time. Therefore, traditional load analysis often cannot accurately reflect the actual computing load, resulting in uneven resource allocation and reduced utilization efficiency of computing resources. Summary of the Invention

[0003] The embodiment of the present invention provides a task processing load analysis method and device, which can calculate the load by analyzing the processing period of the subtasks of the processing task, more reasonably calculate the actual load of the computing resource node, improve the effectiveness of load analysis, and improve resource utilization efficiency.

[0004] In a first aspect, an embodiment of the present invention provides a method for analyzing task processing load, the method comprising:

[0005] In response to a load analysis request from a computing resource node, determining a load analysis time period;

[0006] Based on the load analysis time period, obtain the task information processed by the computing resource node;

[0007] Determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information;

[0008] The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period and the task processing period corresponding to each subtask in the processing task within the load analysis time period.

[0009] In a second aspect, an embodiment of the present invention provides a task processing load analysis device, the device comprising:

[0010] A load analysis time period determination module, configured to determine a load analysis time period in response to a load analysis request from a computing resource node;

[0011] The task information acquisition module is used to obtain the task information processed by the computing resource node based on the load analysis time period;

[0012] The task processing period determination module is used to determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information;

[0013] The load condition determination module is used to determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, comprising:

[0015] one or more processors;

[0016] a memory for storing one or more programs;

[0017] When the one or more programs are executed by one or more processors, the one or more processors implement the task processing load analysis method provided by any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the task processing load analysis method provided by any embodiment of the present invention.

[0019] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the task processing load analysis method provided by any embodiment of the present invention.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] The embodiment of the present invention determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; determines the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information; and determines the load of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load. The load can be calculated by analyzing the processing time period of the subtask of the processing task, and the actual load of the computing resource node can be more reasonably calculated, thereby improving the effectiveness of the load analysis and improving the efficiency of resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of a task processing load analysis method provided by an embodiment of the present invention;

[0023] Figure 2 This is a flow chart of a task processing load analysis method provided by an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a task processing provided by an embodiment of the present invention;

[0025] Figure 4 This is a flow chart of a task processing load analysis method provided by an embodiment of the present invention;

[0026] Figure 5 This is a flow chart of a task processing load analysis method provided by an embodiment of the present invention;

[0027] Figure 6 This is a structural diagram of a task processing load analysis device provided by an embodiment of the present invention;

[0028] Figure 7 It is a structural diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0030] Figure 1This is a flowchart of a task processing load analysis method provided in an embodiment of the present invention. This embodiment is applicable to scenarios where task processing load analysis is required. The method can be performed by a task processing load analysis device, which can be implemented using software and / or hardware and integrated into a computer device with application development capabilities.

[0031] like Figure 1 As shown, the task processing load analysis method of this embodiment includes the following steps:

[0032] S110 : In response to a load analysis request from a computing resource node, determine a load analysis time period.

[0033] A computing resource node can be any node in a distributed computing system that is used to perform computing tasks, and specifically can be a computing node that includes at least one or a combination of several of hardware computing resources, software computing resources, and network computing resources. In response to a load analysis request from a computing resource node, the load analysis time period is determined based on the request time of the load analysis request, and the load analysis request can be triggered based on the acquisition of a new task. Since, in this embodiment, the processing of computing tasks is dynamic, the computing resource node is always in a state of continuous computing and processing and continuous reception of new tasks. Therefore, the preset time length before or after the request time or the preset time length before and after the request time can be taken as the load analysis time period. The preset time can be 30 seconds, one minute, five minutes, etc. This embodiment does not limit the length of the preset time.

[0034] S120. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0035] Based on the time interval corresponding to the load analysis time period, task information within the time interval is obtained. The task information can be obtained from log information, and the task information can include task event marking information generated by marking / reporting tasks during task execution.

[0036] S130 : Determine, based on the task event tag information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0037] A processing task can consist of at least two subtasks. For example, the natural language processing task of a natural language model can include an input understanding and initialization subtask and a recursive reasoning and decoding output subtask. The input understanding and initialization subtask can first tokenize the user input, calculate self-attention and generate a KV (key-value) cache, and then sample the output and return it to the customer; the recursive reasoning and decoding output subtask predicts and calculates attention word by word, calculates each token, and then performs decoding conversion to obtain the model processing result.

[0038] The task event marking information may be marking information used to characterize the task processing status of each subtask in the processing task. The task time marking information may include the time point information of the task starting processing, completing processing, pausing processing, and restarting processing after pausing. Based on the above time point information, the intersection period of the load analysis time cycle and the processing time of each subtask is determined, and the intersection period is determined as the task processing period corresponding to each subtask in the processing task within the load analysis time cycle. It can be understood that the subtask processing time and the load analysis time cycle do not necessarily have an inclusive relationship. For example, if the subtask has started processing before the start time of the load analysis time cycle, or the subtask has completed processing after the end time of the load analysis time cycle, only the processing time within the load analysis time cycle is calculated.

[0039] S140 , determining the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0040] According to the load analysis time cycle and the task processing time period corresponding to each subtask in the processing task within the load analysis time cycle, and according to the characteristics of the task processing method of each subtask, the load duration of the task processing time period and the relationship between the load duration and the load analysis time cycle are determined. According to the relationship between the equivalent duration of the task processing time period and the load analysis time cycle, the load situation of the computing resource node within the load analysis time cycle is determined. The processing method may include a serial processing method and a parallel processing method. The subtask of the serial processing method may be a subtask that cannot be processed simultaneously with the subtasks of other processing tasks, that is, only one subtask of a processing task can be processed at a time. The parallel processing method may refer to a subtask that is processed simultaneously with the subtasks of other processing tasks, that is, multiple subtasks of multiple processing tasks can be processed at a time.

[0041] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; determines the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information; and determines the load of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load. The load can be calculated by analyzing the processing time period of the subtask of the processing task, and the actual load of the computing resource node can be more reasonably calculated, thereby improving the effectiveness of the load analysis and improving the efficiency of resource utilization.

[0042] Figure 2 This is a flowchart of a task processing load analysis method provided in an embodiment of the present invention. This embodiment, which shares the same inventive concept as the task processing load analysis method described in the previous embodiment, further describes the process of determining a task processing period and determining the load based on the period. This method can be performed by a task processing load analysis device, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.

[0043] like Figure 2 As shown, the task processing load analysis method of this embodiment includes the following steps:

[0044] S210 : Determine a load analysis time period in response to a load analysis request from a computing resource node.

[0045] S220. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0046] S230: Determine the start and end time of each subtask according to the task event mark information in the task information.

[0047] Task event marker information can be collected through logging and reporting. For example, the general process of logging and reporting is to record the behavior or status that requires collection in a log. For example, the start and end times of each subtask in each processing task are collected events. After logging and collecting task event marker information, the log containing the task event marker information is reported at an appropriate time.

[0048] The specific methods of tracking can be any of the following: code tracking, declared tracking, invisible tracking, and no tracking. Code tracking means manually adding tracking code to specific business codes to collect task event marker information. Declarative tracking means collecting task event marker information by adding event identifiers and business fields as attributes to response controls. This simplifies the amount of code for code tracking. In invisible tracking, all operations are obtained and then the task event marker information that needs to be reported is determined. In no tracking, all operations are reported and the server filters the task event marker information.

[0049] S240 : Determine a task processing period corresponding to each subtask within the load analysis time period based on the start and end time of each subtask and the load analysis time period.

[0050] like Figure 3 As shown, Figure 3The system includes four processing tasks, each consisting of two subtasks, namely prefile and decoding. Prefile can be the input understanding and initialization subtask, while decoding can be the recursive reasoning and decoding output subtask. Prefile is processed first, followed by decoding. If a new task is received during decoding, decoding is paused to process the prefile for the new task. After processing the prefile for the new task is complete, decoding resumes, along with decoding for the new task. Prefile processing is serial, while decoding can be performed in parallel.

[0051] For example, the subtasks of Task 1 complete their prefile processing between seconds 0 and 30 of the load analysis time period. During decoding, processing begins a few seconds later at second 0 of the load analysis time period. The decoding process pauses at second 4, resumes at second 10, pauses again at second 13, resumes at second 20, and completes at second 24. The decoding task processing period is the intersection of the decoding subtask processing time within the load analysis time period: seconds 0 to 4, seconds 10 to 13, and seconds 20 to 24.

[0052] S250 , determining the serial task load duration of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed serially in the task processing period.

[0053] like Figure 3 As shown in the figure, the task processing periods corresponding to the serial processing subtask profiles of all processing tasks are 4 to 10 seconds, 13 to 20 seconds, and 28 to 30 seconds. The serial task load duration of the computing resource node within the load analysis time period is determined as the duration of the task processing period, that is, the serial task load duration is 10-4 = 6 seconds, 20-13 = 7 seconds, and 30-28 = 2 seconds.

[0054] S260: Determine the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks.

[0055] The equivalent duration of the parallel task load is the duration that can characterize the actual load of the task processing period corresponding to the subtasks processed in parallel. In the task processing period corresponding to the subtasks processed in parallel, there are cases where the number of subtasks does not reach the preset maximum number of parallel processing tasks. Therefore, the actual load cannot be measured by the actual processing duration. Instead, the equivalent duration of the parallel task load of the computing resource node within the load analysis time period is calculated based on the numerical relationship between the number of subtasks processed in the task processing period and the preset maximum number of parallel processing tasks. The actual load of the subtasks processed in parallel is measured by the equivalent duration of the parallel task load. The preset maximum number of parallel processing tasks can be set according to actual needs, and this embodiment does not limit this.

[0056] In an optional embodiment, determining the equivalent duration of the parallel task load of the computing resource node within the load analysis time period based on the task processing period corresponding to the subtasks processed in parallel within the task processing period and a preset maximum number of parallel processing tasks may include the following steps A1-A3:

[0057] Step A1: Determine the number of subtasks to be processed in parallel within the same task processing period according to the task processing period corresponding to the subtasks to be processed in parallel within the same task processing period.

[0058] like Figure 3 As shown, the task processing periods corresponding to the subtasks processed in parallel include: period one, period two, period three and period four, corresponding to seconds 0 to 4, seconds 10 to 13, seconds 20 to 24 and seconds 24 to 28, respectively, and the corresponding numbers of subtasks processed in parallel are 1, 2, 3 and 2, respectively.

[0059] Step A2: Calculate the ratio of the number of subtasks processed in parallel within the same time period to the preset maximum number of tasks processed in parallel.

[0060] For example, the preset maximum number of parallel processing tasks is 3, and the ratio of the number of subtasks processed in parallel in the same period to the preset maximum number of parallel processing tasks, that is, the ratios corresponding to periods one to four, are 1 / 3, 2 / 3, 1 and 2 / 3.

[0061] Step A3: Obtain the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the ratio value and the duration of the task processing period of the corresponding parallel processed subtask.

[0062] The equivalent duration of the parallel task load of the computing resource node within the load analysis time period is: period one: (4-0)*1 / 3=4 / 3s, period two: (13-10)*2 / 3=2s, period three: (24-20)*3 / 3=4s and period four: (28-24)*2 / 3=8 / 3s.

[0063] S270: Determine the load condition of the computing resource node within the load analysis time period according to the serial task load duration and the parallel task load equivalent duration.

[0064] Based on the calculated serial task load duration and parallel task load equivalent duration corresponding to the task processing period, the load situation of the computing resource node within the load analysis time period is calculated and determined.

[0065] In an optional embodiment, the load condition of the computing resource node within the load analysis time period is determined based on the serial task load duration and the equivalent parallel task load duration. The serial task load duration and the equivalent parallel task load duration can be accumulated to obtain the total load time; the ratio of the total load time to the duration of the load analysis time period is calculated, and the ratio is used as the load condition of the computing resource node within the load analysis time period.

[0066] For example, the cumulative load duration of serial tasks is 6 + 7 + 2 = 15 seconds. The cumulative equivalent load duration of parallel tasks is 4 / 3 + 2 + 4 + 8 / 3 = 10 seconds. The total load duration is the sum of the load durations or equivalent load durations of the subtasks in the two processing modes, that is, total load time = 15 + 10 = 25 seconds. The load status of the computing resource node during the load analysis time period is the ratio of the total load time to the load analysis time period duration: buzy = 25 / 30 = 83%. Optionally, the difference between the ratio and 1, "idle," and the ratio are used together as the load status of the computing resource node during the load analysis time period. Idle = 1 - 83% = 17%. A busy value of 0% indicates that the computing resource node is completely idle, while a busy value of 100% indicates that the computing resource node cannot take on any more processing tasks. Idle indicates the idle status of the computing resource node. When busy is 0%, idle is 100%, and when busy is 20%, idle is 80%.

[0067] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; obtains the task information processed by the computing resource node based on the load analysis time period; determines the start and end time of each subtask according to the task event mark information in the task information; determines the task processing time period corresponding to each subtask within the load analysis time period based on the start and end time of each subtask and the load analysis time period; determines the serial task load duration of the computing resource node within the load analysis time period according to the task processing time period corresponding to the serially processed subtasks in the task processing time period; determines the equivalent parallel task load duration of the computing resource node within the load analysis time period according to the task processing time period corresponding to the parallel processed subtasks in the task processing time period and the preset maximum number of parallel processing tasks; determines the load condition of the computing resource node within the load analysis time period according to the serial task load duration and the equivalent parallel task load duration. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load. The load can be calculated by analyzing the processing period of the subtask of the processing task, and the actual load of the computing resource node can be more reasonably calculated according to the load duration or load equivalent duration corresponding to serial and parallel processing, thereby improving the effectiveness of load analysis and improving resource utilization efficiency.

[0068] Figure 4 This is a flowchart of a task processing load analysis method provided in an embodiment of the present invention. This embodiment, which shares the same inventive concept as the task processing load analysis method described in the previous embodiment, further describes the process of determining the cluster load of a computing resource cluster corresponding to a computing resource node within a load analysis period. This method can be performed by a task processing load analysis device, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.

[0069] like Figure 4 As shown, the task processing load analysis method of this embodiment includes the following steps:

[0070] S310 : In response to a load analysis request from a computing resource node, determine a load analysis time period.

[0071] S320. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0072] S330 : Determine, based on the task event tag information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0073] S340: Determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0074] S350: Determine the cluster load condition of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load condition of the computing resource node within the load analysis time period.

[0075] Based on the load conditions of multiple computing resource nodes during the load analysis time period, the cluster load conditions of the computing resource cluster corresponding to the computing resource nodes during the load analysis time period are determined by calculating an average value. For example, the ratio corresponding to the load conditions of two computing resource nodes is added and then divided by 2 to obtain the ratio corresponding to the cluster load conditions of the computing resource cluster composed of the two computing resource nodes during the load analysis time period. For example, if the ratio corresponding to the load condition of one computing resource node is 10% and the ratio corresponding to the load condition of the other computing resource node is 20%, then the ratio corresponding to the cluster load conditions of the computing resource cluster composed of the two computing resource nodes during the load analysis time period is (10% + 20%) / 2 = 15%.

[0076] S360: Send the cluster load situation to the task allocation center, so that the task allocation center schedules and processes tasks according to the cluster load situation to achieve load balancing among computing resource clusters.

[0077] For each computing resource cluster, the cluster load status of the computing resource cluster is sent to the task allocation center, so that the task allocation center schedules and processes tasks according to the cluster load status to achieve load balancing and refined scheduling among computing resource clusters.

[0078] The technical solution of this embodiment is to determine the load analysis time period by responding to the load analysis request of the computing resource node; obtain the task information processed by the computing resource node based on the load analysis time period; determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information; determine the load of the computing resource node within the load analysis time period according to the load analysis time period and the task processing period corresponding to each subtask in the processing task within the load analysis time period; determine the cluster load of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load of the computing resource node within the load analysis time period; and send the cluster load to the task allocation center so that the task allocation center can schedule the processing tasks according to the cluster load to achieve load balancing between computing resource clusters. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load. The load can be calculated by analyzing the processing period of the subtask of the processing task, and the actual load of the computing resource node can be more reasonably calculated. The actual load of the computing resource cluster can be determined based on the load of multiple computing resource nodes, thereby improving the effectiveness of load analysis and improving resource utilization efficiency.

[0079] Figure 5 This is a flowchart of a task processing load analysis method provided in an embodiment of the present invention. This embodiment, which shares the same inventive concept as the task processing load analysis method described in the previous embodiment, further describes the process of determining the load of a computing resource node within the load analysis period based on the task attributes of a subtask. This method can be performed by a task processing load analysis device, which can be implemented in software and / or hardware and integrated into a computer device with application development capabilities.

[0080] like Figure 5 As shown, the task processing load analysis method of this embodiment includes the following steps:

[0081] S410 : In response to a load analysis request from a computing resource node, determine a load analysis time period.

[0082] S420. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0083] S430. Obtain task attributes of at least one subtask according to task information processed by the computing resource node.

[0084] A subtask includes at least one subtask corresponding to each processing stage of a processing task. A processing task includes various processing stages, each of which runs a corresponding model algorithm module to implement the corresponding model operation function. Each processing stage corresponds to one subtask. If a processing task includes three processing stages, then the processing task includes three subtasks corresponding to different processing stages.

[0085] The task attributes include serial and parallel processing attributes and / or processing priority attributes.

[0086] Serial-parallel processing attributes include serial processing attributes and parallel processing attributes. Serial processing attributes indicate that the subtask can only be processed independently, while parallel processing attributes indicate that the subtask can be processed simultaneously with other subtasks that also have parallel processing attributes. Subtasks with parallel processing attributes can be of the same type or of different task types. Task type refers to the type of algorithm unit in the subtask's execution model or the functional attributes that the task implements, such as natural language text input understanding and initialization, or recursive reasoning and decoding output.

[0087] The processing priority attribute may be a processing priority among subtasks of multiple different task types.

[0088] S440: Determine a task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task attribute of at least one subtask and the task event mark information in the task information.

[0089] During point reporting, the impact of the subtask's start or completion on the start and end times of other ongoing subtasks is determined based on the subtask's attributes. For example, the start of a high-priority subtask may cause the suspension of other lower-priority subtasks, or the completion of a subtask may cause the simultaneous start of multiple subtasks. The start and end times of each subtask are determined based on the impact of the subtask's start on other ongoing subtasks. Optionally, corresponding events are triggered based on the start and end times of each subtask, generating task event marker information.

[0090] The task processing time period corresponding to each subtask in the processing task within the load analysis time period is determined according to the task event mark information.

[0091] As an example, the relationship between the task attributes of a subtask and its start and end times is explained through the following situations.

[0092] Example 1: A processing task includes two subtasks corresponding to two processing stages, namely subtask A and subtask B. Subtask A is processed serially, while subtask B can be processed in parallel with subtask Bs of other processing tasks. The processing priority is such that when subtask A of a new processing task is obtained, the processing of subtask B is suspended. In other words, subtask A has a higher processing priority than subtask B.

[0093] Example 2: A processing task includes three subtasks corresponding to three processing stages, namely, subtask A, subtask B, and subtask C. Subtask A is processed serially, subtask B can be processed in parallel with at most one subtask B of another processing task, or with at most one subtask C of another processing task, and subtask C can be processed in parallel with at most two subtask Cs of other processing tasks, or with at most one subtask B of another processing task.

[0094] When subtask A exists, subtask A must be processed first. When subtask A does not exist, subtask B must be processed first. When subtask B does not exist, subtask C must be processed first.

[0095] The processing method can also be adjusted based on the number of subtasks. For example, if there are only two subtasks B, subtask B can be processed independently, rather than serially. Similarly, the processing priority can also be adjusted based on the number of subtasks. For example, the processing priority can be 2 subtasks B > 1 subtask B + 1 subtask C > 1 subtask B > subtask C. That is, if two subtasks B and one subtask C are obtained, the two subtasks B will be processed in parallel first, followed by the single subtask C.

[0096] S450. Determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period, the task processing period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask.

[0097] According to the task processing time period corresponding to each subtask with serial and parallel processing attributes within the load analysis time period, the serial load duration and parallel equivalent load duration of each subtask within the load analysis time period are calculated, and the total load time is obtained by adding the serial load duration and the parallel equivalent load duration. The load condition of the resource node within the load analysis time period is calculated, that is, the ratio of the total load time to the duration of the load analysis time period.

[0098] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; obtains the task attributes of at least one subtask according to the task information processed by the computing resource node; determines the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task attributes of at least one subtask and the task event tag information in the task information; determines the load situation of the computing resource node within the load analysis time period according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period and the task attributes of at least one subtask. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex subtask string parallel processing and the load analysis of complex situations according to priority processing. It can determine the processing time period calculation load of the subtask of the processing task by analyzing the task attributes of the subtask, more reasonably calculate the actual load of the computing resource node, improve the effectiveness of load analysis, and improve resource utilization efficiency.

[0099] Figure 6 This is a schematic diagram of the structure of a task processing load analysis device provided by an embodiment of the present invention. This embodiment is applicable to scenarios where task processing load analysis is performed. The task processing load analysis device can be implemented using software and / or hardware and integrated into a computer terminal device with application development capabilities.

[0100] like Figure 6 As shown, the task processing load analysis device includes: a load analysis time period determination module 510, a task information acquisition module 520, a task processing period determination module 530 and a load condition determination module 540.

[0101] Among them, the load analysis time period determination module 510 is used to determine the load analysis time period in response to the load analysis request of the computing resource node; the task information acquisition module 520 is used to obtain the task information processed by the computing resource node based on the load analysis time period; the task processing period determination module 530 is used to determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event mark information in the task information; the load situation determination module 540 is used to determine the load situation of the computing resource node within the load analysis time period according to the load analysis time period and the task processing period corresponding to each subtask in the processing task within the load analysis time period.

[0102] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; determines the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information; and determines the load of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load. The load can be calculated by analyzing the processing time period of the subtask of the processing task, and the actual load of the computing resource node can be more reasonably calculated, thereby improving the effectiveness of the load analysis and improving the efficiency of resource utilization.

[0103] In an optional implementation, the task processing period determination module 530 is specifically configured to:

[0104] Determine the start and end time of each subtask based on the task event tag information in the task information; and determine the task processing period corresponding to each subtask within the load analysis time period based on the start and end time of each subtask and the load analysis time period.

[0105] In an optional implementation, the load condition determination module 540 is specifically configured to:

[0106] According to the task processing period corresponding to the subtasks processed serially in the task processing period, determine the serial task load duration of the computing resource node within the load analysis time period; according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks, determine the equivalent parallel task load duration of the computing resource node within the load analysis time period; according to the serial task load duration and the equivalent parallel task load duration, determine the load situation of the computing resource node within the load analysis time period.

[0107] In an optional implementation, the load condition determination module 540 is further configured to:

[0108] According to the task processing period corresponding to the subtasks processed in parallel in the task processing period, the number of subtasks processed in parallel in the same period is determined; the ratio of the number of subtasks processed in parallel in the same period to the preset maximum number of tasks processed in parallel is calculated; according to the ratio and the duration of the task processing period of the corresponding subtasks processed in parallel, the equivalent duration of the parallel task load of the computing resource node in the load analysis time period is obtained.

[0109] In an optional implementation, the load condition determination module 540 is further configured to:

[0110] The serial task load duration and the equivalent parallel task load duration are added together to obtain the total load time. The ratio of the total load time to the load analysis time period is calculated and used as the load condition of the resource node during the load analysis time period.

[0111] In an optional embodiment, the device further comprises:

[0112] The cluster load analysis module is used to determine the cluster load status of the computing resource cluster corresponding to the computing resource node within the load analysis time period based on the load status of the computing resource node within the load analysis time period; and send the cluster load status to the task allocation center so that the task allocation center can schedule and process tasks according to the cluster load status to achieve load balancing among computing resource clusters.

[0113] In an optional embodiment, the subtask includes at least one subtask corresponding to each processing stage of the processing task, and the apparatus further includes:

[0114] The task attribute acquisition module is used to obtain the task attributes of at least one subtask based on the task information processed by the computing resource node.

[0115] In an optional implementation, the load condition determination module 540 is further configured to:

[0116] The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask.

[0117] In an optional embodiment, the task attributes include serial and parallel processing attributes and / or processing priority attributes, and the task processing period determination module 530 is further configured to:

[0118] According to the task attribute of at least one subtask and the task event marking information in the task information, a task processing time period corresponding to each subtask in the processing task within the load analysis time period is determined.

[0119] In an optional embodiment, the device further comprises:

[0120] The tag information generation module is used to trigger corresponding events based on the start and end time of each subtask and generate task event tag information.

[0121] The task processing load analysis device provided in the embodiment of the present invention can execute the task processing load analysis method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0122] Figure 7 A schematic structural diagram of a computer device provided in an embodiment of the present invention. Figure 7 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 7 The computer device 12 shown is only an example and should not limit the functionality and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as an intelligent controller, a server, a mobile phone, or other terminal devices.

[0123] like Figure 7 As shown, computer device 12 is implemented as a general-purpose computing device. Components of computer device 12 may include, but are not limited to, one or more processors or processing units 16, system memory 28, and a bus 18 that connects various system components (including system memory 28 and processing unit 16).

[0124] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0125] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0126] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be configured to read and write non-removable, non-volatile magnetic media ( Figure 7 Not shown, often called a "hard drive"). Although Figure 7 Not shown, a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of various embodiments of the present invention.

[0127] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0128] The computer device 12 may also communicate with one or more external devices 14 (e.g., a keyboard, a pointing device, a display 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RFID systems, tape drives, and data backup storage systems.

[0129] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the task processing load analysis method provided in the embodiment of the present invention, which includes:

[0130] In response to a load analysis request from a computing resource node, determining a load analysis time period;

[0131] Based on the load analysis time period, obtain the task information processed by the computing resource node;

[0132] Determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information;

[0133] The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period and the task processing period corresponding to each subtask in the processing task within the load analysis time period.

[0134] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for analyzing task processing load provided in any embodiment of the present invention is implemented. The method includes:

[0135] In response to a load analysis request from a computing resource node, determining a load analysis time period;

[0136] Based on the load analysis time period, obtain the task information processed by the computing resource node;

[0137] Determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information;

[0138] The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period and the task processing period corresponding to each subtask in the processing task within the load analysis time period.

[0139] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device.

[0140] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0141] Program code embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0142] The computer program code for performing the operations of the present invention can be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, Python, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0143] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the task processing load analysis method provided in any embodiment of the present application.

[0144] The computer program product, during implementation, may be written in one or more programming languages, or a combination thereof, for performing the operations of the present invention. The programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, Python, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0145] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computing device. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computer device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module. Thus, the present invention is not limited to any specific combination of hardware and software.

[0146] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A task processing load analysis method, characterized in that: include: In response to a load analysis request from a computing resource node, determining a load analysis time period; Based on the load analysis time period, obtaining task information processed by the computing resource node; Determining, based on the task event tag information in the task information, the task processing period corresponding to each subtask in the processing task within the load analysis time period, includes: Determining the start and end time of each subtask according to the task event marking information in the task information; Determining a task processing period corresponding to each subtask within the load analysis time period based on the start and end time of each subtask and the load analysis time period; Determining a load condition of the computing resource node within the load analysis time period according to the load analysis time period and a task processing period corresponding to each subtask in the processing task within the load analysis time period includes: Determining the serial task load duration of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed serially in the task processing period; Determining the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the task processing time periods corresponding to the subtasks processed in parallel in the task processing time period and a preset maximum number of parallel processing tasks, including: Determining the number of subtasks to be processed in parallel within the same task processing period according to the task processing period corresponding to the subtasks to be processed in parallel within the task processing period; Calculate the ratio of the number of subtasks processed in parallel during the same period to the preset maximum number of tasks processed in parallel; Obtaining the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the ratio value and the duration of the task processing period of the corresponding parallel processed subtask; The load condition of the computing resource node within the load analysis time period is determined according to the serial task load duration and the parallel task load equivalent duration.

2. The method according to claim 1, characterized in that The determining, based on the serial task load duration and the parallel task load equivalent duration, of the load condition of the computing resource node within the load analysis time period includes: Add the serial task load duration and the parallel task load equivalent duration to obtain the total load time; The ratio of the total load time to the length of the load analysis time period is calculated, and the ratio is used as the load condition of the computing resource node within the load analysis time period.

3. The method according to claim 1, characterized in that The method also includes: Determine the cluster load of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load condition of the computing resource node within the load analysis time period; The cluster load condition is sent to a task allocation center, so that the task allocation center schedules processing tasks according to the cluster load condition, thereby achieving load balancing among computing resource clusters.

4. The method according to claim 1, wherein The subtask includes at least one subtask corresponding to each processing stage of the processing task, and the method further includes: Obtaining task attributes of at least one subtask based on task information processed by the computing resource node; The determining, based on the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period, of the load condition of the computing resource node within the load analysis time period includes: The load condition of the computing resource node within the load analysis time period is determined based on the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask.

5. The method according to claim 4, characterized in that The task attributes include serial and parallel processing attributes and / or processing priority attributes. The determining, based on the task event tag information in the task information, of the task processing time period corresponding to each subtask in the processing task within the load analysis time period includes: The task processing time period corresponding to each subtask in the processing task within the load analysis time period is determined according to the task attribute of the at least one subtask and the task event marking information in the task information.

6. The method according to claim 4, characterized in that The determining, based on the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask, of the load analysis time period includes: Calculate the serial load duration and parallel equivalent load duration of each subtask within the load analysis time period according to the task processing period corresponding to each subtask with serial and parallel processing attributes within the load analysis time period; Accumulate the serial load duration and parallel equivalent load duration of each subtask to obtain the total load time; The ratio of the total load time to the load analysis time period is used as the load condition of the resource node during the load analysis time period.

7. A task processing load analysis device, characterized in that: include: A load analysis time period determination module, configured to determine a load analysis time period in response to a load analysis request from a computing resource node; A task information acquisition module, configured to acquire task information processed by the computing resource node based on the load analysis time period; a task processing period determination module, configured to determine, based on the task event tag information in the task information, the task processing period corresponding to each subtask in the processing task within the load analysis time period; The task processing period determination module is specifically used to: Determine the start and end time of each subtask based on the task event tag information in the task information; determine the task processing period corresponding to each subtask within the load analysis time period based on the start and end time of each subtask and the load analysis time period; a load condition determination module, configured to determine a load condition of the computing resource node within the load analysis time period according to the load analysis time period and a task processing period corresponding to each subtask in the processing task within the load analysis time period; The load condition determination module is specifically used to: Determine the serial task load duration of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed serially in the task processing period; Determine the equivalent duration of the parallel task load of the computing resource node within the load analysis time period based on the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks; determine the load condition of the computing resource node within the load analysis time period based on the serial task load duration and the equivalent duration of the parallel task load; The load condition determination module is further configured to: According to the task processing period corresponding to the subtasks processed in parallel in the task processing period, the number of subtasks processed in parallel in the same period is determined; the ratio of the number of subtasks processed in parallel in the same period to the preset maximum number of tasks processed in parallel is calculated; according to the ratio and the duration of the task processing period of the corresponding subtasks processed in parallel, the equivalent duration of the parallel task load of the computing resource node in the load analysis time period is obtained.

8. A computer device comprising a memory, a processor or a processing unit, and a computer program stored in the memory and executable on the processor or the processing unit, wherein: When the processor or processing unit executes the program, the task processing load analysis method according to any one of claims 1 to 6 is implemented.

9. A storage medium storing computer executable instructions, characterized in that: When the computer executable instructions are executed by a computer processor, they are used to perform the task processing load analysis method according to any one of claims 1 to 6.

Images