Task Processing Load Analysis Method and Device

Abstract

Embodiments of the present invention disclose a task processing load analysis method and apparatus. The method includes: in response to a load analysis request of a computing resource node, determining a load analysis time period and obtaining task information processed by the computing resource node; determining, according to task event marking information in the task information, task processing time periods corresponding to each subtask in the processing task during the load analysis time period; and determining the load condition of the computing resource node during the load analysis time period according to the load analysis time period and the task processing time periods corresponding to each subtask in the processing task during the load analysis time period. The technical solution of the embodiments of the present invention solves the problem that the current load analysis cannot accurately analyze complex dynamic task processing loads. It can calculate the load by analyzing the processing time periods of the subtasks of the processing task, calculate the actual load of the computing resource node more reasonably, improve the effectiveness of load analysis, and improve resource utilization efficiency.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of computing resource management, and in particular to a method and device for analyzing task processing load. Background Art

[0002] Traditional resource scheduling methods mainly rely on the utilization of CPU (Central Processing Unit) and GPU (Graphics Processing Unit) to estimate the load. This rough estimation method has obvious limitations when dealing with large-scale language model computing tasks. The computing tasks of large-scale language models are highly complex and dynamic. Complexity means that a task needs to be processed in multiple steps, rather than the traditional processing of the entire task. Dynamicity means that the computing power is always in a state of continuous computing and processing and receiving new tasks, rather than the traditional state of processing one by one or processing a batch at the same time. Therefore, traditional load analysis often cannot accurately reflect the actual computing load, resulting in uneven resource allocation and reduced utilization efficiency of computing resources. Summary of the invention

[0003] The embodiment of the present invention provides a task processing load analysis method and device, which can calculate the load by analyzing the processing period of the subtask of the processing task, more reasonably calculate the actual load of the computing resource node, improve the effectiveness of load analysis, and improve resource utilization efficiency.

[0004] In a first aspect, an embodiment of the present invention provides a task processing load analysis method, the method comprising:

[0005] In response to a load analysis request from a computing resource node, determining a load analysis time period;

[0006] Based on the load analysis time period, obtain the task information processed by the computing resource node;

[0007] Determine the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information;

[0008] The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0009] In a second aspect, an embodiment of the present invention provides a task processing load analysis device, the device comprising:

[0010] A load analysis time period determination module, used to determine a load analysis time period in response to a load analysis request of a computing resource node;

[0011] A task information acquisition module is used to obtain task information processed by computing resource nodes based on a load analysis time period;

[0012] The task processing period determination module is used to determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event marking information in the task information;

[0013] The load condition determination module is used to determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, the computer device comprising:

[0015] one or more processors;

[0016] A memory for storing one or more programs;

[0017] When the one or more programs are executed by one or more processors, the one or more processors implement the task processing load analysis method provided by any embodiment of the present invention.

[0018] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a task processing load analysis method as provided in any embodiment of the present invention.

[0019] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the task processing load analysis method provided by any embodiment of the present invention.

[0020] The embodiments of the above invention have the following advantages or beneficial effects:

[0021] The embodiment of the present invention determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; determines the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information; determines the load of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load, and can calculate the load by analyzing the processing time period of the subtask of the processing task, more reasonably calculate the actual load of the computing resource node, improve the effectiveness of the load analysis, and improve the efficiency of resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a task processing load analysis method provided by an embodiment of the present invention;

[0023] Figure 2 is a flow chart of a task processing load analysis method provided by an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a task processing provided by an embodiment of the present invention;

[0025] Figure 4 is a flow chart of a task processing load analysis method provided by an embodiment of the present invention;

[0026] Figure 5 is a flow chart of a task processing load analysis method provided by an embodiment of the present invention;

[0027] Figure 6 It is a structural schematic diagram of a task processing load analysis device provided by an embodiment of the present invention;

[0028] Figure 7 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0030] Figure 1The present invention provides a flowchart of a method for analyzing a task processing load. The present invention can be applied to a scenario where the load of task processing is analyzed. The method can be performed by a task processing load analysis device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0031] like Figure 1 As shown, the task processing load analysis method of this embodiment includes the following steps:

[0032] S110 . Determine a load analysis time period in response to a load analysis request from a computing resource node.

[0033] The computing resource node can be any node in a distributed computing system for executing computing tasks, and specifically can be a computing node including at least one or a combination of several of hardware computing resources, software computing resources, and network computing resources. In response to the load analysis request of the computing resource node, the load analysis time period is determined according to the request time of the load analysis request, and the load analysis request can be triggered based on the acquisition of a new task. Since in this embodiment, the processing of computing tasks has the characteristics of dynamics, the computing resource node is always in a state of continuous computing and processing and continuous reception of new tasks. Therefore, the preset time before the request time or the preset time after the request time or the preset time before and after can be taken as the load analysis time period. The preset time can be 30 seconds, one minute, five minutes, etc. This embodiment does not limit the length of the preset time.

[0034] S120. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0035] Based on the time interval corresponding to the load analysis time period, the task information within the time interval is obtained. The task information can be obtained from the log information, and the task information can include task event marking information generated by marking / reporting tasks during the task execution process.

[0036] S130 . Determine, according to the task event marking information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0037] A processing task can be composed of at least two subtasks. For example, the natural language processing task of a natural language model can include an input understanding and initialization subtask and a recursive reasoning and decoding output subtask. The input understanding and initialization subtask can first tokenize the user input, calculate the self-attention and generate a KV (key-value) cache, and then sample the output and return it to the customer; the recursive reasoning and decoding output subtask predicts and calculates the attention word by word, calculates each token, and then performs decoding conversion to obtain the model processing result.

[0038] Task event marking information may be marking information used to characterize the task processing status of each subtask in the processing task. Task time marking information may include time point information of task start processing, completion processing, suspension processing, and resumption of processing after suspension. Based on the above time point information, the intersection period of the load analysis time cycle and the processing time of each subtask is determined, and the intersection period is determined as the task processing period corresponding to each subtask in the processing task within the load analysis time cycle. It can be understood that the subtask processing time and the load analysis time cycle do not necessarily have an inclusive relationship. For example, if the subtask has started processing before the start time of the load analysis time cycle, or the subtask has completed processing after the end time of the load analysis time cycle, only the processing time within the load analysis time cycle is calculated.

[0039] S140, determining the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0040] According to the load analysis time cycle and the task processing time period corresponding to each subtask in the processing task within the load analysis time cycle, according to the characteristics of the task processing method of each subtask, the load duration of the task processing time period and the relationship between the load duration and the load analysis time cycle are determined, and according to the relationship between the equivalent duration of the task processing time period and the load analysis time cycle, the load situation of the computing resource node within the load analysis time cycle is determined. The processing method may include a serial processing method and a parallel processing method. The subtask of the serial processing method may be a subtask that cannot be processed simultaneously with the subtasks of other processing tasks, that is, only one subtask of a processing task can be processed at a time. The parallel processing method may refer to a subtask that is processed simultaneously with the subtasks of other processing tasks, that is, multiple subtasks of multiple processing tasks can be processed at a time.

[0041] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; determines the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information; determines the load of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load, and can calculate the load by analyzing the processing time period of the subtask of the processing task, more reasonably calculate the actual load of the computing resource node, improve the effectiveness of the load analysis, and improve the efficiency of resource utilization.

[0042] Figure 2 This is a flowchart of a task processing load analysis method provided by an embodiment of the present invention. This embodiment and the task processing load analysis method in the above embodiment belong to the same inventive concept, and further describes the process of determining the task processing period and determining the load condition according to the task processing period. The method can be executed by a task processing load analysis device, which can be implemented by software and / or hardware and integrated in a computer device with application development function.

[0043] like Figure 2 As shown, the task processing load analysis method of this embodiment includes the following steps:

[0044] S210. In response to a load analysis request from a computing resource node, determine a load analysis time period.

[0045] S220. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0046] S230: Determine the start and end time of each subtask according to the task event marking information in the task information.

[0047] Task event marker information in task information can be collected by dotting / reporting. For example, the general process of dotting / reporting is that when the behavior / status that needs to be collected occurs, it is recorded in the log. For example, the start and end time of each subtask of each processing task are all collected events. After dotting and collecting task event marker information, the log containing the task event marker information is reported at a selected time.

[0048] The specific way of marking can be any one of code marking, declaration marking, traceless marking and no marking. Code marking can be manually adding marking code in the specific business code to collect task event marking information. Declaration marking can be collecting task event marking information by adding event identifier and business field as attributes to the response control. Simplify the amount of code for code marking. Traceless marking can be to obtain all operations and then decide the task event marking information that needs to be reported. No marking can be to report all operations and let the server filter the task event marking information.

[0049] S240: Based on the start and end time of each subtask and the load analysis time period, determine the task processing time period corresponding to each subtask within the load analysis time period.

[0050] like Figure 3 As shown, Figure 3There are 4 processing tasks in it. Each processing task includes 2 subtasks. The 2 subtasks are prefile and decoding subtasks. Prefile can be the input understanding and initialization subtask, and decoding can be the recursive reasoning and decoding output subtask. Prefile is processed first and decoding is processed later. If a new task is obtained during the decoding process, decoding needs to be paused and the prefile of the new task is processed. When the prefile processing of the new task is completed, the paused decoding and the decoding of the new task are continued. Prefile is processed serially, while decoding can be processed in parallel.

[0051] Taking the subtask of processing task 1 as an example, the prefile of processing task 1 has been processed before the load analysis time period of 0 to 30 seconds. During the decoding process, the processing starts a few seconds later and enters the load analysis time period of 0 seconds, pauses at 4 seconds, restarts at 10 seconds, pauses again at 13 seconds, restarts at 20 seconds, and completes the processing at 24 seconds. The task processing period of decoding is the intersection of the processing time of the subtask decoding in the load analysis time period of 0 to 4 seconds, 10 to 13 seconds, and 20 to 24 seconds.

[0052] S250. Determine the serial task load duration of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed serially in the task processing period.

[0053] like Figure 3 As shown, the task processing period corresponding to the subtask profile of the serial processing of all processing tasks is 4 to 10 seconds, 13 to 20 seconds, and 28 to 30 seconds. The serial task load duration of the computing resource node within the load analysis time period is determined to be the duration of the task processing period, that is, the serial task load duration is 10-4=6s, 20-13=7s, and 30-28=2s.

[0054] S260. Determine the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks.

[0055] The equivalent duration of the parallel task load is the duration that can characterize the actual load of the task processing period corresponding to the subtasks processed in parallel. In the task processing period corresponding to the subtasks processed in parallel, there is a situation where the number of subtasks does not reach the preset maximum number of parallel processing tasks. Therefore, the actual load cannot be measured by the actual processing duration. Instead, the equivalent duration of the parallel task load of the computing resource node within the load analysis time period is calculated based on the numerical relationship between the number of subtasks processed in the task processing period and the preset maximum number of parallel processing tasks. The actual load of the subtasks processed in parallel is measured by the equivalent duration of the parallel task load. The preset maximum number of parallel processing tasks can be set according to actual needs, and this embodiment does not limit this.

[0056] In an optional implementation, determining the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks may include the following steps A1-A3:

[0057] Step A1: according to the task processing time periods corresponding to the subtasks processed in parallel in the task processing time period, determine the number of subtasks processed in parallel in the same time period.

[0058] like Figure 3 As shown, the task processing periods corresponding to the subtasks processed in parallel include: period one, period two, period three and period four, corresponding to 0 to 4 seconds, 10 to 13 seconds, 20 to 24 seconds and 24 to 28 seconds respectively, and the corresponding numbers of subtasks processed in parallel are 1, 2, 3 and 2 respectively.

[0059] Step A2: Calculate the ratio of the number of subtasks processed in parallel within the same period to the preset maximum number of tasks processed in parallel.

[0060] For example, the preset maximum number of parallel processing tasks is 3, and the ratio of the number of subtasks processed in parallel in the same time period to the preset maximum number of parallel processing tasks, that is, the ratios corresponding to time periods one to four are 1 / 3, 2 / 3, 1 and 2 / 3.

[0061] Step A3: According to the ratio value and the duration of the task processing period of the corresponding parallel processed subtask, the equivalent duration of the parallel task load of the computing resource node within the load analysis time period is obtained.

[0062] The equivalent duration of the parallel task load of the computing resource node within the load analysis time period is: period one: (4-0)*1 / 3=4 / 3s, period two: (13-10)*2 / 3=2s, period three: (24-20)*3 / 3=4s and period four: (28-24)*2 / 3=8 / 3s.

[0063] S270: Determine the load condition of the computing resource node within the load analysis time period according to the serial task load duration and the parallel task load equivalent duration.

[0064] Based on the calculated serial task load duration and parallel task load equivalent duration corresponding to the task processing period, the load condition of the computing resource node within the load analysis time period is calculated and determined.

[0065] In an optional implementation, the load condition of the computing resource node within the load analysis time period is determined based on the serial task load duration and the equivalent parallel task load duration. The serial task load duration and the equivalent parallel task load duration can be added to obtain the total load time; the ratio of the total load time to the load analysis time period is calculated, and the ratio is used as the load condition of the computing resource node within the load analysis time period.

[0066] For example, the cumulative value of the serial task load duration is 6+7+2=15s. The cumulative value of the parallel task load equivalent duration is 4 / 3+2+4+8 / 3=10s. The total load time is the sum of the load duration or the cumulative value of the load equivalent duration of the subtasks of the two processing methods, that is, the total load time = 15+10 = 25s. The load condition of the computing resource node during the load analysis time period, that is, the ratio of the total load time to the duration of the load analysis time period, is buzy=25 / 30=83%. Optionally, the difference between the ratio and 1, idle, and the ratio are used as the load condition of the computing resource node during the load analysis time period. idle=1-83%=17%. When busy is 0%, it means that the computing resource node is completely idle, and when busy is 100%, it means that the computing resource node cannot undertake more processing tasks. Idle indicates the idle status of the computing resource node. When busy is 0%, idle is 100%, and when busy is 20%, idle is 80%.

[0067] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; determines the start and end time of each subtask according to the task event marking information in the task information; determines the task processing time period corresponding to each subtask within the load analysis time period based on the start and end time of each subtask and the load analysis time period; determines the serial task load duration of the computing resource node within the load analysis time period according to the task processing time period corresponding to the subtask processed serially in the task processing time period; determines the parallel task load equivalent duration of the computing resource node within the load analysis time period according to the task processing time period corresponding to the subtask processed in parallel in the task processing time period and the preset maximum number of parallel processing tasks; determines the load condition of the computing resource node within the load analysis time period according to the serial task load duration and the parallel task load equivalent duration. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load. The load can be calculated by analyzing the processing period of the subtask of the processing task, and the actual load of the computing resource node can be more reasonably calculated according to the load duration or load equivalent duration corresponding to the serial and parallel processing, thereby improving the effectiveness of load analysis and improving resource utilization efficiency.

[0068] Figure 4 A flowchart of a task processing load analysis method provided in an embodiment of the present invention, this embodiment and the task processing load analysis method in the above embodiment belong to the same inventive concept, and further describes the process of determining the cluster load of the computing resource cluster corresponding to the computing resource node within the load analysis time period. The method can be executed by a task processing load analysis device, which can be implemented by software and / or hardware and integrated in a computer device with application development function.

[0069] like Figure 4 As shown, the task processing load analysis method of this embodiment includes the following steps:

[0070] S310. In response to a load analysis request from a computing resource node, determine a load analysis time period.

[0071] S320. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0072] S330 , determining the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event marking information in the task information.

[0073] S340, determining the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0074] S350. Determine the cluster load condition of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load condition of the computing resource node within the load analysis time period.

[0075] According to the load conditions of multiple computing resource nodes in the load analysis time period, the cluster load conditions of the computing resource cluster corresponding to the computing resource node in the load analysis time period are determined by calculating the average value. For example, the ratio corresponding to the load conditions of two computing resource nodes is added and then divided by 2 to obtain the ratio corresponding to the cluster load conditions of the computing resource cluster composed of the two computing resource nodes in the load analysis time period. For example, if the ratio corresponding to the load condition of one computing resource node is 10%, and the ratio corresponding to the load condition of another computing resource node is 20%, then the ratio corresponding to the cluster load condition of the computing resource cluster composed of the two computing resource nodes in the load analysis time period is (10%+20%) / 2=15%.

[0076] S360: Send the cluster load status to the task allocation center, so that the task allocation center schedules and processes tasks according to the cluster load status to achieve load balancing among computing resource clusters.

[0077] For each computing resource cluster, the cluster load status of the computing resource cluster is sent to the task allocation center, so that the task allocation center schedules and processes tasks according to the cluster load status to achieve load balancing and refined scheduling among computing resource clusters.

[0078] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; determines the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information; determines the load of the computing resource node within the load analysis time period according to the load analysis time period and the task processing period corresponding to each subtask in the processing task within the load analysis time period; determines the cluster load of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load of the computing resource node within the load analysis time period; sends the cluster load to the task allocation center so that the task allocation center schedules the processing task according to the cluster load to achieve load balancing between computing resource clusters. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load, can calculate the load by analyzing the processing period of the subtask of the processing task, more reasonably calculate the actual load of the computing resource node, and determine the actual load of the computing resource cluster through the load conditions of multiple computing resource nodes, improves the effectiveness of load analysis, and improves resource utilization efficiency.

[0079] Figure 5 A flowchart of a task processing load analysis method provided in an embodiment of the present invention, this embodiment and the task processing load analysis method in the above embodiment belong to the same inventive concept, and further describes the process of determining the load condition of a computing resource node within a load analysis time period according to the task attributes of a subtask. The method can be executed by a task processing load analysis device, which can be implemented by software and / or hardware and integrated into a computer device with application development function.

[0080] like Figure 5 As shown, the task processing load analysis method of this embodiment includes the following steps:

[0081] S410. Determine a load analysis time period in response to a load analysis request from a computing resource node.

[0082] S420. Based on the load analysis time period, obtain task information processed by the computing resource node.

[0083] S430. Obtain task attributes of at least one subtask according to task information processed by the computing resource node.

[0084] The subtask includes at least one subtask corresponding to each processing stage of the processing task. The processing task includes different processing stages, and each stage runs the corresponding model algorithm module to implement the corresponding model operation function. One processing stage corresponds to one subtask. If the processing task includes three processing stages, the processing task includes three subtasks corresponding to different processing stages.

[0085] The task attributes include serial and parallel processing attributes and / or processing priority attributes.

[0086] The serial-parallel processing attributes include serial processing attributes and parallel processing attributes. The serial processing attribute indicates that the subtask can only be processed individually, while the parallel processing attribute indicates that the subtask can be processed simultaneously with other subtasks that also have parallel processing attributes. Subtasks with parallel processing attributes can be subtasks of the same type or subtasks of different task types. The task type refers to the type of algorithm unit running the subtask model or the functional attributes of the task, such as natural language text input understanding and initialization, or recursive reasoning and decoding output.

[0087] The processing priority attribute may be a processing priority between subtasks of multiple different task types.

[0088] S440: Determine a task processing time period corresponding to each subtask in the processing task within a load analysis time period according to a task attribute of at least one subtask and task event marking information in the task information.

[0089] When reporting, the impact of the start or completion of a subtask on the start and end times of other subtasks being processed is determined based on the task attributes of the subtask. For example, the start of a high-priority subtask causes the suspension of other low-priority subtasks, or the end of a subtask causes the start of multiple subtasks to be processed in parallel. The start and end times of each subtask are determined based on the impact of the start of a subtask on other subtasks being processed. Optionally, corresponding events are triggered based on the start and end times of each subtask to generate task event marking information.

[0090] The task processing time period corresponding to each subtask in the processing task within the load analysis time period is determined according to the task event marking information.

[0091] As an example, the relationship between the task attributes of a subtask and its start and end time is described through the following situation.

[0092] Example 1: The processing task includes two subtasks corresponding to two processing stages, namely subtask A and subtask B. Subtask A is processed serially, while subtask B can be processed in parallel with subtask B of other processing tasks. The processing priority is to suspend the processing of subtask B when subtask A of a new processing task is obtained. That is, the processing priority of subtask A is greater than that of subtask B.

[0093] Example 2: The processing task includes three subtasks corresponding to the three processing stages, namely, subtask A, subtask B, and subtask C. Subtask A is processed serially, subtask B can be processed in parallel with at most one subtask B of other processing tasks, or in parallel with at most one subtask C of other processing tasks, subtask C can be processed in parallel with at most two subtask Cs of other processing tasks, or in parallel with at most one subtask B of other processing tasks.

[0094] When subtask A exists, subtask A must be processed first. When subtask A does not exist, subtask B must be processed first. When subtask B does not exist, subtask C must be processed.

[0095] The processing method can also be adjusted according to the number of subtasks. For example, when there are only two subtasks B, subtask B can be processed separately without serial processing. Similarly, the processing priority can also be adjusted according to the number of subtasks. For example, the processing priority can be 2 subtasks B>1 subtask B+1 subtask C>1 subtask B>subtask C. That is, when two subtasks B and one subtask C are obtained, the two subtasks B are processed in parallel first, and then one subtask C is processed.

[0096] S450, determining the load condition of the computing resource node within the load analysis time period according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask.

[0097] According to the task processing time period corresponding to each subtask with serial and parallel processing attributes within the load analysis time cycle, the serial load duration and parallel equivalent load duration of each subtask within the load analysis time cycle are calculated, and the total load time is obtained by adding the serial load duration and the parallel equivalent load duration. The load condition of the resource node within the load analysis time cycle is calculated, that is, the ratio of the total load time to the duration of the load analysis time cycle.

[0098] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; according to the task information processed by the computing resource node, obtains the task attribute of at least one subtask; according to the task attribute of at least one subtask and the task event tag information in the task information, determines the task processing time period corresponding to each subtask in the processing task within the load analysis time period; according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period and the task attribute of at least one subtask, determines the load situation of the computing resource node within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex subtask string parallel processing and the load analysis of complex situations according to priority processing, and can determine the processing time period calculation load of the subtask of the processing task by analyzing the task attribute of the subtask, more reasonably calculate the actual load of the computing resource node, improve the effectiveness of load analysis, and improve resource utilization efficiency.

[0099] Figure 6 The present invention provides a schematic diagram of a task processing load analysis device, which can be applied to the analysis of task processing load. The task processing load analysis device can be implemented by software and / or hardware and integrated into a computer terminal device with application development function.

[0100] like Figure 6 As shown, the task processing load analysis device includes: a load analysis time period determination module 510, a task information acquisition module 520, a task processing period determination module 530 and a load condition determination module 540.

[0101] Among them, the load analysis time period determination module 510 is used to determine the load analysis time period in response to the load analysis request of the computing resource node; the task information acquisition module 520 is used to obtain the task information processed by the computing resource node based on the load analysis time period; the task processing time period determination module 530 is used to determine the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event marking information in the task information; the load condition determination module 540 is used to determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0102] The technical solution of this embodiment determines the load analysis time period by responding to the load analysis request of the computing resource node; based on the load analysis time period, obtains the task information processed by the computing resource node; determines the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information; determines the load of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period. The technical solution of the embodiment of the present invention solves the problem that the current load analysis cannot accurately analyze the complex dynamic task processing load, and can calculate the load by analyzing the processing time period of the subtask of the processing task, more reasonably calculate the actual load of the computing resource node, improve the effectiveness of the load analysis, and improve the efficiency of resource utilization.

[0103] In an optional implementation, the task processing period determination module 530 is specifically configured to:

[0104] According to the task event marking information in the task information, the start and end time of each subtask is determined; based on the start and end time of each subtask and the load analysis time period, the task processing period corresponding to each subtask within the load analysis time period is determined.

[0105] In an optional implementation manner, the load condition determination module 540 is specifically configured to:

[0106] According to the task processing period corresponding to the subtasks processed serially in the task processing period, determine the serial task load duration of the computing resource node within the load analysis time period; according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks, determine the equivalent parallel task load duration of the computing resource node within the load analysis time period; according to the serial task load duration and the equivalent parallel task load duration, determine the load situation of the computing resource node within the load analysis time period.

[0107] In an optional implementation, the load condition determination module 540 is further configured to:

[0108] According to the task processing time periods corresponding to the subtasks processed in parallel in the task processing time period, the number of subtasks processed in parallel in the same time period is determined; the ratio of the number of subtasks processed in parallel in the same time period to the preset maximum number of tasks processed in parallel is calculated; according to the ratio and the duration of the task processing time period of the corresponding subtasks processed in parallel, the equivalent duration of the parallel task load of the computing resource node in the load analysis time period is obtained.

[0109] In an optional implementation, the load condition determination module 540 is further configured to:

[0110] The serial task load duration and the equivalent parallel task load duration are added together to obtain the total load time. The ratio of the total load time to the load analysis time period is calculated and used as the load condition of the computing resource node during the load analysis time period.

[0111] In an optional embodiment, the device further comprises:

[0112] The cluster load analysis module is used to determine the cluster load status of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load status of the computing resource node within the load analysis time period; and send the cluster load status to the task allocation center so that the task allocation center can schedule and process tasks according to the cluster load status to achieve load balancing among computing resource clusters.

[0113] In an optional implementation, the subtask includes at least one subtask corresponding to each processing stage of the processing task, and the apparatus further includes:

[0114] The task attribute acquisition module is used to acquire the task attribute of at least one subtask according to the task information processed by the computing resource node.

[0115] In an optional implementation, the load condition determination module 540 is further configured to:

[0116] The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask.

[0117] In an optional implementation, the task attributes include serial-parallel processing attributes and / or processing priority attributes, and the task processing period determination module 530 is further used to:

[0118] According to the task attribute of at least one subtask and the task event marking information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period is determined.

[0119] In an optional embodiment, the device further comprises:

[0120] The marking information generating module is used to trigger corresponding events based on the start and end time of each subtask and generate task event marking information.

[0121] The task processing load analysis device provided in the embodiment of the present invention can execute the task processing load analysis method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0122] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Figure 7 A block diagram of an exemplary computer device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 7 The computer device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention. The computer device 12 can be any terminal device with computing capabilities, such as an intelligent controller and server, a mobile phone and other terminal devices.

[0123] like Figure 7 As shown, the computer device 12 is in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0124] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MAC) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0125] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0126] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0127] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0128] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), one or more devices that enable a user to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with computer device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RFID systems, tape drives, and data backup storage systems.

[0129] The processing unit 16 executes various functional applications and data processing by running the program stored in the system memory 28, for example, implementing the task processing load analysis method provided in the embodiment of the present invention, the method comprising:

[0130] In response to a load analysis request from a computing resource node, determining a load analysis time period;

[0131] Based on the load analysis time period, obtain the task information processed by the computing resource node;

[0132] Determine the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information;

[0133] The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0134] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the method for analyzing the task processing load provided in any embodiment of the present invention is implemented. The method includes:

[0135] In response to a load analysis request from a computing resource node, determining a load analysis time period;

[0136] Based on the load analysis time period, obtain the task information processed by the computing resource node;

[0137] Determine the task processing time period corresponding to each subtask in the processing task within the load analysis time period according to the task event tag information in the task information;

[0138] The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period.

[0139] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to: an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0140] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0141] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0142] Computer program code for performing the operation of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, Python, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0143] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the task processing load analysis method provided in any embodiment of the present application.

[0144] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, Python, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0145] It should be understood by those skilled in the art that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented by a program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0146] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A task processing load analysis method, characterized in that: include: In response to a load analysis request from a computing resource node, determining a load analysis time period; Based on the load analysis time period, obtaining task information processed by the computing resource node; Determine, according to the task event marking information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period; Determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period; The subtask includes at least one subtask corresponding to each processing stage of the processing task, and the method further includes: Acquire task attributes of at least one subtask according to task information processed by the computing resource node; The determining, according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period, the load condition of the computing resource node within the load analysis time period comprises: The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask.

2. The method according to claim 1, characterized in that The determining, according to the task event marking information in the task information, a task processing period corresponding to each subtask in the processing task within the load analysis time period comprises: Determine the start and end time of each subtask according to the task event marking information in the task information; Based on the start and end time of each of the subtasks and the load analysis time period, a task processing time period corresponding to each of the subtasks within the load analysis time period is determined.

3. The method according to claim 2, characterized in that The determining, according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period, the load condition of the computing resource node within the load analysis time period comprises: Determine the serial task load duration of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed serially in the task processing period; Determine the equivalent duration of the parallel task load of the computing resource node within the load analysis time period according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks; The load condition of the computing resource node within the load analysis time period is determined according to the serial task load duration and the parallel task load equivalent duration.

4. The method according to claim 3, characterized in that The determining, according to the task processing period corresponding to the subtasks processed in parallel in the task processing period and the preset maximum number of parallel processing tasks, the equivalent duration of the parallel task load of the computing resource node in the load analysis time period comprises: Determine the number of subtasks to be processed in parallel within the same task processing period according to the task processing period corresponding to the subtasks to be processed in parallel within the task processing period; Calculate the ratio of the number of subtasks processed in parallel within the same period to the preset maximum number of tasks processed in parallel; The equivalent duration of the parallel task load of the computing resource node within the load analysis time period is obtained according to the ratio value and the duration of the task processing period of the corresponding parallel processed subtask.

5. The method according to claim 3, characterized in that: The determining, according to the serial task load duration and the parallel task load equivalent duration, the load condition of the computing resource node within the load analysis time period comprises: The serial task load duration and the parallel task load equivalent duration are added together to obtain a total load time; The ratio of the total load time to the length of the load analysis time period is calculated, and the ratio is used as the load condition of the computing resource node within the load analysis time period.

6. The method according to claim 1, characterized in that The method also includes: Determine the cluster load condition of the computing resource cluster corresponding to the computing resource node within the load analysis time period according to the load condition of the computing resource node within the load analysis time period; The cluster load condition is sent to a task allocation center so that the task allocation center schedules processing tasks according to the cluster load condition to achieve load balancing among computing resource clusters.

7. The method according to claim 1, characterized in that The task attributes include serial-parallel processing attributes and / or processing priority attributes, and determining, based on the task event tag information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period includes: According to the task attribute of the at least one subtask and the task event marking information in the task information, the task processing time period corresponding to each subtask in the processing task within the load analysis time period is determined.

8. The method according to claim 1, characterized in that: The method further includes: triggering corresponding events based on the start and end times of each subtask, and generating the task event marking information.

9. A task processing load analysis device, characterized in that: include: A load analysis time period determination module, used to determine a load analysis time period in response to a load analysis request of a computing resource node; A task information acquisition module, used to acquire task information processed by the computing resource node based on the load analysis time period; A task processing period determination module, used to determine the task processing period corresponding to each subtask in the processing task within the load analysis time period according to the task event marking information in the task information; A load condition determination module, used to determine the load condition of the computing resource node within the load analysis time period according to the load analysis time period and the task processing time period corresponding to each subtask in the processing task within the load analysis time period; The subtask includes at least one subtask corresponding to each processing stage of the processing task, and the device further includes: A task attribute acquisition module, used to acquire the task attribute of at least one subtask according to the task information processed by the computing resource node; The load condition determination module is further used for: The load condition of the computing resource node within the load analysis time period is determined according to the load analysis time period, the task processing time period corresponding to each subtask in the processing task within the load analysis time period, and the task attribute of at least one subtask.

Images