A resource processing method, device and equipment for computing tasks and a readable storage medium

Abstract

The application discloses a resource processing method, device and equipment of a computing task and a readable storage medium. The method comprises the following steps: obtaining running index information of a computing task in a first time period; the computing task comprises N subtasks running concurrently, and the running index information of the computing task comprises running index information of each subtask; based on the running index information of each subtask, the demand amount of computing resources of each subtask in a second time period is predicted and analyzed to obtain the predicted demand resource amount corresponding to each subtask; based on the predicted demand resource amount corresponding to the N subtasks respectively, M subtasks satisfying a resource adjustment condition are obtained from the N subtasks; and the M subtasks are adjusted in terms of resources in the second time period. According to the application, the computing resources of the computing task can be effectively adjusted without changing the concurrency of the computing task, and the task backlog can be reduced.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a resource processing method, apparatus, device, and readable storage medium for computing tasks. Background Technology

[0002] With the development of big data computing, computing platforms are being used in many application scenarios.

[0003] The computing platform runs various computing tasks, each with its own subtasks running concurrently. Since these subtasks differ, their required computing resources also vary. To improve the overall stability of the computing tasks, it's necessary to allocate computing resources reasonably among the subtasks. However, allocating these resources presents a challenge. If too few resources are allocated to a subtask, it may lead to instability, requiring manual adjustments by the user. Conversely, if too many resources are allocated, resource surplus can result in significant waste.

[0004] In traditional technologies, to achieve reasonable resource allocation, the computing resources of computing tasks are usually dynamically adjusted to promptly correct unreasonable resource allocation. Specifically, the computing resources of computing tasks are adjusted synchronously by adjusting the concurrency of computing tasks (which can be understood as adjusting the number of subtasks running in a computing task). However, this method of adjusting the concurrency of computing tasks requires restarting the entire computing task because it changes the concurrency of the computing task. This will cause interruption of data processing of the task, affect the execution of the computing task, and cause task backlog. Summary of the Invention

[0005] This application provides a resource processing method, apparatus, device, and readable storage medium for computing tasks, which can effectively adjust the computing resources of computing tasks and reduce task backlog without changing the concurrency of the computing tasks.

[0006] This application provides a resource processing method for computing tasks, including:

[0007] Obtain the performance metrics information of the computation task within the first time period; the computation task includes N concurrently running subtasks, and the performance metrics information of the computation task includes the performance metrics information of each subtask, where N is a positive integer;

[0008] Based on the operational metrics of each subtask, the computational resource requirements of each subtask in the second time period are predicted and analyzed to obtain the predicted resource requirements for each subtask; the second time period is later than the first time period.

[0009] Based on the predicted resource demand corresponding to each of the N subtasks, select M subtasks from the N subtasks that meet the resource adjustment conditions; M is an integer and M is less than or equal to N;

[0010] During the second time period, resources are adjusted for M subtasks.

[0011] One embodiment of this application provides a resource processing apparatus for computing tasks, including:

[0012] The information acquisition module is used to acquire the running indicator information of the computing task within the first time period. The computing task includes N concurrently running subtasks. The running indicator information of the computing task includes the running indicator information of each subtask, where N is a positive integer.

[0013] The prediction module is used to predict and analyze the computing resource requirements of each subtask in the second time period based on the operation index information of each subtask, and obtain the predicted resource requirements of each subtask; the second time period is later than the first time period.

[0014] The subtask selection module is used to select M subtasks that meet the resource adjustment conditions from the N subtasks based on the predicted resource demand corresponding to each of the N subtasks; M is an integer and M is less than or equal to N.

[0015] The resource adjustment module is used to adjust the resources of M subtasks during the second time period.

[0016] In one embodiment, the prediction module predicts and analyzes the computational resource requirements of each subtask during the second time period based on the operational metrics information of each subtask, and obtains a specific implementation method for the predicted resource requirements corresponding to each subtask, including:

[0017] Based on the operational metrics of each subtask, the resource prediction model is invoked;

[0018] The computational resource requirements of each subtask in the second time period are calculated and analyzed using the model parameters of the resource prediction model, and the predicted resource requirements for each subtask are output.

[0019] In one embodiment, the training process of the resource prediction model includes:

[0020] Obtain the sample operation index information of the sample task within the first sample time period; the sample task includes Q sample subtasks running concurrently, and the sample operation index information of the sample task includes the sample operation index information of each sample subtask, where Q is an integer greater than 1.

[0021] The initial resource prediction model is invoked based on the sample operation index information of each sample subtask;

[0022] The computational resource requirements of each sample subtask in the second sample time period are predicted and analyzed using the initial resource prediction model to obtain the sample predicted resource requirements of each sample subtask; the second sample time period is later than the first sample time period.

[0023] Based on the difference between the total amount of computing resources actually used by each sample subtask in the second sample time period and the corresponding sample predicted resource demand, the initial resource prediction model is trained to obtain the resource prediction model.

[0024] In one embodiment, the computing platform where the computing task resides is deployed with a rule monitoring component, which is used to monitor and record the resource adjustment rules defined by the rule definition object;

[0025] The resource adjustment conditions are determined by the rule monitoring component based on the monitored resource adjustment rules; when the rule monitoring component detects a change in the resource adjustment rules, it dynamically adjusts the resource adjustment conditions.

[0026] In one embodiment, the subtask selection module, based on the predicted resource demand corresponding to each of the N subtasks, selects M subtasks from the N subtasks that meet the resource adjustment conditions. The specific implementation methods include:

[0027] The predicted resource requirements of each of the N sub-tasks are summed to obtain the total predicted resource requirements of the N sub-tasks in the second time period.

[0028] Get the total amount of configured resources corresponding to the computing task; the total amount of configured resources is the total amount of computing resources configured for the computing task.

[0029] Based on the relationship between the total configured resources and the total predicted resource demand, select M subtasks from N subtasks that meet the resource adjustment conditions.

[0030] In one embodiment, the relationship between the total configured resources and the total predicted resource demand is such that the total predicted resource demand is greater than the total configured resources.

[0031] The subtask selection module, based on the total configured resources and the predicted total resource demand, selects M subtasks from N subtasks that meet the resource adjustment conditions, including:

[0032] Sort the N subtasks according to the task sorting rules to obtain the subtask sequence;

[0033] Calculate the total resource requirement for each subtask in the order of the N subtasks in the subtask sequence. The total resource requirement for any subtask is the sum of the predicted resource requirements of the subtask and its predecessor. In the subtask sequence, the predecessor of a subtask is located before the subtask.

[0034] The M subtasks in the subtask sequence whose total required resources are less than the total configured resources are identified as the M subtasks that meet the resource adjustment conditions.

[0035] In one embodiment, the relationship between the total configured resources and the total predicted resource demand is such that the total predicted resource demand is less than the total configured resources.

[0036] The subtask selection module, based on the total configured resources and the predicted total resource demand, selects M subtasks from N subtasks that meet the resource adjustment conditions, including:

[0037] Get the resource adjustment time period corresponding to each subtask; the resource adjustment time period corresponding to the subtask is the time period during which resource adjustments are allowed to be made pre-configured for the subtask;

[0038] The M subtasks that include the second time period in the corresponding resource adjustment period are identified as meeting the resource adjustment conditions.

[0039] In one embodiment, the specific implementation of the resource adjustment module in adjusting resources for M sub-tasks during the second time period includes:

[0040] Get the amount of computing resources used by each of the M subtasks in the first time period;

[0041] Based on the amount of resources used and the corresponding predicted resource demand of each of the M subtasks, determine the resource adjustment method for each of the M subtasks.

[0042] During the second time period, resource expansion processing will be performed on subtasks whose resource adjustment method is resource expansion; and,

[0043] Perform resource reduction processing on subtasks whose resource adjustment method is resource reduction.

[0044] In one embodiment, the resource adjustment module determines the specific implementation method of the resource adjustment method for each of the M subtasks based on the used resource amount and the corresponding predicted resource demand of each subtask, including:

[0045] Choose any one of the M subtasks as the target subtask;

[0046] If the amount of resources used in the target subtask is greater than the corresponding predicted resource demand, then the resource adjustment method for the target subtask is determined to be the resource reduction method.

[0047] If the amount of resources used in the target subtask is less than the corresponding predicted resource demand, then the resource adjustment method for the target subtask is determined to be the resource expansion method.

[0048] In one embodiment, the specific implementation method of the resource adjustment module for performing resource expansion processing on subtasks whose resource adjustment method is resource expansion includes:

[0049] The subtask that adjusts resources by expanding them is designated as the expansion subtask.

[0050] Send a first parameter increase instruction to the configuration management component of the expansion subtask; the first parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the container memory parameters and processor resource parameters of the expansion subtask.

[0051] Receive the first processing result notification returned by the configuration management component of the expansion subtask;

[0052] If the first processing result notification indicates that the container memory parameters of the expansion subtask have been successfully adjusted, a second parameter increase instruction for the heap memory parameters of the expansion subtask is sent to the configuration management component of the expansion subtask. The second parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the heap memory parameters of the expansion subtask.

[0053] In one embodiment, after the resource adjustment module sends an instruction to the configuration management component of the expansion subtask to increase the second parameter of the heap memory parameter of the expansion subtask, the resource adjustment module is further specifically used for:

[0054] Receive the second processing result notification returned by the configuration management component of the expansion subtask;

[0055] If the second processing result notification indicates that the adjustment of the heap memory parameters of the expansion subtask has failed, a first rollback instruction for the container memory parameters of the expansion subtask is sent to the configuration management component of the expansion subtask. The first rollback instruction is used to instruct the configuration management component of the expansion subtask to restore the container memory parameters and processor resource parameters of the expansion subtask.

[0056] In one embodiment, the specific implementation of the resource adjustment module for performing resource scaling-down processing on subtasks whose resource adjustment method is resource scaling-down includes:

[0057] The subtask that adjusts resources by reducing their size is designated as the reduction subtask.

[0058] Send a first parameter reduction instruction to the configuration management component of the shrinking subtask; the first parameter reduction instruction is used to instruct the configuration management component of the shrinking subtask to reduce the heap memory parameters and processor resource parameters of the shrinking subtask.

[0059] Receive the third processing result notification returned by the configuration management component of the scaling subtask;

[0060] If the third processing result notification indicates that the heap memory parameters of the shrinking subtask have been successfully adjusted, a second parameter reduction instruction for the container memory parameters of the shrinking subtask is sent to the configuration management component of the shrinking subtask. The second parameter reduction instruction is used to instruct the configuration management component of the shrinking subtask to reduce the container memory parameters of the shrinking subtask.

[0061] One embodiment of this application provides a computer device, including: a processor and a memory;

[0062] The memory stores a computer program, which, when executed by a processor, causes the processor to perform the methods described in the embodiments of this application.

[0063] One aspect of this application provides a computer-readable storage medium storing a computer program, which includes program instructions. When executed by a processor, the program instructions perform the methods described in this application.

[0064] One aspect of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in one aspect of the embodiments of this application.

[0065] This application provides a scheme for elastically scaling computing resources for computing tasks, which can effectively adjust the allocation of computing resources without changing the concurrency of computing tasks, thereby reducing the backlog of computing tasks. Specifically, for any computing task, its operating index information in a first time period can be obtained. Since the computing task concurrently runs with N (N is a positive integer) subtasks, this operating index information will include the operating index information of each subtask. After obtaining the operating index information of each subtask, this application can predict and analyze the computing resource requirements of each subtask in a second time period (later than the first time period) based on the operating index information of each subtask. In this way, the predicted resource requirement of each subtask in the second time period can be obtained. This predicted resource requirement is the total amount of computing resources required by each subtask in the second time period as predicted by this application. Thus, when the second time period arrives, based on the predicted resource requirement of each subtask, M subtasks that meet the resource adjustment conditions can be identified, and then resource adjustments (e.g., resource expansion or resource reduction) can be performed on these M subtasks. As can be seen, this application, by collecting operational metrics information of each subtask in an earlier time period (such as the first time period), can predict the total amount of computing resources required by each subtask in a later time period. Then, based on the predicted demand, resource adjustments (e.g., resource expansion or reduction) can be made to subtasks that meet the resource adjustment conditions. This achieves elastic scaling of resources for each subtask without changing the concurrency level of the computing tasks or restarting them, enabling zero-downtime elastic adjustment of computing task resources. This not only reduces the backlog of computing tasks caused by restarts but also effectively improves the efficiency of computing resource adjustment. In summary, this application can effectively adjust the computing resources of computing tasks and reduce task backlog without changing the concurrency level of the computing tasks. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 This is a schematic diagram of the architecture of a resource adjustment system provided by an exemplary embodiment of this application;

[0068] Figure 2 This is a flowchart illustrating a resource adjustment method for a computing task provided by an exemplary embodiment of this application;

[0069] Figure 3 This is a schematic diagram of a system logic architecture provided in an embodiment of this application;

[0070] Figure 4 This is a schematic diagram of a process for adjusting the resources of subtasks that meet the resource adjustment conditions, provided in an embodiment of this application.

[0071] Figure 5 This is a schematic diagram of the logical architecture of a resource adjustment process provided in an embodiment of this application;

[0072] Figure 6 This is a schematic diagram illustrating the effect of elastic scaling of memory resources provided in an embodiment of this application;

[0073] Figure 7 This is a schematic diagram illustrating the effect of elastic scaling of processor resources provided in an embodiment of this application;

[0074] Figure 8 This is a schematic diagram of the structure of a resource processing device for computing tasks provided in an embodiment of this application;

[0075] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0076] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0077] This application relates to technologies related to container orchestration systems. For ease of understanding, the following will focus on explaining related concepts such as container orchestration systems.

[0078] Container orchestration systems can schedule, scale, and manage containers. Specifically, a container orchestration system is an open-source system for managing containerized applications across multiple hosts on a cloud platform. Its goal is to make deploying containerized applications simple and efficient, providing a mechanism for application deployment, planning, updating, and maintenance. A container orchestration system can include a node cluster consisting of one or more (usually at least two) compute nodes. Each compute node in this cluster can be a node computer used to run containerized applications; that is, each compute node can provide services to containerized applications. Running a container orchestration system can be understood as running a node cluster.

[0079] Containers: Containers are a method of implementing operating system virtualization, which allows processes to run in an isolated environment.

[0080] In real-world applications, different business applications (which may include, but are not limited to, social applications, multimedia applications (such as short video applications), entertainment applications (such as game applications), educational applications, and live streaming applications) are deployed in container orchestration systems. A specific container within the orchestration system provides the corresponding services for each business application. These applications running in the container orchestration system can each run different computing tasks to process different data. Each computing task can run multiple subtasks concurrently. Based on this, the computing resources configured for the computing task (which can refer to the hardware or network resources required to execute the computing task, typically including central processing unit (CPU) computing power, memory resources, etc.) can be allocated to each concurrently running subtask to ensure the smooth execution of each subtask. However, since each subtask is different, the amount of computing resources required will also vary. In order to improve the overall operational stability of the computing task, it is necessary to reasonably allocate the computing resources of each subtask in the computing task. However, how to allocate computing resources for the computing task has become a problem. If too few resources are allocated to a certain subtask, it may cause the subtask to run unstablely, and the user needs to manually adjust the resource configuration. If too many resources are allocated to a certain subtask, the excess resources will lead to serious resource waste.

[0081] In traditional technologies, to achieve rational resource allocation, a common approach is to dynamically adjust the computing resources of computing tasks to promptly correct unreasonable resource allocations. Specifically, this involves adjusting the concurrency of computing tasks (which can be understood as adjusting the number of subtasks running within a single task) to synchronously adjust computing resources. However, this method of adjusting concurrency requires restarting the entire computing task, causing data processing interruptions, impacting task execution, and leading to task backlog. Therefore, the traditional method of adjusting computing task resource configuration by modifying concurrency—a method prone to interruptions—introduces numerous problems, not only affecting task execution and increasing task backlog but also reducing the efficiency of resource adjustment.

[0082] Based on this, in order to reduce the backlog of computing tasks and improve the efficiency of resource adjustment for computing tasks, this application provides a scheme for zero-interruption elastic scaling adjustment of computing resources for computing tasks. This scheme can effectively adjust the allocation of computing resources without changing the concurrency of computing tasks, thereby reducing the backlog of computing tasks and improving the efficiency of resource adjustment. The scheme for zero-interruption elastic scaling of computing resources for computing tasks involved in this application may include at least the following four consecutive steps: 1. For any computing task, multiple subtasks can run concurrently in the system. For each subtask, its running indicator information within a certain historical time period (referred to as the first time period) can be obtained. The running indicator information in this application can refer to data used to reflect the running performance of the system, device, or process when running computing tasks. It may include, but is not limited to: performance indicator information (such as processor running speed, memory utilization, etc.), response indicator information (such as response time, loading time, processing latency (such as backlog duration)), traffic indicator information (such as data production / consumption speed), process indicator information (such as CPU utilization, GC throughput (the running time of the application (such as the process) plus the running time of the application)). The ratio between the total garbage collection time and the total garbage collection time, for example, if the application runtime plus garbage collection time is 5, and the application runtime is 3, then the GC throughput can be 3 / 5), JVM heap (JVM heap is the memory space used to store programming objects (such as Java objects), almost all programming objects and arrays are stored in the JVM heap memory) memory utilization, etc.; then, each subtask can collect its runtime metrics information, and the runtime metrics information of each subtask can be combined to form the overall runtime metrics information of the computing task; 2. After collecting the runtime metrics information of each subtask, for any subtask, based on its runtime metrics information, it can be used for the second time period (which can refer to a time period later than the first time period, that is, the end position of the first time period is earlier than or equal to the start position of the second time period, and the second time period can be understood as a time period that has not yet arrived).For example, if the current time for obtaining operational indicator information is March 10th, the first historical time period can refer to the past week ending the day before March 10th (i.e., March 3rd-March 9th), and the second time period can refer to the future week starting the day after March 10th (i.e., March 11th-March 17th). The total amount of computing resources required (referred to as demanded computing resources) for each subtask (i.e., the subtask's demand for computing resources in the second time period) is predicted and analyzed to obtain a predicted resource demand for the subtask in the second time period. 3. Based on the predicted resource demand for each subtask, this application can pre-verify each subtask to determine whether it meets the resource adjustment conditions (this resource adjustment...). The resource adjustment conditions can be determined by user-defined resource adjustment rules, and finally, M (M is an integer, less than or equal to the total number of subtasks) subtasks that meet the resource adjustment conditions can be obtained; 4. For the M subtasks, resource adjustments can be made based on their predicted resource requirements (here, resource adjustment can refer to resource expansion or resource reduction of the subtasks). For example, based on the predicted resource requirements of each subtask, the subtasks that need resource expansion and the subtasks that need resource reduction can be determined. Then, the subtasks that need resource expansion can be expanded, and the subtasks that need resource reduction can be reduced. In this way, the elastic scaling adjustment of each subtask in the computing task can be realized.

[0083] Therefore, this application can predict the amount of computing resources required by each subtask in a later time period by collecting the operation index information of each subtask in an earlier time period (such as the first time period). Then, based on the predicted demand, resources can be adjusted for subtasks that meet the resource adjustment conditions (e.g., resource expansion or resource reduction). This can achieve elastic scaling of resources for each subtask without changing the concurrency of the computing tasks or restarting the computing tasks. It can achieve zero-interruption elastic adjustment of computing resources for computing tasks. This can not only reduce the backlog of computing tasks caused by restarts, but also effectively improve the efficiency of adjusting computing resources for computing tasks.

[0084] The solution for zero-disruption elastic scaling of computing resources for computing tasks provided in this application can be applied to any business application that runs in a container orchestration system. These applications may include, but are not limited to, short video push applications, game applications, browser applications, educational applications, and cloud storage applications. Specific examples of each application will not be provided here. In other words, the solution for zero-disruption elastic scaling of computing resources for computing tasks provided in this application can be applied to any application scenario, including but not limited to: short video push scenarios, game scenarios, browser search scenarios, educational scenarios, and cloud storage scenarios. For example, a game scenario and a cloud storage scenario are used as examples.

[0085] Cloud storage applications can provide users with various functions (such as creating folders, uploading files, previewing files, sharing to groups, moving files, electronic signatures, saving to favorites, unsaving to favorites, renaming files, accessing the management center, clearing expired files, etc.). Cloud storage scenarios refer to the scenarios in which users store relevant data in the cloud based on the various functions provided by cloud storage applications. Users can perform various operations to store data in a folder, share data in a folder to a group, preview data in a file, etc.

[0086] A game scenario can refer to a situation where a user runs a game application on a terminal device to play a game.

[0087] In summary, the solution provided in this application embodiment enables zero-interruption elastic scaling of computing resources for different computing tasks running by different business applications, and the resource adjustments of each sub-task are independent and do not interfere with each other, thus effectively improving business coverage to a certain extent (such as expanding applicable scenarios).

[0088] It should be noted that the above-mentioned application scenarios are merely examples and do not limit the application scenarios to which the solutions provided in the embodiments of this application are applicable.

[0089] Furthermore, the solution provided in this application embodiment can be executed by a computer device, which can refer to any computing node in a container orchestration system. Specifically, a computing node can include a terminal or a server, and the computer device can also include both a terminal and a server. To facilitate understanding of the solution provided in this application embodiment, the following is combined with… Figure 1 The resource adjustment system shown illustrates the application scenarios involved in the embodiments of this application; wherein, Figure 1 This is a schematic diagram of the architecture of a resource adjustment system provided by an exemplary embodiment of this application, such as... Figure 1As shown, the system includes a terminal 101, a server 102, and a container orchestration system 103; wherein:

[0090] 1) Terminal 101 may include the terminal device used by the user. Of course, depending on the application scenario and field in which this solution is applied, the terminal provided in this application embodiment may vary. Terminal devices may include, but are not limited to: smartphones (such as smartphones running the Android system, or smartphones running the Internetworking Operating System (IOS), tablet computers, portable personal computers, mobile Internet devices (MIDs), in-vehicle devices, head-mounted smart home devices, and smart voice interaction devices, etc. This application embodiment does not limit the type of terminal device, which is stated here.

[0091] 2) Server 102 can be a compute node in a container orchestration system, capable of interacting with terminals to receive business data or requests from the terminals, and creating containers for a specific business application running on the terminal within the container orchestration system, so that the container can provide corresponding computing services for the business application. Server 102 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0092] 3) The container orchestration system 103 may include one or more containers, each created by a compute node to provide corresponding services for a business application.

[0093] The terminal 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this. Furthermore, this application does not limit the number of terminals and servers; Figure 1 The number of terminals 101 and servers 102 is only a single example. In actual applications, multiple servers may be distributed in a distributed manner. This is just a brief explanation.

[0094] Based on the above-described solution and system architecture, the following points should also be noted:

[0095] ①The embodiments of this application mentioned above Figure 1The system shown is for the purpose of more clearly illustrating the technical solutions of the embodiments of this application and does not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will understand that with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems. For example, the above description uses the example of the execution subject "computer device" of this application including a terminal and a server, that is, the terminal and server jointly executing the solution provided by the embodiments of this application, to introduce one application scenario of this solution; it should be understood that in practical applications, the computer device can also be a terminal or a server, that is, it supports the terminal or server alone executing the solution provided by the embodiments of this application.

[0096] ② The data collection and processing in this application embodiment should strictly comply with the requirements of relevant laws and regulations. Obtaining personal information requires the knowledge or consent of the individual (or a legal basis for information acquisition), and subsequent data use and processing should be carried out within the scope of laws, regulations, and the authorization of the personal information subject. For example, when this application embodiment is applied to specific products or technologies, such as obtaining the user's currently playing media data, the user's permission or consent is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant regions.

[0097] Based on the scheme and application scenarios described above, this application proposes a more detailed method for adjusting the resources of computing tasks. The method for adjusting the resources of computing tasks proposed in this application will be described in detail below with reference to the accompanying drawings.

[0098] Please see Figure 2 , Figure 2 This is a flowchart illustrating a resource adjustment method for a computing task provided in an exemplary embodiment of this application. The flowchart can refer to the process provided in this embodiment regarding a scheme for elastically scaling and adjusting the computing resources of a computing task. This resource adjustment method for a computing task can be executed by a computer device in the aforementioned system; the resource adjustment method for a computing task can include at least the following steps S201-S204:

[0099] Step S201: Obtain the running indicator information of the computing task within the first time period; the computing task includes N concurrently running subtasks, and the running indicator information of the computing task includes the running indicator information of each subtask, where N is a positive integer.

[0100] In this application, a container orchestration system can schedule, scale, and manage containers. Specifically, the container orchestration system is used to manage containerized applications on multiple hosts in a cloud platform, aiming to make deploying containerized applications simple and efficient. It provides a mechanism for application deployment, planning, updating, and maintenance. Here, a computing task can refer to a task run by an application, and a subtask can be understood as the various task processes that run concurrently within that computing task. A computing task can have one or more processes running concurrently, and therefore, a computing task can have one or more subtasks running concurrently. This example illustrates the scenario where a computing task runs concurrently with N (N is a positive integer) subtasks.

[0101] The first time period can refer to a certain historical period in the past (e.g., the past week, month, etc.). This application can obtain the operation index information of each subtask in the computing task within this first time period. In this application, the operational metrics information can refer to data reflecting the operational performance of a system, device, or process when running computational tasks. This data may include, but is not limited to: performance metrics (such as processor speed, memory usage, etc.), response metrics (such as response time, loading time, processing latency (e.g., backlog duration)), traffic metrics (such as data production / consumption speed), process metrics (such as CPU utilization, GC throughput (the ratio between the application's (e.g., process's) runtime and the sum of the application's runtime and garbage collection time; for example, if the application's runtime plus garbage collection time is 5 and the application's runtime is 3, then the GC throughput can be 3 / 5), JVM heap (the JVM heap is the memory space used to store programming objects (such as Java objects; almost all programming objects and arrays are stored in the JVM heap memory), memory utilization, etc.). Each subtask can collect its operational metrics information, and the operational metrics information of each subtask can be called operational metrics information. The operational metrics information of each subtask can collectively form the overall operational metrics information of the computational task. In other words, this application can obtain the operational indicator information of N subtasks in the first time period, and the operational indicator information of all subtasks can be combined to form the operational indicator information of the computation task in the first time period.

[0102] Step S202: Based on the operational metrics information of each subtask, predict and analyze the computational resource requirements of each subtask in the second time period to obtain the predicted resource requirements for each subtask; the second time period is later than the first time period.

[0103] In this application, the second time period can be understood as a time period later than the first time period, and can be understood as a time period that has not yet arrived. For example, if the current time for obtaining the operational indicator information is March 10th, the first historical time period can refer to the past week ending the day before March 10th (i.e., March 3rd-March 9th), while the second time period can refer to the future week starting the day after March 10th (i.e., March 11th-March 17th). After obtaining the operational indicator information of each subtask, a resource prediction model can be invoked to calculate and analyze the operational indicator information of each subtask using the model parameters, outputting the computational resource requirements of each subtask within this second time period. The computational resource requirements of the subtask output by the model within the second time period can be referred to in this application as the predicted resource requirement quantity corresponding to the subtask. In other words, the predicted resource requirement quantity of any subtask within the second time period in this application can be obtained by invoking the resource prediction model to predict and analyze its operational indicator information.

[0104] Specifically, for any subtask, its operational indicator information can be input into the resource prediction model. Then, the resource prediction model can use the trained model parameters to calculate and analyze this operational indicator information, and finally output a predicted resource demand for the subtask in the second time period. In order to improve the accuracy of the results output by the resource prediction model, this application can pre-train the model. The training process can include, but is not limited to, the following: First, a certain computational task can be selected as a sample for model training. This computational task can be called a sample task. Similarly, the sample task can also run concurrently with multiple subtasks. Here, we take the example of the sample task running concurrently with Q (Q is an integer greater than 1) subtasks, and each subtask can be called a sample subtask. Then, this application can obtain the operational indicator information of each sample subtask within a certain historical time period. This historical time period can be called the first sample time period. This first sample time period can be earlier than the aforementioned first time period or the same time period as the first time period. The operational indicator information of the sample subtasks obtained within the first sample time period can be called the sample operational indicator information of the sample subtasks. The sample operational indicator information of each sample subtask can be combined to form the sample operational indicator information of the sample task within this first sample time period.

[0105] After obtaining the sample operation index information for each sample subtask, the untrained resource prediction model (referred to as the initial resource prediction model) can be invoked. For any sample subtask, the sample operation index information of the sample subtask can be calculated and analyzed using the model parameters of this initial resource prediction model (referred to as the initial model parameters). Ultimately, the predicted value of the computational resource requirement for the sample subtask during the second sample time period (later than the first sample time period; if the first sample time period is earlier than the first time period, the second sample time period can be earlier than or equal to the first time period; if the first sample time period is the second time period, the second sample time period can be later than the first time period) can be output. This can be referred to as the sample predicted resource requirement for each sample subtask. In other words, the initial resource prediction model can be invoked based on the sample operation index information of each sample subtask. The initial resource prediction model can be used to predict and analyze the computational resource requirement of each sample subtask in the second sample time period to obtain the sample predicted resource requirement for each sample subtask. Then, the total amount of computational resources actually used by each sample subtask in the second sample time period (which can be referred to as the actual used resource amount) can be obtained. Based on the difference between the actual used resource amount of each sample subtask in the second sample time period and the corresponding sample predicted resource requirement, the initial resource prediction model can be trained to obtain the resource prediction model.

[0106] For example, a loss function (such as the cross-entropy loss function) can be used to calculate the error loss between the actual resource usage of each sample subtask in the second sample time period and the corresponding sample predicted resource demand. Then, the error losses corresponding to all sample tasks can be summed to obtain the total error loss. Based on this total error loss, the initial model parameters of the initial resource prediction model can be trained and optimized to finally obtain a resource prediction model that includes the trained and optimized model parameters.

[0107] It is worth noting that the resource prediction model in this application can be any artificial intelligence model with classification prediction function. The predicted value output by the model in this application can actually be understood as a predicted amount of resources. That is, this application can predict the amount of computing resources required by the computing task in a certain period of time based on the operation index information of the computing task in an earlier period of time.

[0108] Step S203: Based on the predicted resource demand corresponding to each of the N subtasks, obtain M subtasks from the N subtasks that meet the resource adjustment conditions; M is an integer and M is less than or equal to N.

[0109] In this application, after calculating the predicted resource requirements for each subtask in the computation task, resource adjustments can be made to each subtask based on these predicted resource requirements. However, in this application, the user can define a custom resource adjustment rule (which can be understood as a user-defined rule for elastically scaling the computational resources of the computation task, or a user-defined elastic rule). Thus, when adjusting resources for the computation task, this application can first determine a resource adjustment condition based on the user-defined resource adjustment rule, and then pre-verify each subtask based on this condition to determine whether it meets the resource adjustment condition. For subtasks that meet the resource adjustment condition, resource adjustment processing can be performed on them in a second time period.

[0110] Specifically, this application can support users in deploying rich elastic scaling adjustment strategies (or rules). This application can deploy a rule monitoring component in the computing platform or container orchestration system where the computing task resides. This rule monitoring component can be used to monitor and record in real time the resource adjustment rules defined by the rule definition object (such as a user, for example: the application maintenance object of the business application corresponding to the computing task). Based on the resource adjustment rules, the resource adjustment conditions can be determined. The rule definition object can modify and update the defined resource adjustment rules. Once the resource adjustment rule changes, the rule monitoring component can detect the change, dynamically load the new resource adjustment rule, and dynamically adjust the resource adjustment conditions based on the new resource adjustment rule to obtain the new resource adjustment conditions.

[0111] In summary, after determining the predicted resource requirements of each subtask in the second time period, the subtasks can be pre-verified according to the resource adjustment conditions indicated by the current resource adjustment rules to identify M (an integer less than or equal to N) subtasks that meet the resource adjustment conditions. The specific implementation process for obtaining the M subtasks that meet the resource adjustment conditions from the N subtasks based on their predicted resource requirements may include, but is not limited to: First, summing the predicted resource requirements of the N subtasks to obtain the total predicted resource requirements of the N subtasks in the second time period (i.e., the total resource requirements of all subtasks); then, obtaining the total configured resource amount corresponding to the computation task, which is the total amount of computing resources configured for the computation task; based on the relationship between the total configured resource amount and the total predicted resource requirements, the M subtasks that meet the resource adjustment conditions can be obtained from the N subtasks.

[0112] For example, if the relationship between the total configured resources and the total predicted resource demand is that the total predicted resource demand is greater than the total configured resources, it can be understood that the total computing resources required by all subtasks in the second time period will be greater than the total configured resources. In this case, the total configured resources cannot meet the needs of all subtasks. Therefore, this application can sort these N subtasks according to the task sorting rules, and further determine which subtasks can meet the resource adjustment conditions based on the sorting results. Specifically, this application can obtain task sorting rules, which can refer to the rules configured by this application for sorting subtasks of a computing task. The specific rules can be set by the user based on actual needs. For example, the task sorting rules can be set as random sorting rules, sorting rules based on task priority (task priority can be the execution priority of the computing task; higher priority computing tasks will be executed earlier. It should be understood that the number of computing tasks in this application may not be more than one; one or more computing tasks will run in the computing platform, and each computing task will have a corresponding task priority. For example, computing tasks with higher execution urgency will have higher priority. Subtasks belonging to the same computing task have the same task priority. When sorting subtasks of the same computing task, it is not necessary to sort them according to task priority; however, if it is necessary to sort subtasks of different computing tasks, then they can be sorted according to the task priority of the subtasks of different computing tasks), and so on.

[0113] Furthermore, after obtaining the task sorting rules, the N subtasks can be sorted to obtain a subtask sequence. For example, the N subtasks can be sorted according to a random sorting rule to obtain a subtask sequence. According to the arrangement order of the N subtasks in the subtask sequence, the total resource requirement corresponding to each subtask can be calculated sequentially. The total resource requirement corresponding to each subtask can refer to the sum of the predicted resource requirements of this subtask and all the subtasks preceding it in the sequence. That is, the total resource requirement corresponding to any subtask refers to the sum of the predicted resource requirements of the subtask and its predecessor subtask (in the subtask sequence, the predecessor subtask is located before the subtask).

[0114] For example, consider N subtasks including subtask a, subtask b, subtask c, and subtask d. Assume subtask a has a priority of 8, subtask b has a priority of 4, subtask c has a priority of 2, and subtask d has a priority of 9. Then, by sorting subtasks a, b, c, and d in descending order of priority, we obtain the subtask sequence {subtask d, subtask a, subtask b, subtask c}. Assume the predicted resource requirement for subtask a is 10, for subtask b it is 5, for subtask c it is 6, and for subtask d it is 9. If the predicted resource requirement is 8, then the total resource requirement for subtask a is 18 (predicted resource requirement of subtask d = 8 + predicted resource requirement of subtask a = 10); the total resource requirement for subtask b is 23 (predicted resource requirement of subtask d = 8 + predicted resource requirement of subtask a = 10 + predicted resource requirement of subtask b = 5); the total resource requirement for subtask c is 29 (predicted resource requirement of subtask d = 8 + predicted resource requirement of subtask a = 10 + predicted resource requirement of subtask b = 5 + predicted resource requirement of subtask c = 6); and the total resource requirement for subtask d is 8 (predicted resource requirement of subtask d = 8).

[0115] Furthermore, after calculating and determining the total resource requirement of each subtask in the subtask sequence, the M subtasks in the subtask sequence whose corresponding total resource requirement is less than the total configured resource amount can be identified as the M subtasks that meet the resource adjustment conditions.

[0116] It can be seen that the resource adjustment rule in this application can be a rule based on the relationship between the total configured resources and the total predicted resource demand. When the total configured resources are less than the total predicted resource demand, the resource adjustment rule can refer to a rule that adjusts resources sequentially according to the task priority of each subtask. Tasks with higher priority can be allocated corresponding computing resources based on their predicted resource demand. The total configured resources allocated to computing tasks are fixed. As computing resources are allocated to each subtask sequentially, the computing resources available to later subtasks will decrease. If, when adjusting resources for a subtask, it is found that the remaining resources allocated to the computing task are insufficient to meet the predicted demand of that subtask, then it can be considered that the subtask does not meet the conditions for resource adjustment and no resource adjustment processing is required for this subtask. Based on this, the resource adjustment condition in this application can refer to the condition that the total value of the required resources for the corresponding subtask should be less than or equal to the total configured resources. In summary, since the total amount of resources allocated to computing tasks in the above methods is fixed, as computing resources are allocated to each subtask sequentially, the later subtasks will receive fewer resources. Furthermore, the total amount of computing resources allocated to all computing tasks across the entire computing platform may also be fixed. When resource adjustments are needed for different computing tasks, the computing tasks can be sorted according to their priority from highest to lowest. Higher-ranked computing tasks have higher priority, and their concurrently running subtasks also have higher priority. These subtasks can then be allocated resources more preferentially, i.e., resource adjustments should be prioritized for them. In other words, when there are multiple computing tasks, each including concurrently running subtasks, resource adjustments can be prioritized for the subtasks of the higher-priority computing tasks.

[0117] The above describes a method for obtaining M subtasks that meet the resource adjustment conditions from N subtasks according to resource adjustment rules when the total configured resources are less than the total predicted resource demand. When the relationship between the total configured resources and the total predicted resource demand is that the total predicted resource demand is less than the total configured resources (i.e., the total configured resources are greater than the total predicted resource demand), the resource adjustment rule can be another rule. For example, this application can pre-configure resource adjustment time periods for each subtask. In this case, the resource adjustment rule can refer to the rule of performing resource adjustment processing according to the resource adjustment time period of each subtask. Only when the second time period falls within the resource adjustment time period of each subtask can the resource adjustment processing be performed on the subtask in the second time period. In other words, when the relationship between the total configured resources and the total predicted resource demand is that the total predicted resource demand is less than the total configured resources, the specific implementation method for obtaining M subtasks that meet the resource adjustment conditions from N subtasks based on the total configured resources and the total predicted resource demand may include, but is not limited to: obtaining the resource adjustment time period corresponding to each subtask; wherein the resource adjustment time period corresponding to the subtask is a pre-configured time period for allowing resource adjustments for the subtask; and then, determining the M subtasks whose corresponding resource adjustment time period includes a second time period as the M subtasks that meet the resource adjustment conditions.

[0118] For example, suppose the second time period is November 10th to November 17th, and the resource adjustment period for a certain subtask is the entire month of November. Therefore, the resource adjustment period for this subtask includes the second time period, and thus the subtask meets the resource adjustment criteria. It can be seen that the resource adjustment criteria here refer to the requirement that the resource adjustment period corresponding to the task should include the second time period.

[0119] It is worth noting that the resource adjustment rules described above are merely illustrative examples for ease of understanding and do not imply that the resource adjustment rules must be the two rules mentioned above. The resource adjustment rules in this application can be set and modified by the user based on actual business needs. For example, in a feasible scenario, the resource adjustment rules can also refer to a combination of the two rules mentioned above. That is, firstly, according to the task priority of each subtask, multiple subtasks whose corresponding total required resource value is less than or equal to the total configured resource value can be selected as candidate subtasks. Then, from these selected candidate subtasks, it can be determined whether the resource adjustment time period of each candidate subtask includes a second time period. For candidate subtasks that include a second time period, they can be used as the final M subtasks that meet the resource adjustment conditions. In other words, the resource adjustment rules in this application are not fixed, and correspondingly, the resource adjustment conditions will also change with the changes in the resource adjustment rules. It is also worth noting that the various time periods illustrated above (e.g., November 10-17, the entire month of November, etc.) are merely illustrative examples for ease of understanding and do not have practical reference value. In practice, more granular time periods can be used. For example, for resource adjustments to computing tasks, resource adjustments can be performed daily. In this case, the historical time period (e.g., the first time period) can be limited to the past week, while the predicted time period (e.g., the second time period) can be selected as a time period within a day (point a-point b). This allows for multiple predictions and adjustments of computing resources throughout the day, resulting in greater real-time performance and accuracy.

[0120] Step S204: Adjust resources for M subtasks during the second time period.

[0121] In this application, after identifying M subtasks that meet the resource adjustment conditions, resource adjustments can be made to the M subtasks within a second time period. The resource adjustments in this application may include resource expansion or resource reduction. Whether a specific subtask undergoes resource expansion or reduction can be determined based on its corresponding predicted resource demand.

[0122] In summary, this application can monitor the historical performance information of each subtask in a computing task in real time, thereby calculating a better resource allocation based on the monitored performance information. Then, based on the better resource allocation calculated, the resource of each subtask can be elastically scaled up and down (expanding the resources of subtasks that need to be expanded and shrinking the resources of subtasks that need to be shrunk). The entire resource adjustment process is achieved without interruption. Compared with the traditional method of restarting computing tasks, which causes interruption and affects the processing efficiency of computing tasks, resulting in task backlog, this application can improve the resource adjustment efficiency of computing tasks and reduce task backlog.

[0123] It should be noted that when adjusting resources for each subtask in this application, the memory resources (specifically including container memory parameters and JVM heap memory parameters) and CPU resources (specifically including processor parameters) can be adjusted. Therefore, the total amount of computing resources configured in this application (i.e., the total configured resources) can include the total amount of memory resources (which may include the maximum value of container memory parameters and the maximum value of JVM heap memory parameters; this total amount of memory resources can be referred to as the configured memory resource amount) and the total amount of CPU resources (which may include the maximum value of processor parameters; this total amount of CPU resources can be referred to as the configured processor resource amount). Correspondingly, the predicted resource demand can include the predicted total amount of memory resources (which may include the predicted container memory parameters and the predicted JVM heap memory parameters; this predicted total amount of memory resources can be referred to as the predicted memory resource demand amount) and the predicted total amount of CPU resources (which may include the predicted processor parameters; this predicted total amount of CPU resources can be referred to as the predicted processor resource demand amount).

[0124] This application provides a scheme for elastically scaling computing resources for computing tasks, which can effectively adjust the allocation of computing resources without changing the concurrency of computing tasks, thereby reducing the backlog of computing tasks. Specifically, for any computing task, its operating index information in a first time period can be obtained. Since the computing task concurrently runs with N (N is a positive integer) subtasks, this operating index information will include the operating index information of each subtask. After obtaining the operating index information of each subtask, this application can predict and analyze the computing resource requirements of each subtask in a second time period (later than the first time period) based on the operating index information of each subtask. In this way, the predicted resource requirement of each subtask in the second time period can be obtained. This predicted resource requirement is the total amount of computing resources required by each subtask in the second time period as predicted by this application. Thus, when the second time period arrives, based on the predicted resource requirement of each subtask, M subtasks that meet the resource adjustment conditions can be identified, and then resource adjustments (e.g., resource expansion or resource reduction) can be performed on these M subtasks. As can be seen, by collecting the operational metrics information of each subtask in an earlier time period (such as the first time period), this application can predict the amount of computing resources required by each subtask in a later time period. Then, based on the predicted demand, resources can be adjusted for subtasks that meet the resource adjustment conditions (e.g., resource expansion or resource reduction). This achieves elastic scaling of resources for each subtask without changing the concurrency of the computing tasks or restarting them. It enables zero-interruption elastic adjustment of computing resources, which not only reduces the backlog of computing tasks caused by restarts but also effectively improves the efficiency of adjusting computing resources.

[0125] To better understand the overall logic of this solution, please refer to [link / reference]. Figure 3 , Figure 3 This is a schematic diagram of a system logical architecture provided in an embodiment of this application, such as... Figure 3 As shown, this logical architecture may include at least the following components: interface opening component, indicator reporting component, resource prediction component, action execution component, rule monitoring component, and resource reuse component. For ease of understanding, the functions implemented by each component will be briefly described below:

[0126] Interface Opening Component: This component exposes the interface of the container orchestration system, enabling dynamic adjustment of computing resources for computational tasks. It may include a task management component and multiple subtask management components. The task management component manages the data configuration of the entire computational task, while each subtask management component corresponds to a subtask within that task, managing its data configuration. Data configuration may include settings for various resource parameters of the computational task (e.g., container memory parameters, processor resource parameters, heap memory parameters, etc.).

[0127] Metric Reporting Component: The metric reporting component can be used to collect historical metric information for each subtask (e.g., the operational metric information of each subtask within the first time period). This metric reporting component may include a software development kit (SDK) for a programming language, such as a Java SDK; it may also include a metric data engine (metriAgent).

[0128] Resource Prediction Component: This component uses collected historical metrics information to call a resource prediction model to predict resource requirements for each subtask within a specific time period. The resource prediction component includes a metric monitoring subcomponent, a resource prediction model, and a model output transmission subcomponent. The metric monitoring subcomponent monitors the metric information collected by the metric reporting component for each subtask in real time and inputs it into the resource prediction model. The resource prediction model then uses this metric information to predict resource requirements for each subtask within a specific time period. The model output transmission subcomponent receives the model's output and transmits it to the action execution component.

[0129] Rule monitoring component: The rule monitoring component can be used to monitor custom resource adjustment rules in real time.

[0130] Action Execution Component: The action execution component is used to pre-verify each subtask according to the resource adjustment rules monitored by the rule monitoring component and the predicted resource requirements of each subtask, to determine the subtasks that meet the resource adjustment conditions and trigger the execution of resource adjustment actions. The action execution component may include a pre-verification subcomponent, used to pre-verify each subtask according to the resource adjustment rules monitored by the rule monitoring component and the predicted resource requirements of each subtask, to determine the subtasks that meet the resource adjustment conditions; it may also include a resource setting subcomponent, used to transmit the predicted resource requirements of subtasks that meet the resource adjustment conditions to the corresponding subtask management components. Specifically, the resource setting subcomponent can send the corresponding predicted resource requirements to its corresponding task management component and its corresponding subtask management component. After receiving the predicted resource requirements, each subtask management component can update the resource configuration of the subtask to the corresponding predicted resource requirements, thereby realizing dynamic updating and adjustment of resource configuration. After the action execution component adjusts the resources of each subtask, it can return the resource adjustment data of the relevant subtasks to the rule monitoring component, which can then store the resource adjustment behavior in the logbook.

[0131] Resource reuse component: After the resource adjustments for each subtask are completed, the results of the resource adjustments can be synchronized to the resource reuse component to realize the reuse of resources in the container orchestration system.

[0132] This application can monitor the historical performance information of each subtask in a computing task in real time, thereby calculating a better resource allocation based on the monitored performance information. Then, based on the better resource allocation calculated, the resource of each subtask can be elastically scaled up and down (expanding the resources of subtasks that need to be expanded and shrinking the resources of subtasks that need to be shrunk). The entire resource adjustment process is achieved without interruption. Compared with the traditional method of restarting computing tasks, which causes interruption and affects the processing efficiency of computing tasks, resulting in task backlog, this application can improve the resource adjustment efficiency of computing tasks and reduce task backlog.

[0133] Furthermore, to facilitate understanding of the specific process of resource adjustment for subtasks that meet the resource adjustment conditions, please also refer to... Figure 4 , Figure 4 This is a schematic flowchart illustrating resource adjustment processing for subtasks that meet resource adjustment conditions, provided in an embodiment of this application. This process can correspond to the above-mentioned... Figure 2 In the corresponding embodiment, the process involves adjusting resources for M subtasks during the second time period. For example... Figure 4 As shown, the process may include at least the following steps S401-S404:

[0134] Step S401: Obtain the amount of resources used by each of the M subtasks in the first time period.

[0135] Specifically, the amount of resources used can refer to the total amount of computing resources used by a subtask in the first time period. This application can count the amount of computing resources used by each subtask in the first time period, which is called the amount of resources used.

[0136] Step S402: Based on the amount of resources used and the corresponding predicted resource demand of each of the M subtasks, determine the resource adjustment method for each of the M subtasks.

[0137] Specifically, after obtaining the used resource amount for each subtask, for any given subtask, its resource adjustment method in the second time period can be determined based on its used resource amount in the first time period and the predicted resource demand in the second time period. This resource adjustment method can include resource expansion or resource reduction. In other words, a subtask can undergo resource expansion or resource reduction processing in the second time period. Specifically, taking any one of the M subtasks as an example, the specific process of determining the resource adjustment method for each of the M subtasks based on the used resources and the corresponding predicted resource demand of each subtask may include, but is not limited to, the following: First, any one of these subtasks can be identified as the target subtask. Then, the used resources of this target subtask can be compared with the predicted resource demand. If the used resources of the target subtask are greater than the corresponding predicted resource demand, the resource adjustment method for the target subtask can be determined to be resource reduction. If the used resources of the target subtask are less than the corresponding predicted resource demand, the resource adjustment method for the target subtask can be determined to be resource expansion.

[0138] It is worth noting that the resource adjustment method in this application may also include the resource retention method. When the amount of resources used by a subtask in the first time period is equal to the predicted resource demand, its resource adjustment method can be determined as the resource retention method. However, when the resource adjustment method of a subtask is the resource retention method, no resource adjustment processing is required. Therefore, it can be considered that a subtask whose resource adjustment method is the resource retention method is a subtask that does not require resource adjustment processing.

[0139] Step S403: During the second time period, perform resource expansion processing on subtasks whose resource adjustment method is resource expansion.

[0140] Specifically, after determining the resource adjustment method for each subtask, the subtasks whose resource adjustment method is resource expansion can be processed for resource expansion. The specific implementation process may include, but is not limited to: identifying the subtasks whose resource adjustment method is resource expansion as expansion subtasks; and then, providing configuration management components for the expansion subtasks (as described above)... Figure 3 The subtask management component shown sends a first parameter increase instruction; this first parameter increase instruction instructs the configuration management component of the expansion subtask to increase the container memory parameters and processor resource parameters of the expansion subtask. The container memory parameter can refer to the memory parameters of the container providing services for this expansion subtask, and increasing it will increase the container memory resources; the processor resource parameter can refer to the CPU resource parameters of the container providing services for this expansion subtask, and increasing it will increase its CPU resources; the configuration management component corresponding to the expansion subtask can adjust the container memory parameters and processor resource parameters accordingly based on the first parameter increase instruction, and return a processing result notification message (which can be called the first processing result notification) to the computer device. This notification message is used to indicate whether it has successfully adjusted the container memory parameters.

[0141] Correspondingly, the computer device can receive a first processing result notification returned by the configuration management component of the expansion subtask. If the first processing result notification indicates that the container memory parameters of the expansion subtask have been successfully adjusted, it can continue to send a second parameter increase instruction for the heap memory parameters of the expansion subtask to the configuration management component of the expansion subtask. This second parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the heap memory parameters of the expansion subtask. These heap memory parameters can refer to the JVM heap memory parameters of the expansion subtask. By increasing these parameters, the JVM heap memory of the expansion subtask can be increased. Similarly, the configuration management component corresponding to the expansion subtask can adjust the heap memory parameters accordingly based on the second parameter increase instruction and return a processing result notification message (referred to as the second processing result notification) to the computer device. This notification message indicates whether the heap memory parameters have been successfully adjusted.

[0142] Correspondingly, the computer device can receive a second processing result notification returned by the configuration management component of the expansion subtask; if the second processing result notification indicates that the adjustment of the heap memory parameters of the expansion subtask has failed, it can send a first rollback instruction for the container memory parameters of the expansion subtask to the configuration management component of the expansion subtask; this first rollback instruction is used to instruct the configuration management component of the expansion subtask to restore the container memory parameters and processor resource parameters of the expansion subtask.

[0143] It is worth noting that during resource expansion, the increase of processor resource parameters can be performed independently from the increase of container memory parameters and heap memory parameters. That is, the increase of processor resource parameters can be executed after increasing heap memory parameters, before increasing heap memory parameters, or even before or after increasing container memory parameters. However, only after both container memory parameters and heap memory parameters have been successfully increased simultaneously can each parameter be considered successfully adjusted. Otherwise, if the adjustment of any one of the container memory parameters or heap memory parameters fails, the other successfully adjusted parameters will be rolled back to their original state. Based on this, this application allows adjusting processor resource parameters only after successfully increasing both container memory parameters and heap memory parameters simultaneously, thus minimizing the number of rollbacks. In other words, when performing resource expansion processing on the expansion subtask, this application can first adjust its container memory resources (increase them). If the container memory parameters are successfully adjusted, then the heap memory parameters of the expansion subtask can be adjusted (increased). If the heap memory parameters are also successfully adjusted, then the processor resources of the container of the expansion subtask can be adjusted (increased). However, if the heap memory parameters are not successfully adjusted, then there is no need to adjust the processor resources of the container of the expansion subtask. In this case, the step of adjusting and increasing the container memory parameters will be rolled back to restore the container memory parameters to the state before the increase.

[0144] Step S404: Perform resource reduction processing on subtasks whose resource adjustment method is resource reduction.

[0145] Specifically, after determining the resource adjustment method for each subtask, subtasks whose resource adjustment method is resource scaling down can be processed for resource scaling down. The specific implementation process may include, but is not limited to: identifying subtasks whose resource adjustment method is resource scaling down as scaling down subtasks; and then, providing configuration management components for scaling down subtasks (as described above)... Figure 3The subtask management component (as shown) sends a first parameter reduction instruction. This first parameter reduction instruction instructs the configuration management component of the shrinking subtask to reduce the heap memory parameters and processor resource parameters of the shrinking subtask. The heap memory parameters of the shrinking subtask are the parameters of the JVM heap memory of the shrinking subtask. Reducing these parameters reduces the heap memory of the shrinking subtask. The processor resource parameters of the shrinking subtask refer to the parameters of the CPU resources of the container of the shrinking subtask. Reducing these parameters reduces the CPU resources of the container of the shrinking subtask. Based on the first parameter reduction instruction, the configuration management component corresponding to the shrinking subtask can adjust its heap memory parameters and processor resource parameters accordingly and return a processing result notification message (which can be called a third processing result notification) to the computer device. This notification message indicates whether it has successfully adjusted the heap memory parameters.

[0146] Correspondingly, the computer device can receive a third processing result notification returned by the configuration management component of the scaling subtask. If the third processing result notification indicates that the heap memory parameters of the scaling subtask have been successfully adjusted, it reflects that the heap memory of the scaling subtask has been successfully reduced. The computer device can then continue to send a second parameter reduction instruction for the container memory parameters of the scaling subtask to the configuration management component of the scaling subtask. The second parameter reduction instruction instructs the configuration management component of the scaling subtask to reduce the container memory parameters of the scaling subtask. The container memory parameters of the scaling subtask refer to the memory parameters of the container providing services to the scaling subtask. Reducing these parameters reduces the container memory of the scaling subtask. Similarly, the configuration management component corresponding to the scaling subtask can adjust the container memory parameters accordingly based on the second parameter reduction instruction and return a processing result notification message (which can be called a fourth processing result notification) to the computer device. This notification message indicates whether it has successfully adjusted the container memory parameters of the scaling subtask.

[0147] Correspondingly, the computer device can receive a fourth processing result notification returned by the configuration management component of the scaling subtask; if the fourth processing result notification indicates that the container memory parameter adjustment of the scaling subtask has failed, a second rollback instruction for the container memory parameters of the scaling subtask can be sent to the configuration management component of the scaling subtask; this second rollback instruction is used to instruct the configuration management component of the scaling subtask to restore the heap memory parameters and processor resource parameters of the scaling subtask.

[0148] It is worth noting that during resource scaling down, the reduction of processor resource parameters can be performed independently of the reduction of container memory parameters and heap memory parameters. That is, the reduction of processor resource parameters can be performed after reducing heap memory parameters, before reducing heap memory parameters, or even before or after reducing container memory parameters. However, only after successfully reducing both container memory parameters and heap memory parameters simultaneously can each parameter be considered successfully adjusted. Otherwise, if any one of the container memory or heap memory parameters fails to adjust, the other successfully adjusted parameters will be rolled back to their original state. Based on this, this application allows adjusting processor resource parameters only after successfully reducing both container memory parameters and heap memory parameters simultaneously, thus minimizing the number of rollbacks. In other words, when performing resource scaling down on a scaling down subtask, this application can first adjust its heap memory parameters (reduce them). If the heap memory parameters are successfully adjusted, the container memory parameters of the scaling down subtask can then be adjusted (reduced). If the container memory parameters are also successfully adjusted, the processor resources of the container of the scaling down subtask can be adjusted (reduced). However, if the container memory parameters are not successfully adjusted, there is no need to adjust the processor resources of the container of the scaling down subtask. In this case, the step of adjusting and reducing the heap memory parameters will be rolled back to restore the heap memory parameters to their state before the reduction.

[0149] For ease of understanding, please refer to the following: Figure 5 , Figure 5 This is a schematic diagram of the logical architecture for resource adjustment processing provided in an embodiment of this application. Wherein, as... Figure 5 The logical architecture for resource adjustment shown may include at least the following components: a computing platform, a task management component, a resource prediction model, and various containers. For ease of understanding, each component will be briefly described below:

[0150] Computing Platform: Computing tasks in the computing platform can be deployed in a container orchestration system, where containers provide corresponding services.

[0151] Task Management Component: The task management component can be used to manage the configuration data of computing tasks (e.g., editing, modifying, deleting, etc.).

[0152] Containers: can provide corresponding services for corresponding computing tasks. Different processes (such as Java processes) can run concurrently in a container, and each process can be understood as a subtask of the computing task.

[0153] Resource prediction model: The resource prediction model can be used to predict the computing resources required for each subtask based on the monitored indicator information of each subtask, and return the prediction results to the computing platform. The computing platform can then send the model prediction results to the task management component, which can then adjust the resources for each subtask accordingly based on the prediction results.

[0154] In this embodiment, by collecting the operational metrics information of each subtask in an earlier time period (such as the first time period), the amount of computing resources required for it in a later time period can be predicted. Then, based on the predicted demand, resources can be adjusted for subtasks that meet the resource adjustment conditions (e.g., resource expansion or resource reduction). Without changing the concurrency of the computing tasks or restarting the computing tasks, the resources of each subtask can be elastically scaled, achieving zero-interruption elastic adjustment of computing resources. This not only reduces the backlog of computing tasks caused by restarts but also effectively improves the efficiency of adjusting computing resources.

[0155] Based on the above description, in order to achieve zero-interruption elastic scaling and adjustment of computing resources for computing tasks, this application makes the following three design aspects on the computing platform side:

[0156] I. Design of a Flexible and Scalable Computing Platform. This application incorporates the following four design aspects on the computing platform side:

[0157] 1. A flexible rule hot-update mechanism is designed. This application supports rich elastic scaling strategies, allowing users to customize elastic rules for the computing platform. These user-defined elastic rules are referred to as resource adjustment rules in this application. A rule monitoring component is set up on the computing platform to monitor resource adjustment rules in real time and dynamically load them. 2. A scalable elastic algorithm management mechanism is designed. In traditional technologies, most elastic scaling solutions rely on statistical calculations of general metrics for computing tasks. However, in data processing for computing tasks, the performance of streaming tasks is affected by various metrics. Therefore, this application designs a scalable elastic algorithm management module that can flexibly introduce resource prediction models and use richer metric information from various subtasks (the specific metric information used can be defined based on user-defined resource adjustment rules) to improve the accuracy of prediction results. 3. A mechanism for managing resource adjustment behavior is designed. In this regard, by dividing computing tasks according to task priority and pre-verifying the resource adjustment behavior of each subtask, the feasibility of resource adjustments is improved. Secondly, by monitoring indicator information and predicting the required computing resources in real time, emergency resource expansion can be supported in the event of sudden flow scenarios. When adjusting resources, unexpected adjustments can be rolled back, thus reducing the problem of resources not being adjusted accordingly, thereby achieving the stability of elastic scaling adjustments. 4. Resource utilization management is designed. In this regard, idle resources of computing tasks can be rationally utilized, and resources can be reallocated according to task priority and resource weight, thereby effectively improving the overall resource utilization of computing tasks.

[0158] II. The design of the computing engine supports dynamic configuration updates. Traditionally, updating the configuration data of computing tasks involves restarting the task for re-initialization, which can cause interruptions. This application, however, opens up the computing engine's interface and incorporates built-in dynamic configuration management components (such as task management and subtask management components). When a configuration update request is received, the subtask management components of each subtask will detect the configuration update event, thus enabling timely dynamic updates of the subtask's configuration. Based on this capability, personalized configuration of computing resources for computing tasks can be achieved.

[0159] Third, the design integrates and adjusts containers with the JVM. While containers in container orchestration systems can dynamically adjust computing resources, they cannot adjust the internal processes, requiring process destruction and reconstruction. The container-JVM integration proposed in this application enables complete resource elasticity in cloud-native scenarios.

[0160] Ultimately, through the above three design aspects, it is possible to adjust CPU and memory resources for computing tasks without interruption, and to achieve elastic scaling of resources for each subtask without stopping or restarting the computing task.

[0161] Based on the above-described solution, this application ultimately achieves the following technical effects: it enables fine-grained dynamic adjustment of CPU and memory resources for each node of a computing task without causing data processing interruptions. Specifically, this is reflected in: 1. In terms of computing resources, this application can achieve on-demand allocation through a resource prediction model, maximizing resource utilization and minimizing waste; 2. In terms of stability, it effectively solves the problem of insufficient computing resources for computing tasks, improving task stability; 3. In terms of data processing, it can dynamically adjust computing resources for computing tasks, achieving zero data processing interruptions and having no negative impact on data processing. For a better understanding of the beneficial effects of this application, please refer to [further details omitted]. Figures 6-7 ,in, Figure 6 This is a schematic diagram illustrating the effect of elastic scaling of memory resources provided in an embodiment of this application. Figure 7 This is a schematic diagram illustrating the effect of elastic scaling of processor resources provided in an embodiment of this application.

[0162] like Figure 6 As shown, with the configured memory resource value remaining constant, the predicted computing resource value (such as the predicted memory resource amount mentioned above) in this application is always greater than the actual computing resource value used (the actual memory resource amount used), which can meet the actual usage requirements. Moreover, the predicted computing resource value is very close to the actual computing resource value used, which can reduce the waste of computing resources while meeting the requirements. Figure 7 As shown, when the configured CPU resource value remains constant, the computing resource value predicted by this application (such as the processor resource quantity predicted above) can always be greater than the actual computing resource value used (the actual processor resource quantity used), which can meet the actual usage requirements. Moreover, the predicted computing resource value is very close to the actual computing resource value used, which can reduce the waste of computing resources while meeting the requirements.

[0163] Further, please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a resource processing device for a computing task provided in an embodiment of this application. The resource processing device for the computing task can be a computer program (including program code) running on a computer device; for example, the resource processing device for the computing task is an application software. The resource processing device for the computing task can be used to execute... Figure 3 The method shown. (As illustrated) Figure 8 As shown, the resource processing device 1 for this computing task may include: an information acquisition module 11, a prediction module 12, a subtask selection module 13, and a resource adjustment module 14.

[0164] The information acquisition module 11 is used to acquire the running indicator information of the computing task in the first time period; the computing task includes N concurrently running subtasks, and the running indicator information of the computing task includes the running indicator information of each subtask, where N is a positive integer;

[0165] Prediction module 12 is used to predict and analyze the computing resource requirements of each subtask in the second time period based on the operation index information of each subtask, and obtain the predicted resource requirements of each subtask; the second time period is later than the first time period.

[0166] The subtask selection module 13 is used to select M subtasks that meet the resource adjustment conditions from the N subtasks based on the predicted resource demand corresponding to each of the N subtasks; M is an integer and M is less than or equal to N.

[0167] Resource adjustment module 14 is used to adjust the resources of M subtasks during the second time period.

[0168] The specific implementation methods of the information acquisition module 11, prediction module 12, subtask selection module 13, and resource adjustment module 14 can be found in the above description. Figure 2 The descriptions of steps S201-S204 in the corresponding embodiments will not be repeated here.

[0169] In one embodiment, the prediction module 12 predicts and analyzes the computational resource requirements of each subtask during the second time period based on the operational metric information of each subtask, and obtains a specific implementation method for the predicted resource requirements corresponding to each subtask, including:

[0170] Based on the operational metrics of each subtask, the resource prediction model is invoked;

[0171] The computational resource requirements of each subtask in the second time period are calculated and analyzed using the model parameters of the resource prediction model, and the predicted resource requirements for each subtask are output.

[0172] In one embodiment, the training process of the resource prediction model includes:

[0173] Obtain the sample operation index information of the sample task within the first sample time period; the sample task includes Q sample subtasks running concurrently, and the sample operation index information includes the sample operation index information of each sample subtask, where Q is an integer greater than 1.

[0174] The initial resource prediction model is invoked based on the sample operation index information of each sample subtask;

[0175] The computational resource requirements of each sample subtask in the second sample time period are predicted and analyzed using the initial resource prediction model to obtain the sample predicted resource requirements of each sample subtask; the second sample time period is later than the first sample time period.

[0176] Based on the difference between the total amount of computing resources actually used by each sample subtask in the second sample time period and the corresponding sample predicted resource demand, the initial resource prediction model is trained to obtain the resource prediction model.

[0177] In one embodiment, the computing platform where the computing task resides is deployed with a rule monitoring component, which is used to monitor and record the resource adjustment rules defined by the rule definition object;

[0178] The resource adjustment conditions are determined by the rule monitoring component based on the monitored resource adjustment rules; when the rule monitoring component detects a change in the resource adjustment rules, it dynamically adjusts the resource adjustment conditions.

[0179] In one embodiment, the subtask selection module 13, based on the predicted resource demand corresponding to each of the N subtasks, selects M subtasks from the N subtasks that meet the resource adjustment conditions. The specific implementation includes:

[0180] The predicted resource requirements of each of the N sub-tasks are summed to obtain the total predicted resource requirements of the N sub-tasks in the second time period.

[0181] Get the total amount of configured resources corresponding to the computing task; the total amount of configured resources is the total amount of computing resources configured for the computing task.

[0182] Based on the relationship between the total configured resources and the total predicted resource demand, select M subtasks from N subtasks that meet the resource adjustment conditions.

[0183] In one embodiment, the relationship between the total configured resources and the total predicted resource demand is such that the total predicted resource demand is greater than the total configured resources.

[0184] Subtask selection module 13, based on the total configured resources and the predicted total resource demand, selects M subtasks from N subtasks that meet the resource adjustment conditions, including:

[0185] Sort the N subtasks according to the task sorting rules to obtain the subtask sequence;

[0186] Calculate the total resource requirement for each subtask in the order of the N subtasks in the subtask sequence. The total resource requirement for any subtask is the sum of the predicted resource requirements of the subtask and its predecessor. In the subtask sequence, the predecessor of a subtask is located before the subtask.

[0187] The M subtasks in the subtask sequence whose total required resources are less than the total configured resources are identified as the M subtasks that meet the resource adjustment conditions.

[0188] In one embodiment, the relationship between the total configured resources and the total predicted resource demand is such that the total predicted resource demand is less than the total configured resources.

[0189] Subtask selection module 13, based on the total configured resources and the predicted total resource demand, selects M subtasks from N subtasks that meet the resource adjustment conditions, including:

[0190] Get the resource adjustment time period corresponding to each subtask; the resource adjustment time period corresponding to the subtask is the time period during which resource adjustments are allowed to be made pre-configured for the subtask;

[0191] The M subtasks that include the second time period in the corresponding resource adjustment period are identified as meeting the resource adjustment conditions.

[0192] In one embodiment, the specific implementation of resource adjustment module 14 for adjusting resources for M subtasks during the second time period includes:

[0193] Get the amount of resources used by each of the M subtasks in the first time period;

[0194] Based on the amount of resources used and the corresponding predicted resource demand of each of the M subtasks, determine the resource adjustment method for each of the M subtasks.

[0195] During the second time period, resource expansion processing will be performed on subtasks whose resource adjustment method is resource expansion; and,

[0196] Perform resource reduction processing on subtasks whose resource adjustment method is resource reduction.

[0197] In one embodiment, the resource adjustment module 14 determines the specific implementation method of the resource adjustment method for each of the M subtasks based on the used resource amount and the corresponding predicted resource demand of each subtask, including:

[0198] Choose any one of the M subtasks as the target subtask;

[0199] If the amount of resources used in the target subtask is greater than the corresponding predicted resource demand, then the resource adjustment method for the target subtask is determined to be the resource reduction method.

[0200] If the amount of resources used in the target subtask is less than the corresponding predicted resource demand, then the resource adjustment method for the target subtask is determined to be the resource expansion method.

[0201] In one embodiment, the specific implementation method of the resource adjustment module 14 for performing resource expansion processing on subtasks whose resource adjustment method is resource expansion includes:

[0202] The subtask that adjusts resources by expanding them is designated as the expansion subtask.

[0203] Send a first parameter increase instruction to the configuration management component of the expansion subtask; the first parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the container memory parameters and processor resource parameters of the expansion subtask.

[0204] Receive the first processing result notification returned by the configuration management component of the expansion subtask;

[0205] If the first processing result notification indicates that the container memory parameters of the expansion subtask have been successfully adjusted, a second parameter increase instruction for the heap memory parameters of the expansion subtask is sent to the configuration management component of the expansion subtask. The second parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the heap memory parameters of the expansion subtask.

[0206] In one embodiment, after the resource adjustment module 14 sends a second parameter increase instruction for the heap memory parameters of the expansion subtask to the configuration management component of the expansion subtask, the resource adjustment module is further specifically used for:

[0207] Receive the second processing result notification returned by the configuration management component of the expansion subtask;

[0208] If the second processing result notification indicates that the adjustment of the heap memory parameters of the expansion subtask has failed, a first rollback instruction for the container memory parameters of the expansion subtask is sent to the configuration management component of the expansion subtask. The first rollback instruction is used to instruct the configuration management component of the expansion subtask to restore the container memory parameters and processor resource parameters of the expansion subtask.

[0209] In one embodiment, the specific implementation of resource scaling-up processing for subtasks whose resource adjustment method is resource scaling-up includes:

[0210] The subtask that adjusts resources by reducing their size is designated as the reduction subtask.

[0211] Send a first parameter reduction instruction to the configuration management component of the shrinking subtask; the first parameter reduction instruction is used to instruct the configuration management component of the shrinking subtask to reduce the heap memory parameters and processor resource parameters of the shrinking subtask.

[0212] Receive the third processing result notification returned by the configuration management component of the scaling subtask;

[0213] If the third processing result notification indicates that the heap memory parameters of the shrinking subtask have been successfully adjusted, a second parameter reduction instruction for the container memory parameters of the shrinking subtask is sent to the configuration management component of the shrinking subtask. The second parameter reduction instruction is used to instruct the configuration management component of the shrinking subtask to reduce the container memory parameters of the shrinking subtask.

[0214] This application provides a scheme for elastically scaling and adjusting computing resources for computing tasks. This allows for effective adjustment of computing resource allocation without changing the concurrency of the computing tasks, thereby reducing the backlog of computing tasks. Specifically, for any computing task, its operational metrics information within a first time period can be obtained. Since the computing task concurrently runs with N (N is a positive integer) subtasks, this operational metrics information includes the operational metrics information of each subtask. After obtaining the operational metrics information of each subtask, this application can predict and analyze the computing resource requirements of each subtask in a second time period (later than the first time period) based on the operational metrics information of each subtask. This yields the predicted resource requirements of each subtask in the second time period, which is the total amount of computing resources predicted by this application for each subtask in the second time period. Thus, upon reaching the second time period, based on the predicted resource requirements of each subtask, M subtasks that meet the resource adjustment conditions can be identified, and resource adjustment processing (e.g., resource expansion or resource reduction) can be applied to these M subtasks. As can be seen, by collecting the operational metrics information of each subtask in an earlier time period (such as the first time period), this application can predict the computing resources required by each subtask in a later time period. Then, based on the predicted computing resource requirements, resource adjustment processing (e.g., resource expansion or resource reduction) can be performed on subtasks that meet the resource adjustment conditions. This achieves elastic scaling of resources for each subtask without changing the concurrency of the computing tasks or restarting the computing tasks. It achieves zero-interruption elastic adjustment of computing resources for computing tasks, which not only reduces the backlog of computing tasks caused by restarts, but also effectively improves the efficiency of adjusting computing resources for computing tasks.

[0215] Further, please see Figure 9 , Figure 9This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 9 As shown, the aforementioned computer device 8000 may include: a processor 8001, a network interface 8004, and a memory 8005. Furthermore, the computer device 8000 also includes: a user interface 8003, and at least one communication bus 8002. The communication bus 8002 is used to enable communication between these components. The user interface 8003 may include a display screen and a keyboard; optionally, the user interface 8003 may also include a standard wired interface or a wireless interface. The network interface 8004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 8005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 8005 may also be at least one storage device located remotely from the aforementioned processor 8001. Figure 9 As shown, the memory 8005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0216] exist Figure 9 In the computer device 8000 shown, the network interface 8004 provides network communication functionality; the user interface 8003 is mainly used to provide an input interface for the user; and the processor 8001 can be used to call the device control application program stored in the memory 8005 to achieve:

[0217] Obtain the performance metrics information of the computation task within the first time period; the computation task includes N concurrently running subtasks, and the performance metrics information of the computation task includes the performance metrics information of each subtask, where N is a positive integer;

[0218] Based on the operational metrics of each subtask, the computational resource requirements of each subtask in the second time period are predicted and analyzed to obtain the predicted resource requirements for each subtask; the second time period is later than the first time period.

[0219] Based on the predicted resource demand corresponding to each of the N subtasks, select M subtasks from the N subtasks that meet the resource adjustment conditions; M is an integer and M is less than or equal to N;

[0220] During the second time period, resources are adjusted for M subtasks.

[0221] In one embodiment,

[0222] In one embodiment, when the processor 8001 performs predictive analysis on the computational resource requirements of each subtask in the second time period based on the running metric information of each subtask, and obtains the predicted resource requirements corresponding to each subtask, it specifically performs the following steps:

[0223] Based on the operational metrics of each subtask, the resource prediction model is invoked;

[0224] The computational resource requirements of each subtask in the second time period are calculated and analyzed using the model parameters of the resource prediction model, and the predicted resource requirements for each subtask are output.

[0225] In one embodiment, the training process of the resource prediction model includes:

[0226] Obtain the sample operation index information of the sample task within the first sample time period; the sample task includes Q sample subtasks running concurrently, and the sample operation index information includes the sample operation index information of each sample subtask, where Q is an integer greater than 1.

[0227] The initial resource prediction model is invoked based on the sample operation index information of each sample subtask;

[0228] The computational resource requirements of each sample subtask in the second sample time period are predicted and analyzed using the initial resource prediction model to obtain the sample predicted resource requirements of each sample subtask; the second sample time period is later than the first sample time period.

[0229] Based on the difference between the actual computational resources used by each sample subtask during the second sample time period and the corresponding sample predicted resource demand, the initial resource prediction model is trained to obtain the resource prediction model.

[0230] In one embodiment, the computing platform where the computing task resides is deployed with a rule monitoring component, which is used to monitor and record the resource adjustment rules defined by the rule definition object;

[0231] The resource adjustment conditions are determined by the rule monitoring component based on the monitored resource adjustment rules; when the rule monitoring component detects a change in the resource adjustment rules, it dynamically adjusts the resource adjustment conditions.

[0232] In one embodiment, when the processor 8001 executes the process of selecting M subtasks that meet the resource adjustment conditions from the N subtasks based on the predicted resource demand corresponding to each of the N subtasks, it specifically performs the following steps:

[0233] The predicted resource requirements of each of the N sub-tasks are summed to obtain the total predicted resource requirements of the N sub-tasks in the second time period.

[0234] Get the total amount of configured resources corresponding to the computing task; the total amount of configured resources is the total amount of computing resources configured for the computing task.

[0235] Based on the relationship between the total configured resources and the total predicted resource demand, select M subtasks from N subtasks that meet the resource adjustment conditions.

[0236] In one embodiment, the relationship between the total configured resources and the total predicted resource demand is such that the total predicted resource demand is greater than the total configured resources.

[0237] When processor 8001 executes the process of selecting M subtasks from N subtasks that meet the resource adjustment conditions based on the total configured resources and the predicted total resource demand, it specifically performs the following steps:

[0238] Obtain the task priorities corresponding to N subtasks, and sort the N subtasks according to their respective task priorities to obtain a subtask sequence;

[0239] Calculate the total resource requirement for each subtask in the order of the N subtasks in the subtask sequence. The total resource requirement for any subtask is the sum of the predicted resource requirements of the subtask and its predecessor. In the subtask sequence, the predecessor of a subtask is located before the subtask.

[0240] The M subtasks in the subtask sequence whose total required resources are less than the total configured resources are identified as the M subtasks that meet the resource adjustment conditions.

[0241] In one embodiment, the relationship between the total configured resources and the total predicted resource demand is such that the total predicted resource demand is less than the total configured resources.

[0242] When processor 8001 executes the process of selecting M subtasks from N subtasks that meet the resource adjustment conditions based on the total configured resources and the predicted total resource demand, it specifically performs the following steps:

[0243] Get the resource adjustment time period corresponding to each subtask; the resource adjustment time period corresponding to the subtask is the time period during which resource adjustments are allowed to be made pre-configured for the subtask;

[0244] The M subtasks that include the second time period in the corresponding resource adjustment period are identified as meeting the resource adjustment conditions.

[0245] In one embodiment, when processor 8001 performs resource adjustment processing on M subtasks during a second time period, it specifically executes the following steps:

[0246] Get the amount of computing resources used by each of the M subtasks in the first time period;

[0247] Based on the amount of resources used and the corresponding predicted resource demand of each of the M subtasks, determine the resource adjustment method for each of the M subtasks.

[0248] During the second time period, resource expansion processing will be performed on subtasks whose resource adjustment method is resource expansion; and,

[0249] Perform resource reduction processing on subtasks whose resource adjustment method is resource reduction.

[0250] In one embodiment, when the processor 8001 determines the resource adjustment method for each of the M subtasks based on the used resource amount and the corresponding predicted resource demand of each subtask, it specifically performs the following steps:

[0251] Choose any one of the M subtasks as the target subtask;

[0252] If the amount of resources used in the target subtask is greater than the corresponding predicted resource demand, then the resource adjustment method for the target subtask is determined to be the resource reduction method.

[0253] If the amount of resources used in the target subtask is less than the corresponding predicted resource demand, then the resource adjustment method for the target subtask is determined to be the resource expansion method.

[0254] In one embodiment, when the processor 8001 performs resource expansion processing on a subtask whose resource adjustment method is resource expansion, it specifically executes the following steps:

[0255] The subtask that adjusts resources by expanding them is designated as the expansion subtask.

[0256] Send a first parameter increase instruction to the configuration management component of the expansion subtask; the first parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the container memory parameters and processor resource parameters of the expansion subtask.

[0257] Receive the first processing result notification returned by the configuration management component of the expansion subtask;

[0258] If the first processing result notification indicates that the container memory parameters of the expansion subtask have been successfully adjusted, a second parameter increase instruction for the heap memory parameters of the expansion subtask is sent to the configuration management component of the expansion subtask. The second parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the heap memory parameters of the expansion subtask.

[0259] In one embodiment, after the processor 8001 sends a second parameter increase instruction for the heap memory parameters of the expansion subtask to the configuration management component of the expansion subtask, the processor 8001 further performs the following steps:

[0260] Receive the second processing result notification returned by the configuration management component of the expansion subtask;

[0261] If the second processing result notification indicates that the adjustment of the heap memory parameters of the expansion subtask has failed, a first rollback instruction for the container memory parameters of the expansion subtask is sent to the configuration management component of the expansion subtask. The first rollback instruction is used to instruct the configuration management component of the expansion subtask to restore the container memory parameters and processor resource parameters of the expansion subtask.

[0262] In one embodiment, when the processor 8001 performs resource scaling down processing on a subtask whose resource adjustment method is resource scaling down, it specifically executes the following steps:

[0263] The subtask that adjusts resources by reducing their size is designated as the reduction subtask.

[0264] Send a first parameter reduction instruction to the configuration management component of the shrinking subtask; the first parameter reduction instruction is used to instruct the configuration management component of the shrinking subtask to reduce the heap memory parameters and processor resource parameters of the shrinking subtask.

[0265] Receive the third processing result notification returned by the configuration management component of the scaling subtask;

[0266] If the third processing result notification indicates that the heap memory parameters of the shrinking subtask have been successfully adjusted, a second parameter reduction instruction for the container memory parameters of the shrinking subtask is sent to the configuration management component of the shrinking subtask. The second parameter reduction instruction is used to instruct the configuration management component of the shrinking subtask to reduce the container memory parameters of the shrinking subtask.

[0267] In this embodiment, by collecting the operational metrics information of each subtask in an earlier time period (such as the first time period), the amount of computing resources required for it in a later time period can be predicted. Then, based on the predicted demand, resources can be adjusted for subtasks that meet the resource adjustment conditions (e.g., resource expansion or resource reduction). Without changing the concurrency of the computing tasks or restarting the computing tasks, the resources of each subtask can be elastically scaled, achieving zero-interruption elastic adjustment of computing resources. This not only reduces the backlog of computing tasks caused by restarts but also effectively improves the efficiency of adjusting computing resources.

[0268] It should be understood that the computer device 8000 described in the embodiments of this application can execute the foregoing text. Figures 3 to 5 The description of the resource processing method for this computing task in the corresponding embodiment can also be executed as described above. Figure 8 The description of the resource processing device 1 for the computing task in the corresponding embodiment will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated here.

[0269] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program executed by the aforementioned data processing computer device 8000. The computer program includes program instructions, and when the processor executes the program instructions, it can execute the aforementioned... Figures 3 to 5 The resource processing method for the above-described computing task has already been described in the corresponding embodiments, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0270] The aforementioned computer-readable storage medium can be a resource processing device for computing tasks provided in any of the foregoing embodiments or an internal storage unit of the aforementioned computer device, such as a hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium may include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0271] One aspect of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in one aspect of the embodiments of this application.

[0272] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0273] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functions of that module or unit.

[0274] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0275] The methods and related apparatuses provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0276] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A resource processing method for a computing task, characterized in that, include: Obtain the operational metrics information of the computing task within the first time period; The computing task includes N concurrently running subtasks, and the computing task's performance metrics include the performance metrics of each subtask, where N is a positive integer. Based on the operational metrics information of each subtask, the computational resource requirements of each subtask in the second time period are predicted and analyzed to obtain the predicted resource requirements for each subtask. The second time period is later than the first time period; Based on the predicted resource demand corresponding to the N sub-tasks, M sub-tasks that meet the resource adjustment conditions are obtained from the N sub-tasks. M is an integer and M is less than or equal to N; During the second time period, resource adjustments are made to the M sub-tasks.

2. The method according to claim 1, characterized in that, Based on the operational metric information of each sub-task, the computational resource requirements of each sub-task during the second time period are predicted and analyzed to obtain the predicted resource requirements for each sub-task, including: Based on the operational metric information of each subtask, the resource prediction model is invoked; The computational resource requirements of each subtask in the second time period are calculated and analyzed using the model parameters of the resource prediction model, and the predicted resource requirements for each subtask are output.

3. The method according to claim 2, characterized in that, The training process of the resource prediction model includes: Obtain sample operation index information of the sample task within the first sample time period; the sample task includes Q sample subtasks running concurrently, and the sample operation index information of the sample task includes the sample operation index information of each sample subtask, where Q is an integer greater than 1. The initial resource prediction model is invoked based on the sample operation index information of each sample subtask; The initial resource prediction model is used to predict and analyze the computational resource requirements of each sample subtask during the second sample time period, thereby obtaining the sample predicted resource requirements for each sample subtask; the second sample time period is later than the first sample time period. The initial resource prediction model is trained based on the difference between the total amount of computing resources actually used by each sample subtask during the second sample time period and the corresponding sample predicted resource requirement.

4. The method according to claim 1, characterized in that, The computing platform where the computing task is located is equipped with a rule monitoring component, which is used to monitor and record the resource adjustment rules defined by the rule definition object. The resource adjustment conditions are determined by the rule monitoring component based on the monitored resource adjustment rules; when the rule monitoring component detects a change in the resource adjustment rules, it dynamically adjusts the resource adjustment conditions.

5. The method according to claim 1, characterized in that, The step of obtaining M subtasks that meet the resource adjustment conditions from the N subtasks based on the predicted resource demand corresponding to each of the N subtasks includes: The predicted resource requirements corresponding to the N sub-tasks are summed to obtain the total predicted resource requirements of the N sub-tasks in the second time period. Obtain the total amount of configured resources corresponding to the computing task; the total amount of configured resources is the total amount of computing resources configured for the computing task; Based on the relationship between the total configured resources and the total predicted resource demand, M subtasks that meet the resource adjustment conditions are selected from the N subtasks.

6. The method according to claim 5, characterized in that, The relationship between the total configured resources and the total predicted resource demand is such that the total predicted resource demand is greater than the total configured resources. The step of obtaining M subtasks that meet the resource adjustment conditions from the N subtasks based on the total configured resources and the total predicted resource demand includes: The N subtasks are sorted according to the task sorting rules to obtain a subtask sequence; According to the arrangement order of the N subtasks in the subtask sequence, the total resource requirement corresponding to each subtask is calculated sequentially; the total resource requirement corresponding to any subtask is the sum of the predicted resource requirements of the subtask and its predecessor subtask; in the subtask sequence, the predecessor subtask of the subtask is located before the subtask. The M subtasks in the subtask sequence whose total required resources are less than the total configured resources are identified as the M subtasks that meet the resource adjustment conditions.

7. The method according to claim 5, characterized in that, The relationship between the total configured resources and the total predicted resource demand is such that the total predicted resource demand is less than the total configured resources. The step of obtaining M subtasks that meet the resource adjustment conditions from the N subtasks based on the total configured resources and the total predicted resource demand includes: Obtain the resource adjustment time period corresponding to each subtask; the resource adjustment time period corresponding to the subtask is a pre-configured time period for the subtask that allows resource adjustments; The M subtasks that include the second time period in the corresponding resource adjustment period are identified as the M subtasks that meet the resource adjustment conditions.

8. The method according to claim 1, characterized in that, The resource adjustment for the M subtasks during the second time period includes: Obtain the amount of computing resources used by each of the M subtasks during the first time period; Based on the amount of resources used and the corresponding predicted resource demand of each of the M subtasks, determine the resource adjustment method for each of the M subtasks; During the second time period, resource expansion processing is performed on subtasks whose resource adjustment method is resource expansion; and, Perform resource reduction processing on subtasks whose resource adjustment method is resource reduction.

9. The method according to claim 8, characterized in that, The method for determining the resource adjustment method for each of the M subtasks based on the used resource amount and the corresponding predicted resource demand of each subtask includes: Choose any one of the M subtasks as the target subtask; If the amount of resources used in the target subtask is greater than the corresponding predicted resource demand, then the resource adjustment method for the target subtask is determined to be the resource reduction method. If the amount of resources used in the target subtask is less than the corresponding predicted resource demand, then the resource adjustment method for the target subtask is determined to be the resource expansion method.

10. The method according to claim 8, characterized in that, The process of expanding the resource capacity of subtasks whose resource adjustment method is resource expansion includes: The subtask that adjusts resources by resource expansion is defined as the expansion subtask; Send a first parameter increase instruction to the configuration management component of the expansion subtask; the first parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the container memory parameters and processor resource parameters of the expansion subtask; Receive the first processing result notification returned by the configuration management component of the expansion subtask; If the first processing result notification indicates that the container memory parameters of the expansion subtask have been successfully adjusted, a second parameter increase instruction for the heap memory parameters of the expansion subtask is sent to the configuration management component of the expansion subtask; the second parameter increase instruction is used to instruct the configuration management component of the expansion subtask to increase the heap memory parameters of the expansion subtask.

11. The method according to claim 10, characterized in that, After sending a second parameter increase instruction for the heap memory parameters of the expansion subtask to the configuration management component of the expansion subtask, the method further includes: Receive the second processing result notification returned by the configuration management component of the expansion subtask; If the second processing result notification indicates that the adjustment of the heap memory parameters of the expansion subtask has failed, a first rollback instruction for the container memory parameters of the expansion subtask is sent to the configuration management component of the expansion subtask; the first rollback instruction is used to instruct the configuration management component of the expansion subtask to restore the container memory parameters and processor resource parameters of the expansion subtask.

12. The method according to claim 9, characterized in that, The process of performing resource scaling down on subtasks whose resource adjustment method is resource scaling down includes: The subtask that adjusts resources by reducing their size is designated as the reduction subtask. Send a first parameter reduction instruction to the configuration management component of the shrinking subtask; the first parameter reduction instruction is used to instruct the configuration management component of the shrinking subtask to reduce the heap memory parameters and processor resource parameters of the shrinking subtask. Receive the third processing result notification returned by the configuration management component of the scaling-down subtask; If the third processing result notification indicates that the heap memory parameters of the shrinking subtask have been successfully adjusted, a second parameter reduction instruction for the container memory parameters of the shrinking subtask is sent to the configuration management component of the shrinking subtask; the second parameter reduction instruction is used to instruct the configuration management component of the shrinking subtask to reduce the container memory parameters of the shrinking subtask.

13. A resource processing apparatus for computing tasks, characterized in that, include: The information acquisition module is used to acquire the operational metrics information of the computing task within the first time period; The computing task includes N concurrently running subtasks, and the computing task's performance metrics include the performance metrics of each subtask, where N is a positive integer. The prediction module is used to predict and analyze the computing resource requirements of each subtask in the second time period based on the operation index information of each subtask, so as to obtain the predicted resource requirements corresponding to each subtask. The second time period is later than the first time period; The subtask selection module is used to select M subtasks that meet the resource adjustment conditions from the N subtasks based on the predicted resource demand corresponding to each of the N subtasks. M is an integer and M is less than or equal to N; The resource adjustment module is used to adjust the resources of the M subtasks during the second time period.

14. A computer device, characterized in that, include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide network communication functions, the memory is used to store computer programs, and the processor is used to call the computer programs to cause the computer device to execute the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and to execute the method of any one of claims 1-12.

16. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium, the computer program being adapted to be read and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-12.

Images