A data processing method, apparatus, and storage medium

By acquiring historical resource allocation and usage information of computing nodes and combining it with weight values ​​to calculate the weighted average of resource usage index, the problem of inaccurate calculation of resource idle rate during container deployment is solved, achieving accuracy in container scheduling and balance in resource utilization.

CN116185555BActive Publication Date: 2026-11-17CHINA MOBILE CHENGDU INFORMATION & TELECOMM TECH CO LTD +1
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202111434382.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-29
Publication Date
2026-11-17
Estimated Expiration
2041-11-29

AI Technical Summary

Technical Problem

In existing technologies, the failure to accurately reflect the expected average value of resources required by containers during deployment leads to inaccurate calculation of resource idle rate, resulting in significant deviations in container scheduling.

Method used

By acquiring historical resource allocation and usage information of computing nodes, and combining it with preset weight values ​​to calculate the weighted average of resource usage index, the resource requirements of the target container are determined, and the best node is selected for deployment based on resource availability parameters.

Benefits of technology

It improves the accuracy of container deployment and ensures the balance and efficiency of resource utilization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116185555B_ABST
    Figure CN116185555B_ABST
Patent Text Reader

Abstract

This application provides a data processing method, apparatus, and storage medium. The method includes: upon receiving a deployment request for a target container, obtaining historical resource allocation information of containers already deployed on a computing node within a preset time period, and searching for first resource usage information of containers already deployed on a computing node within a previous preset time period; if the first resource usage information is found, determining second resource usage information corresponding to a preset time period based on the used resources and the first resource usage information; determining the resource usage requirements corresponding to the target container, and determining the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information, and the resource usage requirements; determining the target computing node from the computing nodes based on the resource idle parameters, and deploying the target container to the target computing node; through the above technical solution, the accuracy of container deployment is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of cloud computing, and in particular to a data processing method and apparatus, and a storage medium. Background Technology

[0002] In some business scenarios, users want to deploy containers to nodes with high resource idle rates so that the nodes can meet the load balancing strategy.

[0003] In existing technologies, the resource idle rate of a node is determined by two parts: the resources used by the container at the current time and the resources required for this scheduling. However, during the operation of a container, the resources used at the current time cannot accurately reflect the expected average number of resources required by the container. Therefore, the existing resource idle rate calculation method cannot accurately reflect the current resource usage, resulting in a large deviation during container scheduling. Summary of the Invention

[0004] This application provides a data processing method, apparatus, and storage medium that can improve the accuracy of container deployment.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a data processing method, the method comprising:

[0007] Upon receiving a deployment request for the target container, the system obtains historical resource allocation information of containers already deployed on the compute node within a preset time period, and searches for the first resource usage information of containers already deployed on the compute node within the previous preset time period. The preset time period is the preset time period prior to receiving the deployment request. The historical resource allocation information includes the resources used by containers already deployed on the compute node within the preset time period.

[0008] If the first resource usage information is found, the second resource usage information corresponding to the preset time period is determined based on the used resources and the first resource usage information.

[0009] Determine the resource usage requirements corresponding to the target container, and determine the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information, and the resource usage requirements;

[0010] Based on the resource availability parameter, a target computing node is determined from the computing nodes, and the target container is deployed to the target computing node.

[0011] In the above data processing method, the used resources include used processor resources and used memory resources, and the first resource usage information includes first processor resource usage information and first memory resource usage information; determining the second resource usage information corresponding to the preset time period based on the used resources and the first resource usage information includes:

[0012] The second processor resource usage information corresponding to the preset time period is determined based on the used processor resources, the first processor resource usage information, and the first preset weight value.

[0013] Based on the used memory resources, the first memory resource usage information, and the first preset weight value, determine the second memory resource usage information corresponding to the preset time period;

[0014] The second processor resource usage information and the second memory resource usage information are determined as the second resource usage information corresponding to the preset time period.

[0015] In the above data processing method, the second resource usage information includes second processor resource usage information and second memory resource usage information, and the resource usage requirements include processor resource usage requirements and memory resource usage requirements. Determining the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information, and the resource usage requirements includes:

[0016] The processor resource idle rate of the computing node is determined based on the historical resource allocation information, the second processor resource usage information, and the processor resource usage requirements.

[0017] The memory resource idle rate of the computing node is determined based on the historical resource allocation information, the second memory resource usage information, and the memory resource usage requirements.

[0018] The resource idle parameters of the computing node are determined based on the processor resource idle rate, the memory resource idle rate, and the second preset weight value.

[0019] In the above data processing method, determining the processor resource idle rate of the computing node based on the historical resource allocation information, the second processor resource usage information, and the processor resource usage demand includes:

[0020] Find the total processor resources of the computing node and the minimum processor resource requirement of the deployed container from the historical resource allocation information;

[0021] Determine the first maximum value between the minimum processor resource requirement and the second processor resource usage information;

[0022] The idle processor resources of the computing node are determined based on the total amount of processor resources, the first maximum value, and the processor resource usage requirements.

[0023] The processor resource idle rate of the computing node is determined based on the processor idle resources and the total amount of processor resources.

[0024] In the above data processing method, determining the memory resource idle rate of the computing node based on the historical resource allocation information, the second memory resource usage information, and the memory resource usage demand includes:

[0025] Find the total memory resources of the computing node and the minimum memory resource requirement of the deployed container from the historical resource allocation information;

[0026] Determine the minimum memory resource requirement and the second maximum value in the second memory resource usage information;

[0027] The free memory resources of the computing node are determined based on the total memory resources, the second maximum value, and the memory resource usage requirements;

[0028] The memory resource idle rate of the computing node is determined based on the free memory resources and the total amount of memory resources.

[0029] In the above data processing method, determining the target computing node from the computing nodes based on the resource idle parameter includes:

[0030] Based on the resource idle parameter, the first computing node with the highest resource idle parameter is determined from the computing nodes, and the first computing node is determined as the target computing node.

[0031] In the above data processing method, the method further includes:

[0032] If the first resource usage information is not found, the second resource usage information corresponding to the preset time period is determined based on the used resources.

[0033] In the above data processing method, before obtaining the first historical resource allocation information of containers already deployed on the computing node within a preset time period upon receiving a deployment request for the target container, the method further includes:

[0034] Every preset time interval, historical resource allocation information of containers deployed on the computing node is obtained and stored in a preset database so that the target computing node can be determined based on the historical resource allocation information when a deployment request is received.

[0035] Secondly, embodiments of this application provide a data processing apparatus, the apparatus comprising:

[0036] The acquisition unit is configured to, upon receiving a deployment request for a target container, acquire historical resource allocation information of containers already deployed on a compute node within a preset time period, and search for first resource usage information of containers already deployed on a compute node within the previous preset time period; the preset time period is the preset time period prior to receiving the deployment request; the historical resource allocation information includes the resources used by containers already deployed on the compute node within the preset time period;

[0037] The determining unit is configured to, when the first resource usage information is found, determine the second resource usage information corresponding to the preset time period based on the used resource and the first resource usage information;

[0038] The determining unit is further configured to determine the resource usage requirements corresponding to the target container, and determine the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information and the resource usage requirements;

[0039] The determining unit is further configured to determine a target computing node from the computing nodes based on the resource idle parameter, and deploy the target container to the target computing node.

[0040] Thirdly, embodiments of this application provide a data processing device, characterized in that the device includes: a processor, a memory, and a communication bus; the processor executes a running program stored in the memory to implement the data processing method as described in any of the above claims.

[0041] Fourthly, embodiments of this application provide a storage medium storing a computer program thereon, characterized in that the computer program, when executed by a processor, implements the data processing method as described in any of the preceding claims.

[0042] This application provides a data processing method, apparatus, and storage medium. The method includes: upon receiving a deployment request for a target container, obtaining historical resource allocation information of containers already deployed on a computing node within a preset time period, and searching for first resource usage information of containers already deployed on the computing node within a previous preset time period; the preset time period is the preset time period before receiving the deployment request; the historical resource allocation information includes the resources used by containers already deployed on the computing node within a preset time period; if the first resource usage information is found, determining second resource usage information corresponding to a preset time period based on the used resources and the first resource usage information; determining the resource usage requirements corresponding to the target container, and determining the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information, and the resource usage requirements; determining the target computing node from the computing nodes based on the resource idle parameters, and deploying the target container to the target computing node. By adopting the above implementation scheme, this application determines the second resource usage information of the historical time period in which the current moment is located based on the used resources of the historical time period in which the current moment is located and the first resource usage information of the previous historical time period in which the current moment is located, and further determines the computing node for deploying the container based on this second resource usage information, thereby achieving the goal of improving the accuracy of container deployment. Attached Figure Description

[0043] Figure 1 A flowchart of a data processing method provided in an embodiment of this application;

[0044] Figure 2 An exemplary flowchart for determining processor resource idle rate is provided for embodiments of this application;

[0045] Figure 3 An exemplary flowchart for determining memory resource free rate is provided for embodiments of this application;

[0046] Figure 4 This is a schematic diagram of the composition structure of a data processing device provided in an embodiment of this application;

[0047] Figure 5 This is a schematic diagram of the composition structure of a data processing device provided in an embodiment of this application. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit the scope of this application.

[0049] This application provides a data processing method applied to a data processing device. Figure 1 A flowchart of a data processing method provided in an embodiment of this application is shown below. Figure 1As shown, data processing methods may include:

[0050] S101. Upon receiving a deployment request for the target container, obtain historical resource allocation information of containers already deployed on the compute node within a preset time period, and search for the first resource usage information of containers already deployed on the compute node within the previous preset time period; the preset time period is the preset time period before the deployment request is received; the historical resource allocation information includes the resources used by containers already deployed on the compute node within a preset time period.

[0051] In this embodiment of the application, when the data processing device receives a deployment request for a target container, it obtains historical resource allocation information of containers already deployed on a computing node within a preset time period, and searches for first resource usage information of containers already deployed on a computing node within the previous preset time period; the preset time period is the preset time period before the deployment request is received; the historical resource allocation information includes the resources used by containers already deployed on a computing node within a preset time period.

[0052] The data processing method proposed in this application can be applied to scenarios where containers are deployed on computing nodes.

[0053] It should be noted that, assuming the deployment request is received at 9:00, and the preset time period is 1 minute, then a preset time period is the period from 8:59 to 9:00, and the preset time period preceding the preset time period is the period from 8:58 to 8:59. The specific length of the preset time period can be determined according to the actual situation, and this application embodiment does not limit it here.

[0054] It should be noted that, in this embodiment of the application, the historical resource allocation information is collected by the container resource monitor in the data processing device every preset time period. Specifically, every preset time period, the historical resource allocation information of the containers deployed on the computing node is obtained and stored in a preset database so that the deployment node can be determined based on the historical resource allocation information when a deployment request is received.

[0055] It should be noted that the preset database can be a distributed key-value pair database, and the specific preset database can be specified according to the actual situation. This application embodiment does not limit it here.

[0056] It should be noted that the historical resource allocation information includes the resource allocation of compute nodes and the containers deployed on the compute nodes, such as the resources used by the deployed containers, the minimum resource requirements of the deployed containers, and the total resources of the compute nodes.

[0057] It should be noted that, in the embodiments of this application, resources include both processor resources and memory resources.

[0058] It should be noted that the first resource usage information can be understood as the resources that the data processing device plans to allocate to the deployed containers within the previous preset time period. The first resource usage information corresponding to the previous preset time period is also determined based on the resource usage information corresponding to the two preset time periods before the previous preset time period and the resources used by the deployed containers within the previous preset time period.

[0059] S102. If the first resource usage information is found, determine the second resource usage information corresponding to a preset time period based on the used resources and the first resource usage information.

[0060] In this embodiment of the application, when the data processing device finds the first resource usage information, it determines the second resource usage information corresponding to a preset time period based on the used resources and the first resource usage information.

[0061] It should be noted that, in this embodiment of the application, the used resources include used processor resources and used memory resources, and the first resource usage information includes first processor resource usage information and first memory resource usage information. Specifically, based on the used processor resources, the first processor resource usage information, and the first preset weight value, a second processor resource usage information corresponding to a preset time period is determined; based on the used memory resources, the first memory resource usage information, and the first preset weight value, a second memory resource usage information corresponding to a preset time period is determined; the second processor resource usage information and the second memory resource usage information are determined as the second resource usage information corresponding to a preset time period.

[0062] For example, the used processor resources are represented as c t1 Then the second processor resource usage information e t1 The calculation formula is shown in formula (1):

[0063] e t1 =α×e t1-1 +(1-α)×c t1 (1)

[0064] In equation (1) above, e t1-1 This refers to the first processor resource usage information, where α is a first preset weight value. In this embodiment, the value of α is 0.9 or 0.99, indicating that e t1-1 For e t1 The degree of influence is 0.9 or 0.99, while c t1 For e t1The degree of influence is 0.1 or 0.01, which means that the planned use of resources for a preset time period is mostly based on the calculated use of resources in the previous time period, and a small part is based on the used resources in a preset time period; the specific value of α can be determined according to the actual situation, and this application embodiment does not limit it here.

[0065] Accordingly, the used memory resources are represented as c t2 Then the second memory resource usage information e t2 The calculation formula is shown in formula (2):

[0066] e t2 =α×e t2-1 +(1-α)×c t2 (2)

[0067] In equation (2) above, e t2-1 This represents the first memory resource usage information, where α is a first preset weight value. In this embodiment, the value of α is 0.9 or 0.99, indicating that e t2-1 For e t2 The degree of influence is 0.9 or 0.99, while c t2 For e t2 The degree of influence is 0.1 or 0.01.

[0068] It should be noted that when calculating the second resource usage information corresponding to the previous preset time period, it can be determined not only based on the first resource usage information of the preset time period, but also based on the third resource usage information and the first resource usage information of the two preset time periods before the preset time period; it can also be determined based on the fourth resource usage information, the third resource usage information and the first resource usage information of the three preset time periods before the two preset time periods, as long as the weight values ​​are added together to 1. The specific determination method can be determined according to the actual situation, and this application embodiment does not limit it here.

[0069] In this embodiment of the application, if the first resource usage information is not found, the second resource usage information corresponding to a preset time period is determined based on the resources already used.

[0070] It should be noted that the situation where the first resource usage information is not found may be that no resource usage information was calculated in the previous time period when the container was deployed. In this case, the second preset resource usage only needs to be obtained based on the first weight value and the used resources. In this case, referring to formulas (1) and (2), since the value of e0 is 0, initially e t The value may be very small and needs to be corrected. An exemplary correction method is given in this application embodiment, as shown in formula (3):

[0071]

[0072] In equation (3) above, the second resource usage information e is obtained based on the first weight value and the used resources. t At that time, it is necessary to deal with e t Make corrections to obtain the corrected version.

[0073] It should be noted that resource usage information can be represented as the resources that have been deployed and are planned to be used by containers within a preset time period.

[0074] S103. Determine the resource usage requirements corresponding to the target container, and determine the resource idle parameters of the computing node based on historical resource allocation information, second resource usage information and resource usage requirements.

[0075] In this embodiment of the application, after determining the second resource usage information, the data processing device determines the resource usage requirements corresponding to the target container, and determines the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information and the resource usage requirements.

[0076] It should be noted that the resource usage requirements of the target container refer to the resources required by the target container at runtime, including the required processor resources and the required memory resources.

[0077] Specifically, the processor resource idle rate of the computing node is determined based on historical resource allocation information, second processor resource usage information, and processor resource usage requirements; the memory resource idle rate of the computing node is determined based on historical resource allocation information, second memory resource usage information, and memory resource usage requirements; and the resource idle parameters of the computing node are determined based on the processor resource idle rate, memory resource idle rate, and second preset weight value.

[0078] This application provides an exemplary flowchart for determining processor resource idle rate, such as... Figure 2 As shown, the method is as follows:

[0079] S201. Find the total processor resources of the computing nodes and the minimum processor resource requirements of the deployed containers from the historical resource allocation information.

[0080] In this embodiment, the data processing device searches for the total processor resources of computing nodes and the minimum processor resource requirements of deployed containers from historical resource allocation information.

[0081] It should be noted that the minimum processor resource requirement is the minimum processor resource required for the deployed container to run.

[0082] S202. Determine the minimum processor resource requirement and the first maximum value in the second processor resource usage information.

[0083] In this embodiment of the application, the data processing device determines the first maximum value in the minimum processor resource requirement and the second processor resource usage information.

[0084] It should be noted that when obtaining the minimum processor resource requirement of the deployed containers, the largest number of resources is selected from the minimum processor resource requirement and the second processor resource usage information as the first maximum value. The second processor resource usage information can be understood as the processor resources that the deployed containers plan to use in the previous preset time period.

[0085] S203. Determine the idle processor resources of the computing node based on the total amount of processor resources, the first maximum value, and the processor resource usage requirements.

[0086] In this embodiment of the application, after obtaining the total amount of processor resources, the first maximum value, and the processor resource usage requirements, the data processing device determines the idle processor resources of the computing node based on the total amount of processor resources, the first maximum value, and the processor resource usage requirements.

[0087] For example, the second processor resource usage information is e t1 Then the processor's idle resources M cpu The calculation formula is shown in formula (4):

[0088] M cpu =CAP cpu -∑max(r1,e t1 )-S cpu (4)

[0089] In equation (4) above, CAP cpu R represents the total amount of processor resources, r1 represents the minimum processor resource requirement, and S represents the minimum processor resource requirement. cpu This represents the processor resource usage requirements.

[0090] S204. Determine the processor resource idle rate of the computing node based on the processor idle resources and the total amount of processor resources.

[0091] In this embodiment of the application, after determining the idle resources of the processor, the data processing device determines the processor resource idle rate of the computing node based on the idle resources of the processor and the total amount of processor resources.

[0092] For example, processor resource idle rate V cpu The calculation formula is shown in formula (5):

[0093]

[0094] This application provides an exemplary flowchart for determining the memory resource idle rate, as shown in the embodiments. Figure 3 As shown, the method is as follows:

[0095] S301. Find the total memory resources of the compute nodes and the minimum memory resource requirements of the deployed containers from the historical resource allocation information.

[0096] In this embodiment of the application, the data processing device searches for the total memory resources of the computing nodes and the minimum memory resource requirements of the deployed containers from historical resource allocation information.

[0097] It should be noted that the minimum memory resource requirement is the minimum memory resource required by the deployed container at runtime.

[0098] S302. Determine the minimum memory resource requirement and the second maximum value in the second memory resource usage information.

[0099] In this embodiment of the application, the data processing device determines the minimum memory resource requirement and the second maximum value in the second memory resource usage information.

[0100] It should be noted that when obtaining the minimum memory resource requirement of the deployed containers, the largest resource number is selected from the minimum memory resource requirement and the second memory resource usage information as the second maximum value. The second memory resource usage information can be understood as the memory resources that the deployed containers plan to use in the previous preset time period.

[0101] S303. Determine the free memory resources of the computing node based on the total memory resources, the second maximum value, and the memory resource usage requirements.

[0102] In this embodiment of the application, after obtaining the total amount of memory resources, the second maximum value, and the memory resource usage requirements, the data processing device determines the free memory resources of the computing node based on the total amount of memory resources, the second maximum value, and the memory resource usage requirements.

[0103] For example, the second memory resource usage information is e t2 Then the free memory resource M memory The calculation formula is shown in formula (6):

[0104] M memory =CAP memory -∑max(r2,e t2 )-S memory (6)

[0105] In equation (6) above, CAP memory Let r1 represent the total amount of memory resources, r2 represent the minimum memory resource requirement, and S represent the minimum memory resource requirement. memoryThis represents the memory resource usage requirement.

[0106] S304. Determine the memory resource idle rate of the computing node based on the free memory resources and the total amount of memory resources.

[0107] In this embodiment of the application, after determining the free memory resources, the data processing device determines the memory resource idle rate of the computing node based on the free memory resources and the total amount of memory resources.

[0108] For example, memory resource idle rate V memory The calculation formula is shown in formula (7):

[0109]

[0110] In this embodiment of the application, after obtaining the processor resource idle rate and the memory resource idle rate, the resource idle parameter N is calculated according to formula (8):

[0111]

[0112] In the above formula (8), λ is the second weight value. In this embodiment of the application, the second weight value λ is 10. The specific second weight value can be specified according to the actual situation. This embodiment of the application does not limit it here.

[0113] S104. Determine the target computing node from the computing nodes based on the resource idle parameters, and deploy the target container to the target computing node.

[0114] In this embodiment of the application, after determining the resource idle parameters, the data processing device determines the target computing node from the computing nodes based on the resource idle parameters, and deploys the target container to the target computing node.

[0115] Specifically, based on the resource idle parameter, the first computing node with the highest resource idle parameter is determined from the computing nodes, and the first computing node is determined as the target computing node.

[0116] It should be noted that after the target computing node is determined, the scheduler in the data processing device will deploy the target container to the target computing node.

[0117] This application provides a data processing method, which includes: upon receiving a deployment request for a target container, obtaining historical resource allocation information of containers already deployed on a computing node within a preset time period, and searching for first resource usage information of containers already deployed on a computing node within the previous preset time period; the preset time period is the preset time period before receiving the deployment request; the historical resource allocation information includes the resources used by containers already deployed on the computing node within a preset time period; if the first resource usage information is found, determining second resource usage information corresponding to the preset time period based on the used resources and the first resource usage information; determining the resource usage requirements corresponding to the target container, and determining the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information, and the resource usage requirements; determining the target computing node from the computing nodes based on the resource idle parameters, and deploying the target container to the target computing node; by adopting the above implementation scheme, this application determines the second resource usage information of the historical time period in which the current moment is located based on the used resources of the historical time period in which the current moment is located and the first resource usage information of the previous historical time period in which the current moment is located, and further determines the computing node for deploying the container based on this second resource usage information, which can achieve the purpose of improving the accuracy of container deployment.

[0118] Based on the above embodiments, this application provides a specific embodiment to exemplify the application of this application in a practical situation.

[0119] By adding the scheduled task CronJob and the container resource monitor cAdvisor to the existing container deployment system, the index-weighted average value of container resources e at time t is collected, calculated, and stored. t (Resource usage information), the scheduler in the container deployment system is based on e t Calculate the resource availability parameters for all nodes, and deploy containers based on these parameters. The specific process is as follows:

[0120] 1. The scheduled task CronJob calls the container resource monitor cAdvisor every minute. cAdvisor collects the current resource usage information of all containers and stores it in ETCD.

[0121] 2. The scheduled task CronJob calculates an index-weighted average value e based on current resource usage information. t And store it in ETCD;

[0122] 3. The scheduler obtains current resource usage information (r1, r2, c) through ETCD. t1 c t2 CAP cpu CAPmemory S cpu and S memory (etc.), and then calculate sequentially based on the current resource usage information. V cpu V memory and N.

[0123] 4. The scheduler selects the node with the highest resource idle parameter N as the node for deploying containers.

[0124] Based on the above embodiments, in another embodiment of this application, a data processing device 1 is provided. Figure 4 This application provides a schematic diagram of the composition of a data processing device, as shown below. Figure 4 As shown, the data processing device 1 includes:

[0125] The acquisition unit 10 is configured to, upon receiving a deployment request for a target container, acquire historical resource allocation information of containers already deployed on a compute node within a preset time period, and search for first resource usage information of containers already deployed on a compute node within the previous preset time period; the preset time period is the preset time period prior to receiving the deployment request; the historical resource allocation information includes the resources used by containers already deployed on the compute node within the preset time period.

[0126] Determining unit 11 is used to determine second resource usage information corresponding to the preset time period based on the used resources and the first resource usage information when the first resource usage information is found;

[0127] The determining unit 11 is further configured to determine the resource usage requirements corresponding to the target container, and determine the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information and the resource usage requirements;

[0128] The determining unit 11 is further configured to determine a target computing node from the computing nodes based on the resource idle parameter, and deploy the target container to the target computing node.

[0129] Optionally, the determining unit 11 is further configured to determine the second processor resource usage information corresponding to the preset time period based on the used processor resources, the first processor resource usage information, and the first preset weight value;

[0130] The determining unit 11 is further configured to determine the second memory resource usage information corresponding to the preset time period based on the used memory resources, the first memory resource usage information, and the first preset weight value;

[0131] The determining unit 11 is further configured to determine the second processor resource usage information and the second memory resource usage information as the second resource usage information corresponding to the preset time period.

[0132] Optionally, the determining unit 11 is further configured to determine the processor resource idle rate of the computing node based on the historical resource allocation information, the second processor resource usage information, and the processor resource usage requirements;

[0133] The determining unit 11 is further configured to determine the memory resource idle rate of the computing node based on the historical resource allocation information, the second memory resource usage information, and the memory resource usage requirements;

[0134] The determining unit 11 is further configured to determine the resource idle parameters of the computing node based on the processor resource idle rate, the memory resource idle rate, and the second preset weight value.

[0135] Optionally, the data processing device 1 further includes: a lookup unit;

[0136] The search module is used to search for the total amount of processor resources of the computing node and the minimum processor resource requirement of the deployed container from the historical resource allocation information.

[0137] The determining unit 11 is further configured to determine the first maximum value in the minimum processor resource requirement and the second processor resource usage information;

[0138] The determining unit 11 is further configured to determine the idle processor resources of the computing node based on the total amount of processor resources, the first maximum value, and the processor resource usage requirements;

[0139] The determining unit 11 is further configured to determine the processor resource idle rate of the computing node based on the processor idle resources and the total amount of processor resources.

[0140] Optionally, the search module is further configured to search for the total memory resources of the computing node and the minimum memory resource requirement of the deployed container from the historical resource allocation information;

[0141] The determining unit 11 is further configured to determine the minimum memory resource requirement and the second maximum value in the second memory resource usage information;

[0142] The determining unit 11 is further configured to determine the free memory resources of the computing node based on the total memory resources, the second maximum value, and the memory resource usage requirements;

[0143] The determining unit 11 is further configured to determine the memory resource idle rate of the computing node based on the memory free resources and the total amount of memory resources.

[0144] Optionally, the determining unit 11 is further configured to determine the first computing node with the highest resource idle parameter from the computing nodes based on the resource idle parameter, and determine the first computing node as the target computing node.

[0145] Optionally, the determining unit 11 is further configured to determine the second resource usage information corresponding to the preset time period based on the used resources if the first resource usage information is not found.

[0146] Optionally, the acquisition unit 10 is further configured to acquire historical resource allocation information of containers deployed on the computing node at preset time intervals, and store the historical resource allocation information in a preset database so as to determine the deployment node based on the historical resource allocation information when a deployment request is received.

[0147] This application provides a data processing apparatus, comprising an acquisition unit and a determination unit. The acquisition unit, upon receiving a deployment request for a target container, acquires historical resource allocation information of containers already deployed on a computing node within a preset time period, and searches for first resource usage information of containers already deployed on a computing node within the previous preset time period. The determination unit, upon finding the first resource usage information, determines second resource usage information corresponding to a preset time period based on the used resources and the first resource usage information; determines the resource usage requirements corresponding to the target container, and determines resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information, and the resource usage requirements; determines the target computing node from among the computing nodes based on the resource idle parameters, and deploys the target container onto the target computing node. By adopting the above implementation scheme, this application determines the second resource usage information of the historical time period in which the current moment is located based on the used resources of the historical time period in which the current moment is located and the first resource usage information of the previous historical time period in which the current moment is located, and further determines the computing node for deploying the container based on this second resource usage information, thereby improving the accuracy of container deployment.

[0148] Figure 5 This is a schematic diagram of the composition structure of a data processing device provided in an embodiment of this application. In practical applications, based on the same disclosed concept of the above embodiments, such as... Figure 5 As shown, the data processing device 2 in this embodiment includes: a processor 20, a memory 21, and a communication bus 22.

[0149] In a specific embodiment, the acquisition unit 10, the determination unit 11, and the search unit described above can be implemented by a processor 20 located on the data processing device 2. The processor 20 can be at least one of the following: an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field-programmable gate array (FPGA), a CPU, a controller, a microcontroller, or a microprocessor. It is understood that for different data processing devices, the electronic device used to implement the above processor functions can also be other types; this embodiment does not specifically limit the specific implementation.

[0150] In this embodiment, the communication bus 22 is used to establish communication between the processor 20 and the memory 21; when the processor 20 executes the running program stored in the memory 21, it implements the following method:

[0151] Upon receiving a deployment request for the target container, the system obtains historical resource allocation information of containers already deployed on the compute node within a preset time period, and searches for the first resource usage information of containers already deployed on the compute node within the previous preset time period. The preset time period is the preset time period prior to receiving the deployment request. The historical resource allocation information includes the resources used by containers already deployed on the compute node within the preset time period.

[0152] If the first resource usage information is found, the second resource usage information corresponding to the preset time period is determined based on the used resources and the first resource usage information.

[0153] Determine the resource usage requirements corresponding to the target container, and determine the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information, and the resource usage requirements;

[0154] Based on the resource availability parameter, a target computing node is determined from the computing nodes, and the target container is deployed to the target computing node.

[0155] Optionally, the processor 20 is further configured to determine second processor resource usage information corresponding to the preset time period based on the used processor resources, the first processor resource usage information, and the first preset weight value; determine second memory resource usage information corresponding to the preset time period based on the used memory resources, the first memory resource usage information, and the first preset weight value; and determine the second processor resource usage information and the second memory resource usage information as the second resource usage information corresponding to the preset time period.

[0156] Optionally, the processor 20 is further configured to determine the processor resource idle rate of the computing node based on the historical resource allocation information, the second processor resource usage information, and the processor resource usage requirements; determine the memory resource idle rate of the computing node based on the historical resource allocation information, the second memory resource usage information, and the memory resource usage requirements; and determine the resource idle parameter of the computing node based on the processor resource idle rate, the memory resource idle rate, and a second preset weight value.

[0157] Optionally, the processor 20 is further configured to: find the total processor resources of the computing node and the minimum processor resource requirement of the deployed container from the historical resource allocation information; determine the minimum processor resource requirement and a first maximum value in the second processor resource usage information; determine the processor idle resources of the computing node based on the total processor resources, the first maximum value and the processor resource usage requirement; and determine the processor idle rate of the computing node based on the processor idle resources and the total processor resources.

[0158] Optionally, the processor 20 is further configured to: find the total memory resources of the computing node and the minimum memory resource requirement of the deployed container from the historical resource allocation information; determine the minimum memory resource requirement and the second maximum value in the second memory resource usage information; determine the memory free resources of the computing node based on the total memory resources, the second maximum value and the memory resource usage requirement; and determine the memory resource idle rate of the computing node based on the memory free resources and the total memory resources.

[0159] Optionally, the processor 20 is further configured to determine, based on the resource idle parameter, a first computing node with the highest resource idle parameter from the computing nodes, and to determine the first computing node as the target computing node.

[0160] Optionally, the processor 20 is further configured to determine the second resource usage information corresponding to the preset time period based on the used resources if the first resource usage information is not found.

[0161] Optionally, the processor 20 is further configured to acquire historical resource allocation information of containers deployed on computing nodes at preset time intervals, and store the historical resource allocation information in a preset database so as to determine the deployment node based on the historical resource allocation information when a deployment request is received.

[0162] This application provides a storage medium storing a computer program thereon. The computer-readable storage medium stores one or more programs, which can be executed by one or more processors and applied in a data processing device. The computer program implements the data processing method described above.

[0163] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0164] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause an image display device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the data processing methods described in the various embodiments of this disclosure.

[0165] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. A data processing method, characterized by, The method includes: Upon receiving a deployment request for the target container, the system obtains historical resource allocation information of containers already deployed on the compute node within a preset time period, and searches for first resource usage information of containers already deployed on the compute node within the previous preset time period. The preset time period is the preset time period prior to receiving the deployment request. The historical resource allocation information includes the resources used by containers already deployed on the compute node within the preset time period. The first resource usage information is the resources planned to be allocated to the deployed container within the previous preset time period. If the first resource usage information is found, the second resource usage information corresponding to the preset time period is determined based on the used resources and the first resource usage information; the second resource usage information is the resources planned to be allocated to the deployed container within the preset time period. The resource usage requirements corresponding to the target container are determined, and the resource idle parameters of the computing node are determined based on the historical resource allocation information, the second resource usage information, and the resource usage requirements; the resource idle parameters are determined based on the processor resource idle rate of the computing node, the memory resource idle rate of the computing node, and a second preset weight value. Based on the resource availability parameter, a target computing node is determined from the computing nodes, and the target container is deployed to the target computing node.

2. The method of claim 1, wherein, The used resources include used processor resources and used memory resources, and the first resource usage information includes first processor resource usage information and first memory resource usage information; determining the second resource usage information corresponding to the preset time period based on the used resources and the first resource usage information includes: The second processor resource usage information corresponding to the preset time period is determined based on the used processor resources, the first processor resource usage information, and the first preset weight value. Based on the used memory resources, the first memory resource usage information, and the first preset weight value, determine the second memory resource usage information corresponding to the preset time period; The second processor resource usage information and the second memory resource usage information are determined as the second resource usage information corresponding to the preset time period.

3. The method of claim 1, wherein, The second resource usage information includes second processor resource usage information and second memory resource usage information. The resource usage requirements include processor resource usage requirements and memory resource usage requirements. Determining the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information, and the resource usage requirements includes: The processor resource idle rate of the computing node is determined based on the historical resource allocation information, the second processor resource usage information, and the processor resource usage requirements. The memory resource idle rate of the computing node is determined based on the historical resource allocation information, the second memory resource usage information, and the memory resource usage requirements. The resource idle parameters of the computing node are determined based on the processor resource idle rate, the memory resource idle rate, and the second preset weight value.

4. The method according to claim 3, characterized in that, Determining the processor resource idle rate of the computing node based on the historical resource allocation information, the second processor resource usage information, and the processor resource usage demand includes: Find the total processor resources of the computing node and the minimum processor resource requirement of the deployed container from the historical resource allocation information; Determine the first maximum value between the minimum processor resource requirement and the second processor resource usage information; The idle processor resources of the computing node are determined based on the total amount of processor resources, the first maximum value, and the processor resource usage requirements. The processor resource idle rate of the computing node is determined based on the processor idle resources and the total amount of processor resources.

5. The method according to claim 3, characterized in that, Determining the memory resource idle rate of the computing node based on the historical resource allocation information, the second memory resource usage information, and the memory resource usage demand includes: Find the total memory resources of the computing node and the minimum memory resource requirement of the deployed container from the historical resource allocation information; Determine the minimum memory resource requirement and the second maximum value in the second memory resource usage information; The free memory resources of the computing node are determined based on the total memory resources, the second maximum value, and the memory resource usage requirements; The memory resource idle rate of the computing node is determined based on the free memory resources and the total amount of memory resources.

6. The method according to claim 1, characterized in that, The step of determining the target computing node from the computing nodes based on the resource idle parameter includes: Based on the resource idle parameter, the first computing node with the highest resource idle parameter is determined from the computing nodes, and the first computing node is determined as the target computing node.

7. The method according to claim 1, characterized in that, The method further includes: If the first resource usage information is not found, the second resource usage information corresponding to the preset time period is determined based on the used resources.

8. The method according to claim 1, characterized in that, Before obtaining the first historical resource allocation information of containers already deployed on a compute node within a preset time period upon receiving a deployment request for the target container, the method further includes: Every preset time interval, historical resource allocation information of containers deployed on the computing node is obtained and stored in a preset database so that the target computing node can be determined based on the historical resource allocation information when a deployment request is received.

9. A data processing apparatus, characterized in that, The device includes: The acquisition unit is configured to, upon receiving a deployment request for a target container, acquire historical resource allocation information of containers already deployed on a compute node within a preset time period, and search for first resource usage information of containers already deployed on a compute node within a previous preset time period; the preset time period is the preset time period prior to receiving the deployment request; the historical resource allocation information includes the resources used by containers already deployed on the compute node within the preset time period; the first resource usage information is the resources planned to be allocated to the deployed containers within the previous preset time period. The determining unit is configured to, upon finding the first resource usage information, determine the second resource usage information corresponding to the preset time period based on the used resources and the first resource usage information; the second resource usage information is the resources planned to be allocated to the deployed container within the preset time period; The determining unit is further configured to determine the resource usage requirements corresponding to the target container, and determine the resource idle parameters of the computing node based on the historical resource allocation information, the second resource usage information and the resource usage requirements; the resource idle parameters are determined based on the processor resource idle rate of the computing node, the memory resource idle rate of the computing node and a second preset weight value; The determining unit is further configured to determine a target computing node from the computing nodes based on the resource idle parameter, and deploy the target container to the target computing node.

10. A data processing device, characterized in that, The device includes: a processor, a memory, and a communication bus; when the processor executes the running program stored in the memory, it implements the method as described in any one of claims 1-8.

11. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-8.

Citation Information

Patent Citations

  • A dynamic scheduling method and device for a container cluster management system

    CN109710376A