Resource allocation method, device and system for image processor

Abstract

The invention discloses a resource allocation method, device and system for an image processor, and belongs to the technical field of computer resource allocation. The method comprises the steps that a driving interface of an image processor is called to execute a calculation program, calculation resource usage amount corresponding to execution of the calculation program is obtained, a sampling data point set is obtained, and each sampling point in the sampling data point set corresponds to one calculation resource usage amount; determining a resource use prediction function of the computing program based on the sampling data point set; determining a target resource usage amount of the calculation program corresponding to a target moment based on the resource usage prediction function; and based on the target resource usage amount, allocating image processing resources to the computing program. The resource usage amount of the calculation program does not need to be sampled regularly in the whole execution process of the calculation program, the influence on hardware performance can be reduced, and the resource allocation effect is ensured.

Description

Technical Field

[0001] The present application belongs to the technical field of computer resource allocation, and in particular, relates to a resource allocation method, device and system for an image processor. Background Art

[0002] Graphics Processing Unit (GPU) has a highly parallel structure and can process large amounts of data and perform a large amount of parallel computing at the same time. GPU can efficiently handle complex computing tasks and has strong large-scale parallel computing capabilities. In order to improve GPU resource utilization and reduce hardware costs, GPUs are usually virtualized, that is, a physical GPU is virtualized into multiple computing instances, so that multiple computing instances can perform computing tasks in multiple virtual machines or containers at the same time.

[0003] Currently, during the execution of a program, the GPU resources used by the program are usually sampled at regular intervals, and resources are allocated to the program based on the resource usage obtained through the sampling. However, when the sampling interval is short, it may affect hardware performance, and when the sampling interval is long, it may affect resource allocation. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a resource allocation method, device and system for an image processor, which can reduce the impact on hardware performance and ensure the resource allocation effect.

[0005] In a first aspect, the present application provides a resource allocation method for an image processor, the method comprising:

[0006] Calling a driver interface of an image processor to execute a computing program, obtaining a computing resource usage corresponding to executing the computing program, and obtaining a sampling data point set, wherein each sampling point in the sampling data point set corresponds to one computing resource usage;

[0007] determining a resource usage prediction function for the computing program based on the set of sampled data points;

[0008] Determining a target resource usage of the computing program corresponding to a target time based on the resource usage prediction function;

[0009] Image processing resources are allocated to the computing program based on the target resource usage.

[0010] According to the resource allocation method of the image processor of the present application, the computing resource usage corresponding to the computing program is sampled in advance to obtain a sampling data point set, and the resource usage prediction function of the computing program is obtained based on the sampling data point set. After obtaining the resource usage prediction function, the target resource usage of the computing program at the target time can be determined through the resource usage prediction function, and image processing resources are allocated to the computing program. There is no need to perform periodic sampling of the resource usage of the computing program during the entire execution of the computing program, which can reduce the impact on hardware performance and ensure the resource allocation effect.

[0011] According to one embodiment of the present application, determining the resource usage prediction function of the computing program based on the sampled data point set includes:

[0012] constructing a resource usage curve based on the set of sampled data points;

[0013] The resource usage prediction function is determined based on the slope change rate of the resource usage curve.

[0014] According to an embodiment of the present application, determining the resource usage prediction function based on the slope change rate of the resource usage curve includes:

[0015] When the slope change rate is greater than or equal to a change rate threshold, calculating a first reference value of the sampling point in the sampling data point set;

[0016] Compare each of the sampling points with the first reference value to obtain a first target data point of the sampling data point set;

[0017] Calculating a second reference value of the first target data point;

[0018] Compare each of the first target data points with the second reference value to obtain a second target data point of the sampled data point set;

[0019] Based on the second target data point, the resource usage prediction function is determined by polynomial fitting.

[0020] According to an embodiment of the present application, determining the resource usage prediction function based on the second target data point by polynomial fitting includes:

[0021] segmenting the resource usage curve based on the second target data point;

[0022] Polynomial fitting is performed on each section of the resource usage curve to determine the resource usage prediction function.

[0023] According to an embodiment of the present application, determining the resource usage prediction function based on the slope change rate of the resource usage curve includes:

[0024] When the slope change rate is less than the change rate threshold, the resource usage prediction function is determined by polynomial fitting.

[0025] According to an embodiment of the present application, allocating image processing resources to the computing program based on the target resource usage includes:

[0026] When the target resource usage is greater than a first threshold, controlling the computing program to wait for the image processor to release resources;

[0027] Alternatively, when the target resource usage is less than or equal to the first threshold, the image processing resource is allocated to the computing program.

[0028] According to one embodiment of the present application, the step of calling a driver interface of an image processor to execute the calculation program includes:

[0029] When the current resource usage of the computing program is greater than a second threshold, controlling the computing program to wait for the image processor to release resources;

[0030] In a case where the current resource usage of the computing program is less than or equal to a second threshold, the computing program is executed.

[0031] In a second aspect, the present application provides a resource allocation device for an image processor, the device comprising:

[0032] A first processing module is used to call a driving interface of an image processor to execute a computing program, obtain a computing resource usage corresponding to the execution of the computing program, and obtain a sampling data point set, wherein each sampling point in the sampling data point set corresponds to one computing resource usage;

[0033] A second processing module, configured to determine a resource usage prediction function of the computing program based on the set of sampled data points;

[0034] A third processing module, configured to determine a target resource usage of the computing program corresponding to a target time based on the resource usage prediction function;

[0035] A fourth processing module is used to allocate image processing resources to the computing program based on the target resource usage.

[0036] According to the resource allocation device of the image processor of the present application, the computing resource usage corresponding to the computing program is sampled in advance to obtain a sampling data point set, and the resource usage prediction function of the computing program is obtained based on the sampling data point set. After obtaining the resource usage prediction function, the target resource usage of the computing program at the target time can be determined through the resource usage prediction function, and image processing resources are allocated to the computing program. There is no need to perform periodic sampling of the resource usage of the computing program during the entire execution of the computing program, which can reduce the impact on hardware performance and ensure the resource allocation effect.

[0037] In a third aspect, the present application provides a resource allocation system for an image processor, the system comprising:

[0038] A program application module, the program application module is used to receive a user's program execution request;

[0039] a resource scheduling module, the resource scheduling module being connected to the program application module, and the resource scheduling module being configured to execute the resource allocation method for the image processor described in the first aspect above when receiving the program execution request;

[0040] A resource driver library stores image processing resources. The resource driver library is connected to the resource scheduling module. The resource scheduling module is used to schedule the image processing resources in the resource driver library.

[0041] According to the resource allocation system of the image processor of the present application, the computing resource usage corresponding to the computing program is sampled in advance to obtain a sampling data point set, and the resource usage prediction function of the computing program is obtained based on the sampling data point set. After obtaining the resource usage prediction function, the target resource usage of the computing program at the target time can be determined through the resource usage prediction function, and image processing resources are allocated to the computing program. There is no need to perform periodic sampling of the resource usage of the computing program during the entire execution of the computing program, which can reduce the impact on hardware performance and ensure the resource allocation effect.

[0042] In a fourth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the resource allocation method for the image processor as described in the first aspect above is implemented.

[0043] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the resource allocation method for the image processor as described in the first aspect above is implemented.

[0044] In a sixth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the resource allocation method for the image processor as described in the first aspect above.

[0045] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0047] Figure 1 This is one of the flowcharts of the resource allocation method of the image processor provided in the embodiment of the present application;

[0048] Figure 2 It is a schematic diagram of resource usage sampling in related art;

[0049] Figure 3 is a schematic diagram of a resource usage curve provided in an embodiment of the present application;

[0050] Figure 4 It is one of the schematic diagrams of fitting results corresponding to the resource usage curve provided in the embodiment of the present application;

[0051] Figure 5 This is the second schematic diagram of the fitting result corresponding to the resource usage curve provided in the embodiment of the present application;

[0052] Figure 6 is a structural schematic diagram of a resource allocation device for an image processor provided in an embodiment of the present application;

[0053] Figure 7 is a structural diagram of a resource allocation system for an image processor provided in an embodiment of the present application;

[0054] Figure 8 This is a second flow chart of a resource allocation method for an image processor provided in an embodiment of the present application;

[0055] Fig. 9 is a schematic diagram of utilization of an image processor provided in an embodiment of the present application;

[0056] Fig.10 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0057] Reference numerals:

[0058] Resource allocation system 700 of image processor, program application module 710, resource scheduling module 720,

[0059] Resource driver library 730. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0061] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0062] In conjunction with the accompanying drawings, the resource allocation method of the image processor, the resource allocation device of the image processor, the resource allocation system 700 of the image processor and the electronic device provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0063] The resource allocation method of the image processor may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0064] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0065] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0066] The resource allocation method for an image processor provided in an embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the resource allocation method for the image processor. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The resource allocation method for the image processor provided in the embodiment of the present application is described below using an electronic device as an example of the execution subject.

[0067] like Figure 1 As shown, the resource allocation method of the image processor includes: step 110 to step 140.

[0068] Step 110: Call the driver interface of the image processor to execute the calculation program, obtain the calculation resource usage corresponding to the execution of the calculation program, and obtain a sampling data point set, where each sampling point in the sampling data point set corresponds to a calculation resource usage.

[0069] The computing program is a program that a user requests to be executed on a graphics processing unit (GPU), and the computing program is used to execute corresponding computing tasks.

[0070] For example, the computing program may be a program corresponding to an artificial intelligence (AI) algorithm, and the computing program may perform reasoning tasks.

[0071] In this embodiment, the driving interface of the image processor is an interface for the computing program to interact with the image processor and obtain resources of the image processor.

[0072] It should be noted that the driver interface of the image processor may be called at a first time interval within a target period after receiving the computing program execution request, and the computing program may be executed each time the driver interface is called.

[0073] The target period may be a set period, and the target period may start from the moment when the computing program execution request is received. The duration of the target period may be preset according to the periodic characteristics of the computing program execution.

[0074] When the volatility of the computing program is large, the duration corresponding to the target period can also be adjusted according to the processing power of the GPU.

[0075] In this embodiment, the first time interval may be a set value, and the first time interval is the time interval between two calls to the driving interface to execute the calculation program within the target time period.

[0076] In this embodiment, each time the driver interface is called to execute a computing program, the computing resource usage corresponding to the execution of the computing program is obtained, wherein the computing resource usage is the amount of resources of the image processor occupied when the computing program is executed.

[0077] In this embodiment, within the target time period, the driver interface of the image processor is called at a first time interval to obtain the computing resource usage corresponding to each execution of the computing program, and the computing resource usage obtained each time is used as a sampling point to obtain a sampling data point set, and each sampling point in the sampling data point set corresponds to a computing resource usage.

[0078] Step 120: Determine a resource usage prediction function of the computing program based on the set of sampled data points.

[0079] The resource usage prediction function is a function for predicting the resource usage of a computing program.

[0080] In this step, fitting can be performed based on the sampling points in the sampling data point set, or the sampling points in the sampling data point set can be input into a model such as a neural network to obtain a resource usage prediction function.

[0081] It is understandable that a computing program may include multiple execution cycles, and the resource usage prediction function may characterize the correspondence between the resource usage of the computing program and the execution time of the computing program in one or more computing program execution cycles.

[0082] Step 130: Determine the target resource usage of the computing program corresponding to the target time based on the resource usage prediction function.

[0083] The target time may be any time during the execution of the computing program after the resource usage prediction function is obtained, and the target time may be the current time corresponding to the real-time execution of the computing program.

[0084] The target resource usage is predicted by the resource usage prediction function, which calculates the resource usage of the program at the target time.

[0085] In this step, the target time may be input into a resource usage prediction function to obtain a target resource usage amount corresponding to the target time.

[0086] Step 140: Allocate image processing resources to the computing program based on the target resource usage.

[0087] The image processing resources are resources in the image processor used to execute computing programs.

[0088] In this step, according to the target resource usage, an amount of image processing resources corresponding to the target resource usage may be allocated to the computing program, or the computing program may be controlled to wait for the image processor to release resources.

[0089] In the related art, during the execution of a program, the GPU resources used by the program are usually sampled at regular intervals, and resources are allocated to the program based on the resource usage obtained through the sampling. However, when the sampling interval is short, the hardware performance may be affected.

[0090] In an embodiment of the present application, after receiving a request to execute a computing program, the driver interface of the image processor is called at a first time interval within a target time period to execute the computing program, and the computing resource usage corresponding to each execution of the computing program is obtained to obtain a set of sampling data points. Based on the set of sampling data points, a resource usage prediction function of the computing program can be obtained. After obtaining the resource usage prediction function, the target resource usage of the computing program at the target time can be determined through the resource usage prediction function, and image processing resources can be allocated to the computing program. There is no need to perform timed sampling of the resource usage of the computing program during the entire execution of the computing program, which can reduce the impact on hardware performance.

[0091] In the related art, the GPU resources used by the program are sampled at regular intervals, and resources are allocated to the program based on the resource usage obtained through the sampling. Within a sampling period, the resource allocation remains unchanged. This is not suitable for programs whose resource usage changes in a short period of time, which affects the resource allocation effect.

[0092] like Figure 2 As shown in the figure, taking video memory sampling as an example, the GPU resources used by the program are sampled once every time period, and the resource usage value obtained by sampling is used as the resource upper limit threshold of the program in this execution cycle. If the resource application amount of the program is greater than the threshold, it will be suspended and waited. Figure 2 The horizontal axis t represents time, and the vertical axis mem represents the resource usage value.

[0093] When the sampling period is set too long, the sampling data is distorted and cannot reflect the actual operation status, affecting resource allocation and use. Figure 2 At the midpoint A, there is a peak in resource requests, but the resource upper limit threshold selected at this time is T1. Therefore, at time A, the program will be forced to pause, even though the system has sufficient resources to use at this time. Figure 2 At midpoint B, the resource request reaches a low point. However, since the resource upper limit threshold selected at this time is T3, even though the current program does not need T3-level resources, the system will still reserve this amount of resources for the program. The resources that other programs on the same GPU card can apply for will be reduced accordingly.

[0094] In an embodiment of the present application, an accurate resource usage prediction function can be obtained through prediction based on the periodic characteristics of the computing program, and the target resource usage can be predicted in real time according to the resource usage prediction function. The obtained target resource usage is accurate, and image processing resources are allocated to the computing program based on the target resource usage. This can reduce unnecessary waiting and at the same time reduce the occupation of resources requested by other computing programs, thereby ensuring the resource allocation effect.

[0095] According to the resource allocation method of the image processor provided in the embodiment of the present application, the computing resource usage corresponding to the computing program is sampled in advance to obtain a sampling data point set, and the resource usage prediction function of the computing program is obtained based on the sampling data point set. After the resource usage prediction function is obtained, the target resource usage of the computing program at the target time can be determined through the resource usage prediction function, and image processing resources are allocated to the computing program. There is no need to perform periodic sampling of the resource usage of the computing program during the entire execution of the computing program, which can reduce the impact on hardware performance and ensure the resource allocation effect.

[0096] In some embodiments, determining a resource usage prediction function of a computing program based on a set of sampled data points includes:

[0097] Based on the set of sampled data points, a resource usage curve is constructed;

[0098] A resource usage prediction function is determined based on the rate of change of the slope of the resource usage curve.

[0099] The resource usage curve is a curve that characterizes the changes in the sampling points in the sampling data point set, that is, a curve that characterizes the changes in the usage of the computing resources corresponding to the computing program.

[0100] In this embodiment, the time may be used as the horizontal axis and the calculated resource usage may be used as the vertical axis. The position of each sampling point may be determined in a two-dimensional coordinate system, and adjacent sampling points may be connected to obtain a resource usage curve.

[0101] In this embodiment, the slope change rate is the frequency of slope changes in the resource usage curve, and the slope change rate can represent a sudden change in the resource usage curve.

[0102] For example, Figure 5 As shown in Figure 2, the resource usage curve is Figure 5 The solid lines in the figure correspond to 4 different slopes, and the slope change rate is 4.

[0103] In this embodiment, the resource usage prediction function may be determined in different ways when the slope change rates are different.

[0104] In some embodiments, determining a resource usage prediction function based on a slope change rate of a resource usage curve includes:

[0105] When the slope change rate is greater than or equal to the change rate threshold, calculating a first reference value of the sampling point in the sampling data point set;

[0106] Compare each sampling point with a first reference value to obtain a first target data point of the sampling data point set;

[0107] calculating a second reference value of the first target data point;

[0108] Compare each first target data point with a second reference value to obtain a second target data point of the sampled data point set;

[0109] Based on the second target data point, a resource usage prediction function is determined by polynomial fitting.

[0110] Among them, the change rate threshold is a preset value. If the slope change rate is greater than or equal to the change rate threshold, it can indicate that there are more mutations in the resource usage curve. If the slope change rate is less than the change rate threshold, it can indicate that there are fewer mutations in the resource usage curve.

[0111] In this embodiment, when the slope change rate is greater than or equal to the change rate threshold, that is, when there are more mutations in the resource usage curve, the first reference value of the sampling point in the sampling data point set is calculated.

[0112] The first reference value is a value used to screen the sampling points. The first reference value may be calculated based on the sampling points in the sampling data point set. For example, the first reference value may be an average value of all the sampling points in the sampling data point set.

[0113] In this embodiment, each sampling point in the sampling data point set is compared with the first reference value to screen out a first target data point, where the first target data point is a point screened out from the sampling points.

[0114] In actual implementation, sampling points that are greater than or equal to the first reference value may be retained as first target data points.

[0115] In this embodiment, after the first target data point is obtained, the second reference value of the first target data point is calculated, wherein the second reference value is a value used to screen the first target data point.

[0116] The second reference value may be calculated based on the first target data points. For example, the second reference value may be an average value of all first target data points.

[0117] In this embodiment, each first target data point is compared with the second reference value to screen out the second target data point, where the second target data point is the point screened out from the first target data points.

[0118] In actual implementation, the first target data point that is greater than or equal to the second reference value may be retained as the second target data point.

[0119] In this embodiment, after obtaining the second target data point, the resource usage curve may be segmented or smoothed according to the second target data point, and then a polynomial fitting may be performed on the processed resource usage curve to obtain a resource usage prediction function.

[0120] In some embodiments, determining a resource usage prediction function based on the second target data point by polynomial fitting includes:

[0121] Segmenting the resource usage curve based on the second target data point;

[0122] Polynomial fitting is performed on each section of the resource usage curve to determine the resource usage prediction function.

[0123] In this embodiment, the resource usage curve between two adjacent second target data points may be used as a segment of the segmented resource usage curve.

[0124] In this embodiment, after obtaining the segmented resource usage curve, each segment of the resource usage curve is processed separately, and polynomial fitting is performed on each segment of the resource usage curve respectively, and then the results after the polynomial fitting are integrated to obtain the resource usage prediction function.

[0125] The following describes a specific embodiment of determining a resource usage prediction function when the slope change rate is greater than or equal to a change rate threshold.

[0126] like Figure 3 As shown in FIG. 1 , there are many mutations in the resource usage curve. The average value of all sampling points in the sampling data point set is taken as the first reference value. Each sampling point is compared with the first reference value. The sampling points with a value less than the first reference value are removed. The remaining sampling points are taken as the first target data points. The average value of all first target data points is calculated to obtain the second reference value. The reference line is determined parallel to the horizontal axis through the second reference value, as shown in FIG. Figure 3 The line segment corresponding to the AF shown is the reference line.

[0127] All sampling points are grouped by the sampling points higher than the baseline, i.e., the second target data points. That is, the resource usage curve is segmented based on the second target data points, such as Figure 3 As shown, it is divided into AB, BC, CD and other segments.

[0128] The AB segment is fitted by a 4th-order polynomial, and the BC segment is fitted by a 30th-order polynomial. The fitting results are as follows: Figure 4As shown, the curve corresponding to the dotted line is the fitting result, and the corresponding resource usage prediction function can be determined according to the fitting result.

[0129] It should be noted that Figure 3 and Figure 4 The horizontal axis t represents time, and the vertical axis mem represents the usage of computing resources.

[0130] In some embodiments, determining a resource usage prediction function based on a slope change rate of a resource usage curve includes:

[0131] When the slope change rate is less than the change rate threshold, a resource usage prediction function is determined by polynomial fitting.

[0132] In this embodiment, the slope change rate being less than the change rate threshold may indicate that there are fewer sudden changes in the resource usage curve.

[0133] In this embodiment, when the slope change rate is less than the change rate threshold, that is, when there are fewer mutations in the resource usage curve, a polynomial fitting is performed on the entire resource usage curve, and a resource usage prediction function is obtained according to the fitting result.

[0134] For example, Figure 5 As shown in Figure 2, the resource usage curve is Figure 5 The solid line in the middle has fewer mutations and the overall transformation is smooth. The fitting result is obtained through polynomial fitting. The curve corresponding to the dotted line is the fitting result. According to the fitting result, the corresponding resource usage prediction function can be determined.

[0135] It should be noted that Figure 5 The horizontal axis t represents time, and the vertical axis mem represents the usage of computing resources.

[0136] In some embodiments, allocating image processing resources to a computing program based on a target resource usage includes:

[0137] When the target resource usage is greater than a first threshold, controlling the computing program to wait for the image processor to release the resource;

[0138] Alternatively, when the target resource usage is less than or equal to the first threshold, the image processing resource is allocated to the computing program.

[0139] The first threshold is a preset value. When the target resource usage is greater than the first threshold, it indicates that the idle image processing resources of the image processor are insufficient to execute the computing program, and the computing program is controlled to wait for the image processor to release resources before continuing to execute.

[0140] In this embodiment, when the target resource usage is less than or equal to the first threshold, the idle image processing resources of the image processor can meet the needs of executing the computing program, and the image processing resources corresponding to the target resource usage can be allocated to the computing program.

[0141] In some embodiments, calling a driver interface of an image processor to execute a computing program includes:

[0142] When the current resource usage of the computing program is greater than a second threshold, controlling the computing program to wait for the image processor to release resources;

[0143] When the current resource usage of the computing program is less than or equal to the second threshold, the computing program is executed.

[0144] The current resource usage may be the computing resource usage sampled when the driver interface of the image processor is called to execute the computing program.

[0145] In this embodiment, the second threshold is a preset value, and the second threshold and the first threshold may be equal. When the current resource usage is greater than the second threshold, it indicates that the idle image processing resources of the image processor are insufficient to execute the computing program, and the computing program is controlled to wait for the image processor to release resources before continuing to execute.

[0146] In this embodiment, when the current resource usage is less than or equal to the second threshold, it indicates that the idle image processing resources of the image processor are sufficient to execute the computing program, and the computing program can be executed.

[0147] The resource allocation method for an image processor provided in the embodiment of the present application may be executed by a resource allocation device for an image processor. In the embodiment of the present application, the resource allocation device for an image processor executing the resource allocation method for an image processor is taken as an example to illustrate the resource allocation device for an image processor provided in the embodiment of the present application.

[0148] The embodiment of the present application also provides a resource allocation device for an image processor.

[0149] like Figure 6 As shown, the resource allocation device of the image processor includes:

[0150] The first processing module 610 is used to call the driver interface of the image processor to execute the calculation program, obtain the calculation resource usage corresponding to the execution of the calculation program, and obtain a sampling data point set, where each sampling point in the sampling data point set corresponds to a calculation resource usage;

[0151] A second processing module 620, for determining a resource usage prediction function of a computing program based on the set of sampled data points;

[0152] The third processing module 630 is used to determine the target resource usage of the computing program corresponding to the target time based on the resource usage prediction function;

[0153] The fourth processing module 640 is used to allocate image processing resources to the computing program based on the target resource usage.

[0154] According to the resource allocation device of the image processor provided in the embodiment of the present application, the computing resource usage corresponding to the computing program is sampled in advance to obtain a sampling data point set, and the resource usage prediction function of the computing program is obtained based on the sampling data point set. After the resource usage prediction function is obtained, the target resource usage of the computing program at the target time can be determined through the resource usage prediction function, and image processing resources are allocated to the computing program. There is no need to perform periodic sampling of the resource usage of the computing program during the entire execution of the computing program, which can reduce the impact on hardware performance and ensure the resource allocation effect.

[0155] In some embodiments, the second processing module 620 is used to construct a resource usage curve based on the set of sampled data points;

[0156] A resource usage prediction function is determined based on the rate of change of the slope of the resource usage curve.

[0157] In some embodiments, the second processing module 620 is used to calculate a first reference value of a sampling point in the sampling data point set when the slope change rate is greater than or equal to the change rate threshold;

[0158] Compare each sampling point with a first reference value to obtain a first target data point of the sampling data point set;

[0159] calculating a second reference value of the first target data point;

[0160] Compare each first target data point with a second reference value to obtain a second target data point of the sampled data point set;

[0161] Based on the second target data point, a resource usage prediction function is determined by polynomial fitting.

[0162] In some embodiments, the second processing module 620 is configured to segment the resource usage curve based on the second target data point;

[0163] Polynomial fitting is performed on each section of the resource usage curve to determine the resource usage prediction function.

[0164] In some embodiments, the second processing module 620 is configured to determine the resource usage prediction function by polynomial fitting when the slope change rate is less than a change rate threshold.

[0165] In some embodiments, the fourth processing module 640 is used to control the computing program to wait for the image processor to release resources when the target resource usage is greater than the first threshold;

[0166] Alternatively, when the target resource usage is less than or equal to the first threshold, the image processing resource is allocated to the computing program.

[0167] In some embodiments, the first processing module 610 is used to control the computing program to wait for the image processor to release resources when the current resource usage of the computing program is greater than the second threshold;

[0168] When the current resource usage of the computing program is less than or equal to the second threshold, the computing program is executed.

[0169] The resource allocation device of the image processor in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0170] The resource allocation device of the image processor in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0171] The resource allocation device of the image processor provided in the embodiment of the present application can realize Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0172] The embodiment of the present application also provides a resource allocation system 700 for an image processor.

[0173] like Figure 7 As shown, the resource allocation system 700 of the image processor includes a program application module 710 , a resource scheduling module 720 and a resource driver library 730 .

[0174] The program application module 710 is used to receive a program execution request from a user, where the program execution request is a request to execute a computing program on an image processor.

[0175] In this embodiment, the program application module 710 is for users to use, and users can develop, train, and deploy reasoning for algorithms in the program application module 710 .

[0176] In this embodiment, the resource scheduling module 720 is connected to the program application module 710, and the resource scheduling module 720 is used to execute the above-mentioned resource allocation method of the image processor when receiving a program execution request.

[0177] Among them, the resource scheduling module 720 can implement resource and function scheduling based on the container orchestration system (Kubernetes). The virtual computing unified device architecture (Virtual Compute Unified Device Architecture, vCuda) component can be introduced in the resource scheduling module 720. The vCuda component is a GPU resource limitation component that can intercept user requests and provide access to and utilization of the image processor in a virtual machine environment.

[0178] The vCuda component intercepts the user's program execution request and constrains and controls the allocation of image processing resources before forwarding the program execution request to the driver interface.

[0179] In addition, the resource scheduling module 720 may also include a Compute Unified Device Architecture (Cuda) component to implement the scheduling and use of the entire GPU hardware card.

[0180] In this embodiment, the resource driver library 730 stores image processing resources. The resource driver library 730 is connected to the resource scheduling module 720 . The resource scheduling module 720 is used to schedule the image processing resources in the resource driver library 730 .

[0181] In this embodiment, the GPU may be a Nvidia GPU, and the Nvidia GPU container runtime uses the Nvidia container runtime (Nvidia Container Runtime) environment to expose the Nvidia GPU to the application in the container during container creation.

[0182] In this embodiment, the GPU driver and GPU physical resources support graphics cards such as Nvidia A100, V100, T4, and 1080Ti.

[0183] According to the resource allocation system 700 of the image processor provided in the embodiment of the present application, the computing resource usage corresponding to the computing program is sampled in advance to obtain a sampling data point set, and the resource usage prediction function of the computing program is obtained based on the sampling data point set. After obtaining the resource usage prediction function, the target resource usage of the computing program at the target time can be determined through the resource usage prediction function, and image processing resources are allocated to the computing program. There is no need to perform periodic sampling of the resource usage of the computing program during the entire execution of the computing program, which can reduce the impact on hardware performance and ensure the resource allocation effect.

[0184] A specific embodiment of a resource allocation method for an image processor is introduced below.

[0185] like Figure 8 As shown, in step 1, the calculation program corresponding to the AI ​​algorithm executes neural network algorithm modeling and calls the deep learning framework interface during execution.

[0186] Step 2: When the computing program needs to use image processing resources to accelerate the computing process, the deep learning framework calls the vCuda component to apply for the creation of GPU kernel function resources.

[0187] Among them, the vCuda component is a GPU resource restriction component that can intercept user requests and provide access to and utilization of the image processor in a virtual machine environment. The GPU kernel function resources are interfaces, and computing programs can obtain image processing resources on the image processor hardware through the GPU kernel function resources.

[0188] Step 3: Within the initial T time period after vCuda receives the image processing resource call request corresponding to the computing program, vCuda performs two operations when forwarding the call request to the Nvidia driver library:

[0189] Operation 1: At intervals of time t, vCuda calls the GPU driver interface to sample the thread usage on the GPU card, that is, the computing resource usage, and records the sampled data point set X. The sampled data point set X is the predicted action of the GPU usage rate after T time, that is, the target resource usage provides sample data, where T>100t.

[0190] Operation 2: When the current GPU Thread usage, i.e., the current resource usage, is less than the set second threshold, the calculation program continues to execute;

[0191] When the current GPU Thread usage is greater than a set second threshold, the computing program waits for GPU resources to be released.

[0192] Step 4: Nvidia GPU driver libcuda.so.xxx.xxx.xx receives the image processing resource call request, completes resource application and creation, and feeds back the image processing resources corresponding to the GPU kernel function resources to the upper-level algorithm through vCuda.

[0193] Among them, Nvidia GPU is a driver interface of an image processor, and libcuda.so.xxx.xxx.xx can store functions and resources that can be called when a computing program is executed on the image processor.

[0194] Step 5: After vCuda receives the image processing resource call request for a period of T, sufficient sample point data has been collected, and the resource usage prediction function f(X) is constructed with the sampled data point set X.

[0195] Step 6: After receiving the image processing resource call request, vCuda uses the resource usage prediction function f(X) to calculate the GPU Thread usage, that is, the target resource usage.

[0196] When the GPU Thread usage is less than the set first threshold, the calculation program continues to execute; when the GPU Thread usage is greater than the set first threshold, the calculation program waits for the image processor to release resources.

[0197] It should be noted that the hardware card corresponding to the image processor is divided into multiple computing instances, the first threshold and the second threshold can be determined according to the number of image processor divisions, and the first threshold and the second threshold can be inversely proportional to the number of image processor divisions.

[0198] The hardware card corresponding to the image processor is divided into multiple computing instances. At the same time, multiple computing programs can be executed simultaneously on the above computing instances.

[0199] In this embodiment, the image processor is divided into multiple computing instances and allocated to multiple virtual machines or containers. The resource consumption of the multiple computing instances is not affected by each other. Different divisions of the image processor correspond to different first thresholds and second thresholds. The first threshold and the second threshold are adjusted to adjust the resource allocation of the computing program. It is estimated that the hardware cost can be saved by 20%-30%.

[0200] In this embodiment, fine-grained division and reuse of GPU resources are achieved through vCuda, Cuda API is intercepted and redirected at the user level, and GPU Thread is used as the GPU resource indicator. When the Thread usage of the computing program is less than or equal to the set threshold, the computing program continues to execute. When the Thread usage of the computing program is greater than the set threshold, the computing program enters a waiting state and waits for the GPU resources to be released before running again. During the execution of this computing program, other computing programs can be executed normally according to logic, thereby achieving effective utilization and scheduling of GPU resources.

[0201] like Fig. 9 As shown in the figure, the straight line parallel to the horizontal axis is the set GPU core utilization threshold, and the fluctuating curve is the actual GPU utilization. Fig. 9 It can be seen that the actual GPU utilization fluctuates around the set GPU core utilization threshold, achieving effective utilization of GPU resources.

[0202] In some embodiments, Fig.10 As shown, an embodiment of the present application further provides an electronic device 1000, including a processor 1001, a memory 1002, and a computer program stored in the memory 1002 and executable on the processor 1001. When the program is executed by the processor 1001, each process of the resource allocation method embodiment of the above-mentioned image processor is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0203] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0204] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the resource allocation method embodiment of the above-mentioned image processor are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0205] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0206] An embodiment of the present application further provides a computer program product, including a computer program, which implements the resource allocation method of the above-mentioned image processor when executed by a processor.

[0207] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0208] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the resource allocation method embodiment of the above-mentioned image processor, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0209] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0210] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0211] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0212] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0213] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0214] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A resource allocation method for an image processor, characterized in that: include: Calling a driver interface of an image processor to execute a computing program, obtaining a computing resource usage corresponding to executing the computing program, and obtaining a sampling data point set, wherein each sampling point in the sampling data point set corresponds to one computing resource usage; determining a resource usage prediction function for the computing program based on the set of sampled data points; Determining a target resource usage of the computing program corresponding to a target time based on the resource usage prediction function; Image processing resources are allocated to the computing program based on the target resource usage.

2. The resource allocation method of the image processor according to claim 1, characterized in that: The step of determining a resource usage prediction function of the computing program based on the set of sampled data points comprises: constructing a resource usage curve based on the set of sampled data points; The resource usage prediction function is determined based on the slope change rate of the resource usage curve.

3. The resource allocation method of the image processor according to claim 2, characterized in that: The determining the resource usage prediction function based on the slope change rate of the resource usage curve includes: When the slope change rate is greater than or equal to a change rate threshold, calculating a first reference value of the sampling point in the sampling data point set; Compare each of the sampling points with the first reference value to obtain a first target data point of the sampling data point set; Calculating a second reference value of the first target data point; Compare each of the first target data points with the second reference value to obtain a second target data point of the sampled data point set; Based on the second target data point, the resource usage prediction function is determined by polynomial fitting.

4. The resource allocation method of the image processor according to claim 3, characterized in that: The determining the resource usage prediction function based on the second target data point by polynomial fitting includes: segmenting the resource usage curve based on the second target data point; Polynomial fitting is performed on each section of the resource usage curve to determine the resource usage prediction function.

5. The resource allocation method of the image processor according to claim 2, characterized in that: The determining the resource usage prediction function based on the slope change rate of the resource usage curve includes: When the slope change rate is less than the change rate threshold, the resource usage prediction function is determined by polynomial fitting.

6. The resource allocation method of an image processor according to any one of claims 1 to 5, characterized in that: The allocating image processing resources to the computing program based on the target resource usage includes: When the target resource usage is greater than a first threshold, controlling the computing program to wait for the image processor to release resources; Alternatively, when the target resource usage is less than or equal to the first threshold, the image processing resource is allocated to the computing program.

7. The resource allocation method of an image processor according to any one of claims 1 to 5, characterized in that: The calling of the driving interface of the image processor to execute the calculation program includes: When the current resource usage of the computing program is greater than a second threshold, controlling the computing program to wait for the image processor to release resources; In a case where the current resource usage of the computing program is less than or equal to a second threshold, the computing program is executed.

8. A resource allocation device for an image processor, characterized in that: include: A first processing module is used to call a driving interface of an image processor to execute a computing program, obtain a computing resource usage corresponding to the execution of the computing program, and obtain a sampling data point set, wherein each sampling point in the sampling data point set corresponds to one computing resource usage; A second processing module, configured to determine a resource usage prediction function of the computing program based on the set of sampled data points; A third processing module, configured to determine a target resource usage of the computing program corresponding to a target time based on the resource usage prediction function; A fourth processing module is used to allocate image processing resources to the computing program based on the target resource usage.

9. A resource allocation system for an image processor, characterized in that: include: A program application module, the program application module is used to receive a user's program execution request; a resource scheduling module, the resource scheduling module being connected to the program application module, and the resource scheduling module being configured to execute the resource allocation method for the image processor according to any one of claims 1 to 7 upon receiving the program execution request; A resource driver library stores image processing resources. The resource driver library is connected to the resource scheduling module. The resource scheduling module is used to schedule the image processing resources in the resource driver library.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the resource allocation method of the image processor according to any one of claims 1 to 7 is implemented.

Images