User side force calculation method based on coprocessor platform and program product

Abstract

The invention relates to the technical field of computers, in particular to a user side calculation force measurement method based on a coprocessor platform and a program product. The method comprises the following steps that arithmetic operation calling sent by a user to a coprocessor is obtained in a code interception mode; the obtained specific content called by the arithmetic operation is sent to a profiler with an application program interface, so that corresponding basic operation is analyzed and obtained; wherein the profilter refers to tool software capable of being used for monitoring the performance of the user program; indexes based on basic operations are used to characterize the computing power of co-processing consumed by the user. According to the method, all computing power values of consumed co-processing in the running life cycle of all the processes started later can be monitored, measurement of each process is automatically completed, a user does not need to operate again, and application program codes do not need to be modified.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a user-side computing power measurement method and program product based on a coprocessor platform. Background Art

[0002] With the development of cloud computing, many computing tasks are performed on the cloud, and "computing power" is used to evaluate the computing capabilities of various cloud computing centers. The computer academic community has a general consensus on the concept of "computing power", which refers to the computing performance of a device or system, or "computing power" is a colloquial expression of computing performance, similar to "computer" being a colloquial expression of computer. People generally use computing speed as an indicator to characterize computing performance, including millions of instructions per second (MIPS), floating-point operations per second (FLOPS), fixed-point operations per second (OPS), etc., to indicate the performance of fixed-point, half-precision / single-precision / double-precision floating-point, and 8-bit integer operations commonly used in artificial intelligence.

[0003] For computing power users, if they want to use computing power, the current method is that users need to rent cloud servers on the public cloud and charge according to the length of use. The coprocessor mentioned in the present invention refers to a computing device that uses dedicated driver software. Typical coprocessors include GPU (graphics processing unit), TPU (tensor processing unit), etc., specifically DCU produced by Haiguang Company, MI300 produced by AMD Semiconductor Company and similar computing devices. Taking GPU as an example, for GPU users, the current use methods are mainly to buy GPU cards or rent GPUs on public cloud platforms. Whether buying or renting a GPU, there are certain requirements for the user's professional background knowledge, otherwise there will be a risk of buying or renting an unmatched GPU. There is a relatively large cost overhead for purchasing a GPU, and there will be a situation where it is no longer used after it is used up. On the one hand, the cost of renting a GPU is high, and on the other hand, there will be a problem of mismatch between actual demand and the rented configuration, and there is a problem of waste of resources.

[0004] The future development trend is to make computing power infrastructure from the user side, and characterizing and counting the computing power usage of users is the key element of computing power infrastructure. Characterizing the computing power usage of users requires a unified measurement indicator. At present, the measurement indicators of various computing powers are different. For example, the XPU indicator is OPS (8 / 16-bit integer operations) and the GPU indicator is FLOPS (16 / 32-bit floating-point operations). How to more conveniently measure the computing power of coprocessors of different architectures in a unified manner is an urgent problem to be solved. Summary of the invention

[0005] In view of the above problems, the present invention uses indicators based on basic operations (BOPs, Basic Operations) to uniformly measure the computing power of heterogeneous coprocessors. Specifically, the acquisition of basic operations is achieved on the coprocessor platform by combining code interception technology and a profiler with an application program interface. When using a coprocessor, the application will eventually directly or indirectly link with the exported function symbols in the driver file library (such as the libcuda.so file provided by NVIDIA for its graphics card). In this method, a pseudo driver file for measuring the computing power of the program will be generated, which has the same exported function symbol name as the existing driver file. When the application calls the driver file, it will first link to the corresponding function symbol in the pseudo driver file. The pseudo driver file will call the application program interface of the performance test tool provided by the graphics card to obtain the data required for the above measurement, and at the same time call the function symbol in the original dynamic link library as is to complete the real function of the function call required by the user. The interface interception technology allows the upper-level application to trigger the measurement-related code and perform the coprocessor platform computing power measurement directly on the computer where the pseudo driver file is installed without modifying the code.

[0006] Profiler refers to a type of tool software that can be used to monitor the performance of user programs. The profile used in the present invention must provide the function of classifying and counting the machine instructions generated by the specified code segment on the GPU in the form of an application program interface (the categories must include but are not limited to single-precision, double-precision, half-precision and integer arithmetic instructions). For example, for NVIDIA GPUs, "profiler with an application program interface" generally refers to CUPTI, namely CUDA Performance Tool Interface, CUDA (Compute Unified Device Architecture) performance tool interface.

[0007] The present invention provides the following technical solutions:

[0008] A user-side computing power measurement method based on a coprocessor platform, wherein the coprocessor refers to a computing device using dedicated driver software, comprising the following steps:

[0009] S1: using code interception to obtain the arithmetic operation call issued by the user to the coprocessor; and sending the specific content of the obtained arithmetic operation call to a profiler with an application program interface to analyze and obtain the corresponding basic operation; wherein the profiler refers to a tool software that can be used to monitor the performance of the user program;

[0010] S2: Use metrics based on basic operations to characterize the co-processing power consumed by users.

[0011] Preferably, the index based on basic operations is specifically a basic operand of calculation operations, comparison operations and address operations corresponding to integer and floating-point calculations normalized to a preset number of bits.

[0012] Preferably, the preset number of bits is 64 bits.

[0013] Preferably, the process of obtaining the arithmetic operation call issued by the user to the coprocessor by means of code interception and sending the specific content of the obtained arithmetic operation call to the profiler with the application program interface is as follows:

[0014] Create a pseudo dynamic link library and replace the original dynamic link library; wherein the pseudo dynamic link library satisfies: each function exposed to the user in the original dynamic link library has a function with the same function signature in the pseudo dynamic link library, wherein the function signature includes three parts: function name, parameter list and return value type; wherein the original dynamic link library refers to the dynamic link library corresponding to the dedicated driver software of the coprocessor;

[0015] The arithmetic operation call issued by the user to the coprocessor is obtained through the pseudo dynamic link library, and the specific content of the arithmetic operation call obtained is sent to the profiler with an application program interface; then the function of the user calling the coprocessor is realized by calling the original dynamic link library through the pseudo dynamic link library.

[0016] Preferably, the behavior of the function in the pseudo dynamic link library is divided into the following three cases according to the type of the function:

[0017] Case 1: The function is an arithmetic operation type, and the function behavior is to send the specific content of calling this function to the profiler with an application program interface;

[0018] Then the user calls the co-processing function by calling the original dynamic link library;

[0019] Case 2: The function is a type that obtains the entry address of other functions required for calling. The function behavior is to determine whether the function queried by the user exists by calling the original dynamic link library; if it exists, the returned address is replaced with the address of the function in the pseudo dynamic link library. If it does not exist, the user is prompted that the function does not exist;

[0020] Case 3: The function is of other types. The function's behavior is to call the corresponding function in the original dynamic link library and return the result to the user as is.

[0021] A computer program product comprises a computer program, and the computer program implements the above method when executed by a processor.

[0022] Beneficial effects:

[0023] Compared with the existing technology, the present invention has the following advantages: The present invention provides a monitoring and statistical method for heterogeneous coprocessor computing power values. After starting the tool once, all the computing power values ​​of the coprocessor consumed during the life cycle of all processes started thereafter can be monitored. The measurement of each process is completed automatically without the need for the user to operate again, and there is no need to modify the application code. Starting from the user side, the computing power value of the coprocessor actually used by the user can be measured to achieve on-demand use of the coprocessor computing power. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 In the embodiment, the operation process of the pseudo dynamic link library;

[0025] Figure 2 This is a C language implementation code example of the "start counting" function in the pseudo code of the embodiment;

[0026] Figure 3 This is a C language implementation code example of the "get counting result" function in the pseudo code of the embodiment; DETAILED DESCRIPTION

[0027] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] A user-side computing power measurement method based on a coprocessor platform, wherein the coprocessor refers to a computing device using dedicated driver software. Common coprocessors include GPU, TPU, DCU, etc., and include the following steps:

[0029] S1: Obtain arithmetic operation calls issued by the user to the coprocessor by code interception; and send the specific content of the obtained arithmetic operation calls to a profiler with an application program interface to analyze and obtain corresponding basic operations (BOPs); wherein the profiler refers to a tool software that can be used to monitor the performance of the user program;

[0030] Specifically, the process of using code interception to obtain the arithmetic operation call issued by the user to the coprocessor and sending the specific content of the obtained arithmetic operation call to the profiler with an application program interface is as follows:

[0031] Create a pseudo dynamic link library and replace the original dynamic link library; wherein the pseudo dynamic link library satisfies: each function exposed to the user in the original dynamic link library has a function with the same function signature in the pseudo dynamic link library, wherein the function signature includes three parts: function name, parameter list and return value type; wherein the original dynamic link library refers to the dynamic link library corresponding to the dedicated driver software of the coprocessor;

[0032] The arithmetic operation call issued by the user to the coprocessor is obtained through the pseudo dynamic link library, and the specific content of the arithmetic operation call obtained is sent to the profiler with an application program interface; and the function of the user calling the coprocessor is realized by calling the original dynamic link library through the pseudo dynamic link library;

[0033] Specifically, the behavior of functions in pseudo dynamic link libraries can be divided into the following three cases according to the type of function:

[0034] Case 1: The function is an arithmetic operation type, and the function behavior is to send the specific content of calling this function to the profiler with an application program interface;

[0035] Then the user calls the co-processing function by calling the original dynamic link library;

[0036] Case 2: The function is a type that obtains the entry address of other functions required for calling. The function behavior is to determine whether the function queried by the user exists by calling the original dynamic link library; if it exists, the returned address is replaced with the address of the function in the pseudo dynamic link library. If it does not exist, the user is prompted that the function does not exist;

[0037] Case 3: The function is of other types. The function behavior is to call the corresponding function in the original dynamic link library and return the result to the user as it is.

[0038] S2: Use an indicator based on basic operations to characterize the computing power of the co-processing consumed by the user; wherein the indicator based on basic operations is specifically the basic operands of calculation operations, comparison operations and address operations corresponding to integer and floating-point calculations normalized to a preset number of bits.

[0039] In this embodiment, the preset bit number is 64 bits, so basic operands of different bit numbers are normalized to 64 bits. For example, the basic operations of 32-bit single-precision arithmetic instructions are counted as 32 / 64=0.5 each, and the basic operations of 16-bit half-precision arithmetic instructions are counted as 16 / 64=0.25 each.

[0040] The following takes the co-processor as GPU as an example to provide a more detailed implementation method:

[0041] Step 1: Generate a pseudo dynamic link library.

[0042] Use nm -D or similar system tool commands to obtain the function symbol list exposed to the user by the original dynamic link library. After obtaining the function symbol list, define the functions one by one in the pseudo dynamic link library in accordance with the function parameters and return value types provided by the manufacturer's documents (such as cuda.h), so that for each function exposed to the user in the original dynamic link library, there is a function with the same function signature (including function name, parameter list and return value type) in the pseudo dynamic link library. For the specific content of the defined function, please refer to the description of the tool running process in step 3.

[0043] Step 2: Install the pseudo dynamic link library.

[0044] After generating the appropriate dynamic link library file, back up the original dynamic link library to a specified location and record the backup location, and copy the pseudo dynamic link library to the file path where the original dynamic link library is located, and rename the copy of the pseudo dynamic link library to the same name as the original dynamic link library.

[0045] Step 3: Run the tool.

[0046] After the installation is correct, when the client program tries to call the function in the original dynamic link library according to the original path, it will actually call the function with the same signature in the pseudo dynamic link library. Different functions have different functions, and the functions implemented by the defined functions are different, such as Figure 1 As shown, the specific behavior of the function can be divided into the following three cases:

[0047] 1) Case 1: For calls that use the GPU to perform actual arithmetic operations (for GPUs produced by NVIDIA, including but not limited to cuLaunchKernel functions), first use the profiler's application program interface to enable the GPU to start counting the number of instructions on the GPU generated by subsequent computing tasks, and then make the same call to the original dynamic link library in the backup location with the parameters sent by the user application to complete the actual computing task. Finally, the data required to calculate the number of BOPs and BOPS during the actual execution of the call are recovered from the GPU through the profiler's application program interface, and the GPU is configured to terminate the statistical state;

[0048] 2) Case 2: For calls whose function is to obtain the function entry address required for other calls (for GPUs produced by NVIDIA, including but not limited to cuGetProcAddress and cuGetProcAddress_v2 functions), first make the same call to the original dynamic link library in the backup location with the parameters sent by the user application, but replace the function entry address returned by the original dynamic link library with the function entry address of this dynamic link library, and then return it to the user;

[0049] 3) Case 3: For other types of calls, no changes are made, and the original dynamic link library in the backup location is directly called with the parameters sent by the user application, and the results generated are returned to the user as is.

[0050] The pseudo code corresponding to the tool running process is as follows, where the "function names" need to be replaced one by one with the functions in the function symbol list exposed to the user by the original dynamic link library:

[0051] function proxy function (parameter list):

[0052] Real function = Find the function by name in the dynamic link library (original dynamic link library, proxy function.name)

[0053] If case 1:

[0054] start_count(command_type list);

[0055] return value = true function (parameter list);

[0056] stopCount();

[0057] Count result = get count result();

[0058] BOPs value = calculateBOPs(count result);

[0059] Send (BOPs value) to the database;

[0060] return return value;

[0061] else if case 2:

[0062] The function name that needs to query the address for the user = excerpt (parameter list);

[0063] if function name is valid (need to query the function name for the user address) is true:

[0064] Return address = find the function by name in the dynamic link library (pseudo dynamic link library, the function name of the address needs to be queried for the user);

[0065] else:

[0066] return address = invalid address;

[0067] return return address;

[0068] else if case three:

[0069] return value = true function (parameter list);

[0070] return return value;

[0071] In the above pseudo code, the actual implementation of "start counting", "stop counting" and "get counting results" needs to be determined according to the form of the application program interface provided by the profiler. The implementation method varies according to the manufacturer, the version of the basic software such as the driver, the device model, etc. "Start counting" is implemented in C language using the Event interface of CUPTI as follows Figure 2 As shown; "Getting the counting result" is implemented in C language using the Event interface of CUPTI as follows Figure 3 shown.

[0072] In this embodiment, the preset number of bits is 64 bits, and the "basic operation-based index" is specifically the basic operands of calculation operations, comparison operations, and address operations corresponding to integer and floating-point calculations normalized to 64 bits. Depending on the design of the instruction counting application program interface corresponding to the GPU, the function for calculating the "basic operation-based index" may need to have different mathematical forms. For example, if it is assumed that the collected data content is the number of integer arithmetic operation instructions (assuming that all integers are 32-bit integers, i.e., INT32, and need to be multiplied by 32 / 64, i.e., 0.5, when normalized), the number of double-precision (FP64, need to be multiplied by 64 / 64, i.e., 1, when normalized) arithmetic operation instructions, the number of single-precision (FP32, need to be multiplied by 32 / 64, i.e., 0.5, when normalized) arithmetic operation instructions, and the number of half-precision (FP16, need to be multiplied by 16 / 64, i.e., 0.25, when normalized) arithmetic operation instructions, the implementation of the function for calculating the "basic operation-based index" should be similar to the following pseudo code:

[0073] Function calculates BOPs (counting results):

[0074] BOPs value = 0;

[0075] BOPs value += count result. Number of integer arithmetic operation instructions * 0.5;

[0076] BOPs value += count result. Number of half-precision floating point arithmetic operation instructions * 0.25;

[0077] BOPs value += count result. Number of full-precision floating-point arithmetic operation instructions * 0.5;

[0078] BOPs value += count result. Number of double-precision floating-point arithmetic operation instructions * 1;

[0079] return BOPs value;

[0080] Among them, the "BOPs value" in the pseudocode refers to the "basic operation-based index" in this embodiment, that is, the basic operands of calculation operations, comparison operations and address operations corresponding to integer and floating-point calculations normalized to 64 bits.

[0081] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A user-side computing power measurement method based on a coprocessor platform, wherein the coprocessor refers to a computing device using dedicated driver software, characterized in that: The following steps are involved: S1: using code interception to obtain the arithmetic operation call issued by the user to the coprocessor; and sending the specific content of the obtained arithmetic operation call to a profiler with an application program interface to analyze and obtain the corresponding basic operation; wherein the profiler refers to a tool software that can be used to monitor the performance of the user program; S2: Use metrics based on basic operations to characterize the co-processing power consumed by users.

2. According to a method for measuring computing power on the user side based on a coprocessor platform according to claim 1, it is characterized in that: The index based on basic operations is specifically a basic operand of calculation operations, comparison operations and address operations corresponding to integer and floating point calculations normalized to a preset number of bits.

3. A user-side computing power measurement method based on a coprocessor platform according to claim 2, characterized in that: The preset number of bits is 64 bits.

4. The method for measuring computing power on the user side based on a coprocessor platform according to claim 1, characterized in that: The process of using code interception to obtain the arithmetic operation calls issued by the user to the coprocessor and sending the specific content of the obtained arithmetic operation calls to the profiler with an application program interface is as follows: Create a pseudo dynamic link library and replace the original dynamic link library; wherein the pseudo dynamic link library satisfies: each function exposed to the user in the original dynamic link library has a function with the same function signature in the pseudo dynamic link library, wherein the function signature includes three parts: function name, parameter list and return value type; wherein the original dynamic link library refers to the dynamic link library corresponding to the dedicated driver software of the coprocessor; The arithmetic operation call issued by the user to the coprocessor is obtained through the pseudo dynamic link library, and the specific content of the arithmetic operation call obtained is sent to the profiler with an application program interface; then the function of the user calling the coprocessor is realized by calling the original dynamic link library through the pseudo dynamic link library.

5. A user-side computing power measurement method based on a coprocessor platform according to claim 4, characterized in that: The behavior of functions in pseudo dynamic link libraries can be divided into the following three cases according to the type of function: Case 1: The function is an arithmetic operation type, and the function behavior is to send the specific content of calling this function to the profiler with an application program interface; Then the user calls the co-processing function by calling the original dynamic link library; Case 2: The function is a type that obtains the entry address of other functions required for calling. The function behavior is to determine whether the function queried by the user exists by calling the original dynamic link library; if it exists, the returned address is replaced with the address of the function in the pseudo dynamic link library. If it does not exist, the user is prompted that the function does not exist; Case 3: The function is of other types. The function's behavior is to call the corresponding function in the original dynamic link library and return the result to the user as is.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images