Method and system for optimizing GPU remote call performance based on pre-analysis and calculation business

Abstract

The present application relates to the field of computer technology, and in particular to a method and system for optimizing GPU remote call performance based on pre-analysis of computing services, wherein the method includes extracting device code blocks from computing services, and the device code blocks are used to represent code snippets running on the GPU. The extracted device code blocks are pre-processed, the computing services are analyzed, and the optimal remote call strategy is determined. According to the determined remote call strategy, a GPU remote call operation is executed, the computing task is sent to a remote GPU for execution, and the execution result is obtained. Performance optimization is performed based on the execution result of the remote call. The present application can optimize the performance of GPU remote calls, reduce client burden and transmission delay, and improve computing efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and system for optimizing GPU remote call performance based on pre-analysis of computing services. Background Art

[0002] With the rapid development of big data and artificial intelligence technologies, the powerful computing power of graphics processing units (GPUs) has been widely used in various computing-intensive tasks, such as deep learning training, scientific computing, and image processing. Compared with traditional central processing units (CPUs), GPUs can process large amounts of data in parallel, giving them significant performance advantages when handling complex computing tasks. However, the high cost and limited popularity of GPU resources make remote calls to GPU computing resources an important solution. In this way, users can use the GPU resources of remote servers to perform computing tasks that cannot be efficiently processed locally, thereby improving overall computing efficiency.

[0003] Existing GPU remote call solutions usually rely on real-time analysis of computing services on the client side and transmission of analysis results and code snippets to the server side for processing. These solutions include, but are not limited to, real-time parsing of the code to be processed by the client, identifying the GPU device code therein, and sending it to the remote server for execution. In this process, the client needs to undertake a lot of computing and data transmission work, including code parsing, data packaging and transmission, etc., to ensure that the remote server can accurately receive and execute these tasks. Although this method can realize remote calls to GPUs, it has obvious shortcomings in efficiency and resource utilization.

[0004] The existing technology mainly faces the following problems when implementing GPU remote calls: First, the real-time analysis and computing services will consume a large amount of client resources, resulting in excessive client load and affecting its normal operation; second, the real-time analysis and transmission process takes a long time, which increases the delay of business processing; third, the large amount of data transmission easily occupies too high network bandwidth and affects network performance. The above problems cause the remote GPU computing service to start slowly and take a long time to calculate, which seriously affects the overall performance of the system and user experience. In summary, how to optimize the performance of GPU remote calls, reduce client burden and transmission delay, and improve computing efficiency has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] This application provides a method and system for optimizing GPU remote call performance based on pre-analysis of computing services, which can optimize the performance of GPU remote calls, reduce client burden and transmission delay, and improve computing efficiency. This application provides the following technical solutions:

[0006] In a first aspect, the present application provides a method for optimizing GPU remote call performance based on pre-analysis of computing services, the method comprising:

[0007] Extracting a device code block from a computing business, where the device code block is used to represent a code fragment running on a GPU;

[0008] Preprocessing the extracted device code block, analyzing the computing business and determining an optimal remote call strategy;

[0009] According to the determined remote call strategy, a GPU remote call operation is executed, the computing task is sent to the remote GPU for execution, and the execution result is obtained;

[0010] Performance optimization is performed according to the execution result of the remote call.

[0011] In a specific implementation scheme, extracting the device code block from the computing service includes:

[0012] Specify the code storage path for the computing business, and traverse and search all files in the directory;

[0013] Identify the file containing the GPU device code using the command line tool and the "cuobjdump filename" command;

[0014] Store the file in the specified directory.

[0015] In a specific possible implementation scheme, extracting the device code block from the computing service further includes:

[0016] Prioritize all files and extract device code blocks in sequence according to the priority ranking results; use the dynamic content similarity and path adaptation formula to calculate the priority score of each file, and sort the files in descending order according to the priority score. The calculation formula for the priority score is as follows:

[0017] in, For the The content vector of each file and the vector of the entire content library The cosine similarity between For the The normalized path length of each file is For the The normalized size of the file.

[0018] In a specific implementation scheme, the identifying of the file containing the GPU device code by using a command line tool and a command named cuobjdump file name includes:

[0019] Extract the cubin file using the command cuobjdump -lelf filename and cuobjdump -xelf .cubin filename;

[0020] Extract the ptx file using the command cuobjdump -ptx filename and cuobjdump -xptx filename;

[0021] Extract the arrangement order of ptx files and cubin files in the binary through the command named cuobjdump -lptx file name;

[0022] The cubin files and ptx files are organized into fatbin files in order through the command named nvcc –create.

[0023] In a specific implementation scheme, the identifying of the file containing the GPU device code by using a command line tool and a command named cuobjdump file name also includes:

[0024] Extract the function definition file through the command named cuobjdump -elf .cubin file name and parse the length of the parameters in the function;

[0025] The collected fatbin files and function analysis results are compressed and stored in the database.

[0026] In a specific possible implementation scheme, executing the GPU remote call operation according to the determined remote call strategy, sending the computing task to the remote GPU for execution, and obtaining the execution result includes:

[0027] Start a service process on a server with a GPU to process computing requests transmitted from the client;

[0028] Connect the server to the database that stores the fatbin file and function definition file, deploy the client on another server, start the computing service, and the client is responsible for intercepting the API of the computing service and forwarding the request to the server.

[0029] The client calculates the MD5 value of the code block to be executed and sends the calculated MD5 value to the server. The server queries the corresponding fatbin file and function definition file from the database based on the received MD5 value.

[0030] After the server obtains the code block to be run, it loads it into memory and calls the local class library to transfer the code to the GPU to perform function operations. After the calculation is completed, the server returns the result to the client, and the client receives the result and processes subsequent business logic.

[0031] In a specific implementation scheme, preprocessing the extracted device code block, analyzing the computing service and determining the best remote call strategy includes:

[0032] Perform format conversion on the device code block to convert it into a format suitable for transmission and storage, and perform data compression operations to reduce the size of data transmission;

[0033] Analyze the computing business to be processed, including the data packet size and dependent CUDA library, and determine the best remote call strategy;

[0034] Analyze the extracted device code blocks, determine the CUDA libraries they depend on, evaluate the packet size and code block complexity, and select appropriate compression and transmission strategies;

[0035] Determine the best remote call strategy based on the results of data preprocessing and computing business analysis.

[0036] In the second aspect, the present application provides a system for optimizing GPU remote call performance based on pre-analysis of computing services, which adopts the following technical solutions:

[0037] A system for optimizing GPU remote call performance based on pre-analysis of computing services, comprising:

[0038] A code extraction module, used to extract a device code block from a computing service, wherein the device code block is used to represent a code fragment running on a GPU;

[0039] A business analysis module, used to pre-process the extracted device code block, analyze the computing business and determine the best remote call strategy;

[0040] A remote call module is used to execute a GPU remote call operation according to the determined remote call strategy, send the computing task to the remote GPU for execution, and obtain the execution result;

[0041] The performance optimization module is used to perform performance optimization according to the execution result of the remote call.

[0042] In a third aspect, the present application provides an electronic device, comprising a processor and a memory; the memory stores a program, and the program is loaded and executed by the processor to implement a method for optimizing GPU remote call performance based on pre-analysis of computing services as described in the first aspect.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, in which a program is stored. When the program is executed by a processor, it is used to implement a method for optimizing GPU remote call performance based on pre-analysis of computing services as described in the first aspect.

[0044] In summary, the beneficial effects of this application include at least:

[0045] (1) By transferring the extraction, preprocessing and analysis of device code blocks to the server, this application significantly reduces the client's resource consumption in real-time analysis and data transmission. This reduces the burden on the client and improves the efficiency of the overall system.

[0046] (2) The device code blocks are compressed and format converted in advance, which optimizes the data transmission process and reduces the size and time of data transmission. This effectively reduces the demand for network bandwidth, reduces transmission delays, and improves the response speed and efficiency of remote calls.

[0047] (3) By preprocessing and optimizing device code blocks on the server side and formulating the best remote call strategy based on the analysis results, this application improves the performance and resource utilization of GPU remote computing. This not only shortens the service startup time, but also speeds up the execution of computing tasks, achieving more efficient remote GPU computing.

[0048] The efficiency of GPU remote computing is improved by extracting, preprocessing, analyzing and optimizing device code blocks. The specific steps include: extracting device code blocks in computing services, performing data format conversion and compression to reduce data transmission, analyzing computing services to determine the best remote call strategy, and then executing remote calls based on the strategy, and optimizing performance during the execution process. This solution solves the problems of startup delays and long computing time caused by real-time analysis of business programs and high network bandwidth requirements in the existing technology. Through preprocessing and optimization, it significantly reduces client resource consumption and network transmission burden, thereby improving the performance and computing efficiency of GPU remote calls.

[0049] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present application in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a flowchart of a method for optimizing GPU remote call performance based on pre-analysis of computing services in an embodiment of the present application.

[0051] Figure 2 It is a flowchart of extracting device code blocks from computing services in an embodiment of the present application.

[0052] Figure 3 It is a flowchart of using the extracted device code block in GPU remote computing in an embodiment of the present application.

[0053] Figure 4 It is a structural block diagram of a system for optimizing GPU remote call performance based on pre-analysis of computing services in an embodiment of the present application.

[0054] Figure 5 It is a block diagram of an electronic device for optimizing GPU remote call performance based on pre-analysis of computing services in an embodiment of the present application. DETAILED DESCRIPTION

[0055] The specific implementation methods of the present application are further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present application but are not intended to limit the scope of the present application.

[0056] The following are the special terms in the art that appear in the examples of this application:

[0057] A GPU (Graphics Processing Unit) is a processor designed specifically for processing graphics and image computing tasks. Originally designed to accelerate graphics rendering, modern GPUs have evolved into a powerful computing device that is widely used for a variety of computationally intensive tasks.

[0058] CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model developed by NVIDIA. It allows developers to use the powerful computing power of NVIDIA GPUs to accelerate computationally intensive tasks.

[0059] CUBIN (CUDA Binary) is a binary file format generated after compilation in CUDA programming, which contains low-level instructions that can be executed directly on NVIDIA GPU.

[0060] PTX (Parallel Thread Execution) is an intermediate representation language designed by NVIDIA for its CUDA programming model. PTX code is a virtual instruction set between high-level CUDA C / C++ code and underlying machine code (SASS). PTX code is generated at compile time and further optimized and converted into actual machine instructions by the GPU driver at runtime.

[0061] FATBIN (Fat Binary) is a file format used in the CUDA programming model to package and manage multiple binary code versions. FATBIN files can contain PTX code and SASS code for different GPU architectures (such as Kepler, Maxwell, Pascal, Volta, etc.), allowing one file to run efficiently on different NVIDIA GPUs.

[0062] NVCC (NVIDIA CUDA Compiler) is a compiler driver provided by NVIDIA for the CUDA programming model. NVCC can compile CUDA C / C++ source code into multiple forms of target code, including PTX intermediate representation, GPU device code, and host CPU code.

[0063] cuobjdump is a tool in the NVIDIA CUDA toolchain that is used to analyze and extract information from the binary files of CUDA programs. cuobjdump is mainly used to view the kernel code, PTX (Parallel Thread Execution) code, SASS (Shader Assembly) code and other related information in the binary files of CUDA programs. It is one of the important tools for CUDA developers to debug and optimize CUDA applications.

[0064] Client: A GPU remote computing client is a user or program that connects to a remote GPU resource. Through the client, remote GPU resources can be accessed, managed, and utilized remotely.

[0065] Server: GPU remote computing server refers to the server or computing node that provides remote GPU resources. Users can connect to these servers through remote computing clients and use the GPU resources they provide to perform computing tasks.

[0066] DB: A database is an organized system that can store and manage data. It can be thought of as an electronic filing cabinet that is used to collect, store, organize, and manage large collections of related data. A database system usually consists of data, a database management system (DBMS), and application programs to efficiently manage and access the data.

[0067] Optionally, the present application takes the method provided in each embodiment for optimizing GPU remote call performance based on pre-analysis of computing services as an example for use in an electronic device, where the electronic device is a terminal or a server, and the terminal may be a computer, a tablet computer, etc. This embodiment does not limit the type of electronic device.

[0068] Reference Figure 1 , is a flow chart of a method for optimizing GPU remote call performance based on pre-analysis of computing services provided by an embodiment of the present application, the method comprising at least the following steps:

[0069] Step S101: extract a device code block from a computing service, where the device code block is used to represent a code segment running on a GPU.

[0070] In step S101, a device code block is extracted from the computing service. In the embodiment of the present application, the device code block is used to represent a code snippet running on the GPU, that is, a CUDA code used to execute a specific computing service. Specifically, the code storage path of the computing service is first specified, all files in the directory are traversed and searched, and then the files containing the GPU device code are identified through the command line tool and the "cuobjdump file name" instruction, and finally these files are stored in the specified directory.

[0071] Optionally, in the embodiment of the present application, computing services refer to tasks or programs that require computing and data processing, which may include various computing-intensive applications such as data analysis, image processing, scientific computing, machine learning, etc. For example, image processing computing services may include operations such as image enhancement, edge detection, and image segmentation.

[0072] In addition, as a preferred embodiment of the present application, all files are prioritized, and device code blocks are extracted in sequence according to the priority ranking results. Specifically, the priority score of each file is calculated using the dynamic content similarity and path adaptation formula, and the priority of the files is sorted in descending order according to the priority score, ensuring that important and relevant files are processed first. First, the file content similarity is calculated, and for each file, the device code block is extracted in sequence. Extract its content and represent it as a vector. Optionally, existing methods such as word frequency, word frequency-inverse document frequency, etc. can be used to convert the file content into a vector. Then, the cosine similarity calculation method is used to calculate the file content vector and the overall content library vector. The cosine similarity between the two is used to obtain the similarity score between the file content and the overall content library, which is used to evaluate the relevance of the file. The calculation formula is as follows:

[0073] Then, for each file, calculate the length of the path from the root directory to the file , path length usually refers to the number of directory levels contained in the path from the root directory to the file. The path length is calculated by counting the number of directory levels contained in the path from the root directory to the file. Then use the following formula to standardize the path length:

[0074] in It is the minimum value among all file path lengths. Find the minimum value among all file path lengths. This value represents the length of the shortest path. Is the maximum value of all file path lengths. Find the maximum value among all file path lengths, which represents the length of the longest path. Convert the path length to a standardized value so that shorter paths get lower scores (i.e., higher priority). Use the standardized formula to convert the path length to a value between 0 and 1, so that the path lengths of different files can be compared within a unified range.

[0075] Then, record the size of each file , the file size is normalized using the following formula:

[0076] in is the minimum value among all files, is the maximum value among all files. Normalizes file sizes so that larger files are appropriately adjusted in priority calculations.

[0077] Finally, the priority score of each file is calculated using the following formula:

[0078] The above formula is designed to consider file content similarity, path length normalization, and file size normalization to determine the processing priority of a file. Using content similarity as the numerator emphasizes the relevance of the file content to the overall content library. In the formula, content similarity is weighted by the normalized value of path length. This structure shows that the impact of file similarity on priority adjusts as the path length changes. Files with higher content similarity will have higher priority. An exponential function is used to amplify the impact of similarity, so that the priority of files with higher similarity increases rapidly. The exponential function can increase priority nonlinearly, ensuring that files with high content relevance are given more attention. The normalized value of path length is used as the denominator, which reflects the accessibility of the file relative to the storage system. Files with shorter paths have higher priority because they are more easily accessible. By using the normalized value of path length as the denominator, the priority of files with longer paths can be reduced. The normalized value of file size is added to the denominator, which reflects the resource consumption of processing large files. When the file is larger, the priority decreases as the file size increases. This design takes into account the actual demand for processing resources and avoids excessive processing burden. In summary, the design of this formula structure comprehensively considers the influence of file content similarity, path length and file size, and balances these factors through exponential functions and normalization processing to reasonably evaluate the priority of files. This comprehensive design ensures the effectiveness of file processing and the rational allocation of resources.

[0079] Reference Figure 2, is a flow chart of extracting device code blocks from computing services in an embodiment of the present application. Specifically, first, the cubin file is extracted through the instructions "cuobjdump -lelf file name" and "cuobjdump -xelf .cubin file name", that is, the device code actually executed on the GPU is extracted from the compiled CUDA program. By extracting the cubin file through these instructions, the binary representation of the device code can be obtained, and these codes can be directly loaded and executed by the GPU. This step is to extract and analyze the code running on the GPU in advance, so as to avoid further analysis and transmission during actual operation, and improve the performance of remote calls. Subsequently, the ptx file, that is, the CUDA code in the intermediate representation form, is extracted through the instructions "cuobjdump -ptx file name" and "cuobjdump -xptx file name", which is convenient for further analysis and optimization. By extracting the ptx file, the intermediate representation of the CUDA program can be analyzed to understand the execution logic and data dependencies of the code. This is of great significance for optimizing and adjusting the remote call strategy of computing services. Then, the command "cuobjdump -lptx file name" is used to extract the arrangement order of the ptx file and cubin file in the binary. By extracting the arrangement order of the ptx file and cubin file, the specific location and order of these code blocks in the binary file can be clearly known, which is very important for understanding the execution flow and data dependencies of the code. Then, the command "nvcc --create" is used to organize the cubin file and ptx file into various fatbin files according to the arrangement order to ensure the correct loading and execution of the code on the GPU. At the same time, the command "cuobjdump -elf .cubin file" is used to extract the function definition file and parse the length of the parameters in the function. Extracting the function definition file can obtain the functions and their detailed information contained in each GPU device code block. This file usually includes the name of the function, return type, parameter type and number, etc. Parsing the length of the parameters in the function is to clarify the parameter data size required for each function call. This is very critical for preparing and transmitting parameter data in remote calls, because accurate parameter length information can ensure that there are no errors in the transmission and calling process. By understanding and parsing the function definition and its parameters in advance, optimization can be performed in advance in the remote call strategy, such as pre-allocating memory, optimizing parameter transmission methods, etc., thereby improving the overall performance. Finally, the collected fatbin files and function analysis results are compressed and stored in the database.

[0080] Step S102: pre-process the extracted device code block, analyze the computing service and determine the best remote call strategy.

[0081] In step S102, the device code block is first formatted into a format suitable for transmission and storage, and a data compression operation is performed to reduce the size of data transmission, thereby reducing bandwidth requirements and transmission time. For example, a PTX file is converted into a standard data format (such as JSON or binary format) for easy transmission and storage on the network. Subsequently, the computing business to be processed is analyzed, including data packet size, dependent CUDA library, etc., to determine the best remote call strategy. In addition, the extracted device code block is analyzed to determine the CUDA library it depends on, evaluate the data packet size and code block complexity, and select an appropriate compression and transmission strategy. Based on the results of data preprocessing and computing business analysis, the best remote call strategy is determined, including data transmission method and call timing. A remote call strategy is formulated to ensure efficient execution of data transmission and computing tasks. Determine the best time and method for data transmission to reduce latency and bandwidth occupancy.

[0082] Step S103: Execute the GPU remote call operation according to the determined remote call strategy, send the computing task to the remote GPU for execution, and obtain the execution result.

[0083] Reference Figure 3 , in step S103, a service process is started on a server with a GPU, and the process will process the computing request transmitted from the client. The server-side service needs to be connected to a database, which stores all processed fatbin files and function definition files, which include device code blocks and their related function definition information. The client is deployed on another server, and the computing service is started. The client is responsible for intercepting the API of the computing service and forwarding the request to the server. In implementation, the client calculates the MD5 value of the code block to be executed. The MD5 value is used to uniquely identify and verify the integrity of the data block. The calculated MD5 value is sent to the server as the unique identifier of the code block. The server queries the corresponding fatbin file and function definition file from the database according to the received MD5 value. The file contains the GPU code and function information to be executed. After the server obtains the code block to be run, it is loaded into the memory and calls the local class library to transfer the code to the GPU to perform function operations. After the calculation is completed, the server returns the result to the client, and the client receives the result and processes the subsequent business logic.

[0084] Step S104: Optimize performance according to the execution result of the remote call.

[0085] In step S104, the execution results are collected from the remote call after completion, including calculation results, log information, running time, system resource usage (such as CPU, GPU utilization and memory usage). The execution results are analyzed to calculate key performance indicators (KPIs), such as task execution time, data transmission delay, GPU computing efficiency, etc. Then, performance analysis tools (such as NVIDIA Nsight, CUDA Profiler, or custom monitoring tools) are used to identify possible bottlenecks, such as:

[0086] Data transmission bottleneck: network bandwidth limitation and high data transmission delay.

[0087] Computational bottleneck: Insufficient GPU computing resources or poor code optimization lead to low execution efficiency.

[0088] Memory bottleneck: Insufficient memory bandwidth or unreasonable memory allocation affects the overall performance.

[0089] Also analyze the root causes of bottlenecks, such as:

[0090] Whether there are large data packets or frequent data exchanges during data transmission.

[0091] The GPU computing cores are not used evenly or there is computing redundancy.

[0092] Whether there is a memory leak or insufficient memory allocation.

[0093] Based on the bottleneck analysis results, select a more efficient transmission protocol or adjust the existing protocol settings. For example, you may select a more suitable network protocol (TCP / UDP) or optimize the existing protocol parameters (such as packet size, transmission window). Re-evaluate and adjust the data compression strategy to further reduce the amount of transmitted data or improve compression efficiency. Evaluate the existing remote call strategy based on the performance analysis results and identify areas that need improvement. Develop a new remote call strategy, including:

[0094] Optimize data transmission method and timing: Ensure the best timing and minimum delay of data transmission.

[0095] Optimize computing task scheduling: adjust the scheduling strategy of computing tasks according to GPU load and task priority.

[0096] Improve error handling and recovery mechanisms: Improve the system's ability to handle failures and errors, and ensure the stability and reliability of remote calls.

[0097] In summary, the present application improves the efficiency of GPU remote computing by extracting, preprocessing, analyzing and optimizing device code blocks. The specific steps include: extracting device code blocks in computing services, performing data format conversion and compression to reduce data transmission, analyzing computing services to determine the best remote call strategy, and then executing remote calls according to the strategy, and optimizing performance during the execution process. This solution solves the problems of startup delays and long computing time caused by real-time analysis of business programs and high network bandwidth requirements in the prior art. Through preprocessing and optimization, it significantly reduces client resource consumption and network transmission burden, thereby improving the performance and computing efficiency of GPU remote calls.

[0098] Figure 4 This is a structural block diagram of a system for optimizing GPU remote call performance based on pre-analysis of computing services provided by an embodiment of the present application. The device includes at least the following modules:

[0099] A code extraction module is used to extract device code blocks from computing services. The device code blocks are used to represent code snippets running on the GPU.

[0100] The business analysis module is used to pre-process the extracted device code blocks, analyze the computing business and determine the best remote call strategy;

[0101] The remote call module is used to execute the GPU remote call operation according to the determined remote call strategy, send the computing task to the remote GPU for execution, and obtain the execution result;

[0102] The performance optimization module is used to optimize the performance according to the execution results of the remote call.

[0103] For relevant details, refer to the above method embodiment.

[0104] Figure 5 4 is a block diagram of an electronic device provided by an embodiment of the present application. The device at least includes a processor 401 and a memory 402.

[0105] The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 401 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0106] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, which is used to be executed by the processor 401 to implement the method for optimizing GPU remote call performance based on pre-analysis of computing services provided in the method embodiment of the present application.

[0107] In some embodiments, the electronic device may further optionally include: a peripheral device interface and at least one peripheral device. The processor 401, the memory 402 and the peripheral device interface may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface via a bus, a signal line or a circuit board. Schematically, the peripheral devices include but are not limited to: a radio frequency circuit, a touch display screen, an audio circuit, and a power supply.

[0108] Of course, the electronic device may also include fewer or more components, which is not limited in this embodiment.

[0109] Optionally, the present application also provides a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the method of optimizing GPU remote call performance based on pre-analysis of computing services in the above method embodiment.

[0110] Optionally, the present application also provides a computer product, which includes a computer-readable storage medium, in which a program is stored, and the program is loaded and executed by a processor to implement the method of optimizing GPU remote call performance based on pre-analysis of computing services in the above-mentioned method embodiment.

[0111] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for optimizing GPU remote call performance based on pre-analysis of computing services, It is characterized in that The method comprises: Extracting a device code block from a computing service, wherein the device code block is used to represent a code snippet running on a GPU; extracting a device code block from a computing service includes: specifying a code storage path for the computing service, traversing and searching all files in the directory; identifying files containing GPU device codes through a command line tool and an instruction named cuobjdump file name; storing the files in a specified directory; prioritizing all files, and extracting device code blocks in sequence according to the priority ranking results; calculating a priority score for each file using a dynamic content similarity and path adaptation formula, and sorting the priorities of the files in descending order according to the priority scores, wherein the priority score calculation formula is as follows: in, For the The content vector of each file and the vector of the entire content library The cosine similarity between For the The normalized path length of each file is For the The normalized size of the files; The identifying of the file containing the GPU device code by using the command line tool and the command named cuobjdump file name includes: extracting the cubin file by using the command named cuobjdump -lelf file name and cuobjdump -xelf .cubin file name; extracting the ptx file by using the command named cuobjdump -ptx file name and cuobjdump -xptx file name; extracting the arrangement order of the ptx file and the cubin file in the binary by using the command named cuobjdump -lptx file name; organizing the cubin file and the ptx file into various fatbin files according to the arrangement order by using the command named nvcc –create; Preprocessing the extracted device code block, analyzing the computing business and determining an optimal remote call strategy; According to the determined remote call strategy, a GPU remote call operation is executed, the computing task is sent to the remote GPU for execution, and the execution result is obtained; Performance optimization is performed according to the execution result of the remote call.

2. The method for optimizing GPU remote call performance based on pre-analysis of computing services according to claim 1, It is characterized in that The method of identifying the file containing the GPU device code by using the command line tool and the command named cuobjdump file name also includes: Extract the function definition file through the command named cuobjdump -elf .cubin file name and parse the length of the parameters in the function; The collected fatbin files and function analysis results are compressed and stored in the database.

3. The method for optimizing GPU remote call performance based on pre-analysis of computing services according to claim 1, It is characterized in that The step of executing the GPU remote call operation according to the determined remote call strategy, sending the computing task to the remote GPU for execution, and obtaining the execution result includes: Start a service process on a server with a GPU to process computing requests transmitted from the client; Connect the server to the database that stores the fatbin file and function definition file, deploy the client on another server, start the computing service, and the client is responsible for intercepting the API of the computing service and forwarding the request to the server. The client calculates the MD5 value of the code block to be executed and sends the calculated MD5 value to the server. The server queries the corresponding fatbin file and function definition file from the database based on the received MD5 value. After the server obtains the code block to be run, it loads it into memory and calls the local class library to transfer the code to the GPU to perform function operations. After the calculation is completed, the server returns the result to the client, and the client receives the result and processes subsequent business logic.

4. The method for optimizing GPU remote call performance based on pre-analysis of computing services according to claim 1, It is characterized in that The preprocessing of the extracted device code block, analyzing the computing service and determining the best remote call strategy includes: Perform format conversion on the device code block to convert it into a format suitable for transmission and storage, and perform data compression operations to reduce the size of data transmission; Analyze the computing business to be processed, including the data packet size and dependent CUDA library, and determine the best remote call strategy; Analyze the extracted device code blocks, determine the CUDA libraries they depend on, evaluate the packet size and code block complexity, and select appropriate compression and transmission strategies; Determine the best remote call strategy based on the results of data preprocessing and computing business analysis.

5. A system for optimizing GPU remote call performance based on pre-analysis of computing services, It is characterized in that include: A code extraction module, used to extract a device code block from a computing service, wherein the device code block is used to represent a code fragment running on a GPU; The method of extracting the device code block from the computing service includes: specifying the code storage path of the computing service, traversing and searching all files in the directory; identifying the file containing the GPU device code through the command line tool and the instruction named cuobjdump file name; storing the file in the specified directory; sorting all the files by priority, and extracting the device code block in sequence according to the result of the priority sorting; calculating the priority score of each file by using the dynamic content similarity and path adaptation formula, and sorting the priority of the files in descending order according to the priority score, and the calculation formula of the priority score is as follows: in, For the The content vector of each file and the vector of the entire content library The cosine similarity between For the The normalized path length of each file is For the The normalized size of the files; The identifying of the file containing the GPU device code by using the command line tool and the command named cuobjdump file name includes: extracting the cubin file by using the command named cuobjdump -lelf file name and cuobjdump -xelf .cubin file name; extracting the ptx file by using the command named cuobjdump -ptx file name and cuobjdump -xptx file name; extracting the arrangement order of the ptx file and the cubin file in the binary by using the command named cuobjdump -lptx file name; organizing the cubin file and the ptx file into various fatbin files according to the arrangement order by using the command named nvcc –create; A business analysis module, used to pre-process the extracted device code block, analyze the computing business and determine the best remote call strategy; A remote call module is used to execute a GPU remote call operation according to the determined remote call strategy, send the computing task to the remote GPU for execution, and obtain the execution result; The performance optimization module is used to perform performance optimization according to the execution result of the remote call.

6. An electronic device, It is characterized in that The device includes a processor and a memory; a program is stored in the memory, and the program is loaded and executed by the processor to implement a method for optimizing GPU remote call performance based on pre-analysis of computing services as described in any one of claims 1 to 4.

7. A computer-readable storage medium, It is characterized in that The storage medium stores a program, which, when executed by a processor, is used to implement a method for optimizing GPU remote call performance based on pre-analysis of computing services as described in any one of claims 1 to 4.

Images