Gpu resource remote invocation method and apparatus, device, and storage medium

Abstract

The application provides a GPU resource remote calling method and device, equipment and storage medium, the method can be applied to the first electronic equipment, including: the first electronic equipment re-implements the first function and the second function in the CUDA kernel function; the first electronic equipment parses the first function, obtains the data block of target function body, the first electronic equipment parses the second function, obtains the content of kernel function parameter, the data block of target function body and the content of kernel function parameter are all the contents pointed to by the typeless pointer parameter; the first electronic equipment sends the data block of target function body and the content of kernel function parameter to the second electronic equipment, so that the second electronic equipment executes new CUDA kernel function, calls the GPU resource in the second electronic equipment, and the new CUDA kernel function is generated by the second electronic equipment according to the data block of target function body and the content of kernel function parameter. In this way, the first electronic equipment can remotely call the GPU resource of the second electronic equipment.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for remotely accessing GPU resources. Background Technology

[0002] With the rapid development of cloud-native technologies, various container orchestration systems have emerged, among which Kubernetes is widely used in industry for building private container clouds. Kubernetes supports the operation of business containers by managing various computing, network, and storage resources. A Kubernetes cloud platform consists of multiple nodes; some nodes have Graphics Processing Unit (GPU) resources, while others do not. For nodes without GPU resources, they can access the GPU resources of other nodes through remote calls.

[0003] When accessing GPU resources, CUDA kernel functions are typically used for remote calls. However, when using CUDA kernel functions to remotely access GPU resources, the CUDA kernel functions may not function properly on the remote end, resulting in the inability to access the GPU resources of electronic devices that have GPU resources. Summary of the Invention

[0004] This application provides a method, apparatus, device, and storage medium for remotely invoking GPU resources, enabling electronic devices to successfully perform remote invoking of GPU resources.

[0005] In a first aspect, this application provides a method for remotely invoking GPU resources, applied to a first electronic device, comprising:

[0006] The first electronic device reimplements the first and second functions in the CUDA kernel function of the parallel computing architecture, wherein the CUDA kernel function is the kernel function used by the first electronic device when remotely calling the GPU resources of the second electronic device;

[0007] The first electronic device parses the first function to obtain the data block of the target function body. The first electronic device parses the second function to obtain the content of the kernel function parameters. The data block of the target function body and the content of the kernel function parameters are both contents pointed to by typeless pointer parameters.

[0008] The first electronic device sends the data block of the target function body and the contents of the kernel function parameters to the second electronic device, so that the second electronic device executes a new CUDA kernel function and calls the GPU resources in the second electronic device. The new CUDA kernel function is generated by the second electronic device based on the data block of the target function body and the contents of the kernel function parameters.

[0009] In one possible implementation, the first electronic device parses the first function to obtain a data block of the target function body, including:

[0010] The first electronic device obtains the location of the first function from the declaration information of the first function, and the location of the first function is used to indicate the position of the first function in the header file of the CUDA kernel function;

[0011] The first electronic device obtains the data block of the target function body and a first pointer to the data block from the header file of the CUDA kernel function, based on the location of the first function.

[0012] In one possible implementation, the CUDA kernel function further includes a third function;

[0013] The first electronic device sends the data block of the target function body to the second electronic device, including:

[0014] The first electronic device obtains the kernel function name in the third function according to the first pointer, and the kernel function name corresponds to the data block of the target function body;

[0015] The first electronic device sends the data block of the target function body and the name of the kernel function to the second electronic device.

[0016] In one possible implementation, the first electronic device parses the second function to obtain the contents of the kernel function parameters, including:

[0017] The first electronic device parses the contents of the header file of the second function in the CUDA kernel function to obtain the number of kernel function parameters;

[0018] The first electronic device obtains the corresponding values ​​of the kernel function parameters from the header file of the CUDA kernel function based on the kernel function name and the number of kernel function parameters.

[0019] Secondly, this application provides a method for remotely invoking GPU resources, applied to a second electronic device, the method comprising:

[0020] The second electronic device receives and stores the data block of the target function body and the contents of the kernel function parameters sent by the first electronic device;

[0021] The second electronic device generates a new CUDA kernel function based on the data block of the target function body and the contents of the kernel function parameters;

[0022] The second electronic device executes the new CUDA kernel function to enable the first electronic device to remotely invoke the GPU resources in the second electronic device.

[0023] In one possible implementation, the content of the kernel function parameter includes the numerical value corresponding to the kernel function parameter;

[0024] The second electronic device receives and stores the data block of the target function body and the contents of the kernel function parameters sent by the first electronic device, including:

[0025] The second electronic device receives the data block of the target function body and loads the data block of the target function body into the third function;

[0026] The second electronic device receives the numerical values ​​corresponding to the kernel function parameters and the data blocks of other function bodies, and stores the numerical values ​​corresponding to the kernel function parameters and the data blocks of other function bodies in the target function. The data blocks of other function bodies include the function bodies in the CUDA kernel function other than the target function body.

[0027] In one possible implementation, the second electronic device generates a new CUDA kernel function based on the data block of the target function body and the contents of the kernel function parameters, including:

[0028] The second electronic device generates the new CUDA kernel function based on the third function and the target function.

[0029] In one possible implementation, the second electronic device executes the new CUDA kernel function to enable the first electronic device to remotely invoke GPU resources in the second electronic device, including:

[0030] The second electronic device receives a call request from the first electronic device, which is sent when the first electronic device is processing a target service, and the target service is a service that requires calling the GPU resources of the second electronic device;

[0031] The second electronic device invokes the new CUDA kernel function according to the invocation request, so that the first electronic device can remotely invoke the GPU resources in the second electronic device.

[0032] Thirdly, this application provides a GPU resource remote invocation device, the GPU resource remote invocation device comprising:

[0033] The processing module is used to reimplement the first and second functions in the CUDA kernel function of the parallel computing architecture, wherein the CUDA kernel function is the kernel function used by the first electronic device to remotely call the GPU resources of the second electronic device;

[0034] The processing module is further configured to parse the first function to obtain a data block of the target function body, and the first electronic device to parse the second function to obtain the content of the kernel function parameters. The data block of the target function body and the content of the kernel function parameters are both contents pointed to by typeless pointer parameters.

[0035] The sending module is used to send the data block of the target function body and the contents of the kernel function parameters to the second electronic device, so that the second electronic device executes a new CUDA kernel function and calls the GPU resources in the second electronic device. The new CUDA kernel function is generated by the second electronic device based on the data block of the target function body and the contents of the kernel function parameters.

[0036] In one possible implementation, the processing module is specifically used to obtain the position of the first function in the declaration information of the first function, the position of the first function being used to indicate the position of the first function in the header file of the CUDA kernel function; and based on the position of the first function, to obtain the data block of the target function body and a first pointer to the data block in the header file of the CUDA kernel function.

[0037] In one possible implementation, the CUDA kernel function further includes a third function; the sending module is further configured to obtain the kernel function name in the third function according to the first pointer, the kernel function name corresponding to the data block of the target function body; and send the data block of the target function body and the kernel function name to the second electronic device.

[0038] In one possible implementation, the processing module is specifically used to parse the content corresponding to the second function in the header file of the CUDA kernel function to obtain the number of kernel function parameters; and to obtain the numerical values ​​corresponding to the kernel function parameters in the header file of the CUDA kernel function based on the kernel function name and the number of kernel function parameters.

[0039] Fourthly, this application provides a GPU resource remote invocation device, the GPU resource remote invocation device comprising:

[0040] The receiving module is used to receive and store the data blocks of the target function body and the contents of the kernel function parameters sent by the first electronic device;

[0041] The generation module is used to generate a new CUDA kernel function based on the data block of the target function body and the content of the kernel function parameters;

[0042] The calling module is used to execute the new CUDA kernel function so that the first electronic device can remotely call the GPU resources in the second electronic device.

[0043] In one possible implementation, the content of the kernel function parameter includes the numerical value corresponding to the kernel function parameter;

[0044] The receiving module is further configured to receive the data block of the target function body and load the data block of the target function body into the third function; the second electronic device receives the value corresponding to the kernel function parameter and the data block of other function bodies, and stores the value corresponding to the kernel function parameter and the data block of other function bodies in the target function, wherein the data block of other function bodies includes function bodies in the CUDA kernel function other than the target function body.

[0045] In one possible implementation, the generation module is specifically used by the second electronic device to generate the new CUDA kernel function based on the third function and the target function.

[0046] In one possible implementation, the receiving module is further configured to receive a call request from the first electronic device, the call request being sent when the first electronic device is processing a target service, the target service being a service that requires calling the GPU resources of the second electronic device.

[0047] The calling module is specifically used to call the new CUDA kernel function according to the calling request, so that the first electronic device can remotely call the GPU resources in the second electronic device.

[0048] Fifthly, this application also provides an electronic device, which includes: a processor and a memory communicatively connected to the processor;

[0049] The memory stores computer-executed instructions;

[0050] The processor executes computer execution instructions stored in the memory to implement the method described in any possible implementation of the first or second aspect above.

[0051] Sixthly, this application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method described in any possible implementation of the first or second aspect above.

[0052] In a seventh aspect, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the method described in any possible implementation of the first or second aspect above.

[0053] Therefore, this application provides a method, apparatus, device, and storage medium for remotely invoking GPU resources. The method is applied to a first electronic device and includes: the first electronic device reimplementing a first function and a second function in the CUDA kernel function of a parallel computing architecture. The CUDA kernel function is used by the first electronic device to remotely invoke GPU resources of a second electronic device; the first electronic device parses the first function to obtain a data block of the target function body; the first electronic device parses the second function to obtain the content of the kernel function parameters. Both the data block of the target function body and the content of the kernel function parameters are content pointed to by typeless pointer parameters; the first electronic device sends the data block of the target function body and the content of the kernel function parameters to the second electronic device, so that the second electronic device executes the new CUDA kernel function and invokes GPU resources in the second electronic device. The new CUDA kernel function is generated by the second electronic device based on the data block of the target function body and the content of the kernel function parameters. In this way, by parsing the CUDA kernel function of the first electronic device, obtaining information about the typeless parameters in the CUDA kernel function, and sending this parameter information to the second electronic device, the CUDA kernel function can be regenerated in the second electronic device. This allows the second electronic device to successfully invoke GPU resources by executing the regenerated CUDA kernel function, thus enabling remote invocation of GPU resources. Attached Figure Description

[0054] Figure 1 A flowchart illustrating a method for remotely invoking GPU resources in a first electronic device, provided as an embodiment of this application;

[0055] Figure 2 A flowchart illustrating a method for remotely invoking GPU resources in a second electronic device, provided as an embodiment of this application;

[0056] Figure 3 A schematic diagram of a CUDA kernel function in a first electronic device and a new CUDA kernel function in a second electronic device, provided for embodiments of this application;

[0057] Figure 4 This is a schematic diagram of the structure of a GPU resource remote access device provided in an embodiment of this application;

[0058] Figure 5 A schematic diagram of another GPU resource remote access device provided in this application embodiment;

[0059] Figure 6 This is a schematic diagram of an electronic device structure provided in this application.

[0060] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0062] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0063] Kubernetes is an open-source application used to manage containerized applications across multiple hosts in a cloud platform. Kubernetes aims to make deploying containerized applications simple and efficient, providing a mechanism for application deployment, updates, and maintenance. In this embodiment, the cloud platform managed by Kubernetes is referred to as the Kubernetes cloud platform.

[0064] With the rapid development of artificial intelligence technology, the demand for high computing power from various machine learning algorithms is increasing daily, and model training has shifted from the Central Processing Unit (CPU) to GPUs. The Kubernetes cloud platform includes multiple electronic devices, each of which can be considered a node within the platform. Some electronic devices on the Kubernetes cloud platform have local GPU resources, while others do not. A key aspect of cloud computing GPU management is the pooling of GPU resources. This involves unified management of all GPUs and provisioning resources locally or remotely. This allows services on nodes on the Kubernetes cloud platform that lack GPU resources to utilize GPU computing power, ensuring that the computing power on nodes with GPU resources is not idle. Therefore, the Kubernetes cloud platform can provide pooled GPU computing power services.

[0065] CUDA is a general-purpose parallel computing architecture programming language proposed by NVIDIA. It simplifies GPU programming to some extent. Kernel functions are the core of CUDA programming; they are function bodies that run on the GPU. When users make remote calls using CUDA kernel functions, they can directly use the more user-friendly runtime API, i.e., the CUDA runtime API, or they can use the driver API, which is closer to the underlying driver.

[0066] However, when electronic device A uses CUDA kernel functions to call GPU resources of electronic device B, there are untyped pointer parameters in the CUDA kernel functions of electronic device A. For example, func and args in the cudaLaunchKernel function are void pointers. Electronic device B cannot determine the contents of these untyped pointer parameters. Therefore, the CUDA kernel functions of electronic device A do not work in electronic device B, and there may be cases where remote calls to GPU resources fail.

[0067] Based on this, this application provides a method for remote GPU resource invocation. It intercepts kernel-related content in the local runtime API, parses relevant data within the CUDA kernel function by reimplementing the CUDA kernel function, including the content pointed to by the typeless pointer parameter, and sends this data to the remotely invoked electronic device. This allows the remote electronic device to obtain all the content of the CUDA kernel function, resulting in a new CUDA kernel function with all its contents. Thus, when the new CUDA kernel function is executed remotely, it can invoke the GPU resources of the remote device, achieving remote GPU resource invocation via the CUDA kernel function.

[0068] The GPU resource remote invocation method provided in this application will be described in detail below through specific embodiments. It is understood that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0069] The GPU resource remote invocation method provided in this application can be applied to a Kubernetes cloud platform, which includes a first electronic device and a second electronic device. This application uses the example of a first electronic device invoking the GPU resources of a second electronic device for illustration, but this does not constitute any limitation.

[0070] For example, the first electronic device may or may not contain GPU resources; this application embodiment does not limit this. The second electronic device contains GPU resources. However, the services that the first electronic device can process include those that require the GPU resources of the remote second electronic device.

[0071] It should be noted that, in the embodiments of this application, the first electronic device that calls the GPU resources can be referred to as the client, and the second electronic device that calls the GPU resources can be referred to as the server.

[0072] The functions of the first and second electronic devices during remote GPU resource invocation are described below.

[0073] Figure 1 This is a flowchart illustrating a method for remotely invoking GPU resources in a first electronic device, as provided in an embodiment of this application. The method can be executed by software and / or hardware devices; for example, the hardware device can be a GPU resource remote invoking apparatus, which can be the first electronic device or a processing chip within the first electronic device.

[0074] For example, such as Figure 1As shown, this GPU resource remote invocation method may include:

[0075] S101. The first electronic device reimplements the first and second functions in the CUDA kernel function of the parallel computing architecture. The CUDA kernel function is the kernel function used by the first electronic device when remotely calling the GPU resources of the second electronic device.

[0076] In this embodiment, the CUDA kernel function can be a kernel function of the CUDA runtime API that is hijacked when the first electronic device remotely calls the GPU resources of the second electronic device. The first function can be the _cudaRegisterFaBinary function in the CUDA kernel function. The second function can be the cudaLaunchKernel function in the CUDA kernel function.

[0077] For example, the first electronic device can analyze the intermediate process file compiled by nvcc, obtain the referenced CUDA header file from the intermediate process file, and CUDA obtains the _cudaRegisterFaBinary function from the header file. Further, the _cudaRegisterFaBinary function is reimplemented in the code of the first electronic device (client side).

[0078] Intermediate process files for nvcc compilation can be retained when nvcc-keep is executed.

[0079] S102. The first electronic device parses the first function to obtain the data block of the target function body. The first electronic device parses the second function to obtain the content of the kernel function parameters. The data block of the target function body and the content of the kernel function parameters are both the content pointed to by the typeless pointer parameters.

[0080] For example, when the first electronic device parses the first function and obtains the data block of the target function body, the first electronic device can obtain the location of the first function from the declaration information of the first function. The location of the first function indicates the position of the first function in the header file of the CUDA kernel function. The first electronic device can also obtain the data block of the target function body and a first pointer to the data block in the header file of the CUDA kernel function based on the location of the first function.

[0081] In this way, the data block of the target function body can be obtained from the header file of the CUDA kernel function through the declaration information of the first function. This allows the second electronic device to know the content pointed to by the typeless pointer parameter by sending the data block of the target function body, so as to facilitate the use of GPU resources.

[0082] In this embodiment of the application, when the first electronic device parses the second function to obtain the content of the kernel function parameters, the first electronic device can parse the content corresponding to the second function in the header file of the CUDA kernel function to obtain the number of kernel function parameters; the first electronic device can obtain the value corresponding to the kernel function parameters in the header file of the CUDA kernel function according to the kernel function name and the number of kernel function parameters.

[0083] For example, by reimplementing the `cudaLaunchKernel` function on the client side, all kernel function parameters represented by `void**args` can be parsed inside the `cudaLaunchKernel` function to obtain their specific values. Furthermore, by analyzing the header files referenced by the nvcc compiler intermediate files and finding that `void**args` is `*char[]`, the number of kernel function parameters can be obtained. The `c++filt` tool can be used to parse the kernel function names in the above embodiment to obtain the kernel function declarations and parameter types, and then obtain the corresponding values ​​of the kernel function parameters.

[0084] When the second electronic device executes a remote call through the CUDA kernel function of the first electronic device, it cannot obtain the number of kernel function parameters and their corresponding values. Therefore, the first electronic device can parse out the number of kernel function parameters and their corresponding values, allowing the second electronic device to know the number of kernel function parameters and their corresponding values ​​when executing the remote call, thus successfully enabling remote access to the GPU resources of the second electronic device.

[0085] S103. The first electronic device sends the data block of the target function body and the contents of the kernel function parameters to the second electronic device so that the second electronic device executes the new CUDA kernel function and calls the GPU resources in the second electronic device. The new CUDA kernel function is generated by the second electronic device based on the contents of the data block of the target function body and the kernel function parameters.

[0086] In this embodiment, the CUDA kernel function may further include a third function, which may be the _cudaRegisterFunction function.

[0087] For example, before the first electronic device sends the data block of the target function body to the second electronic device, the first electronic device can obtain the kernel function name in the third function according to the first pointer obtained in the above steps. The kernel function name corresponds to the data block of the target function body.

[0088] When the first electronic device sends the data block of the target function body and the contents of the kernel function parameters to the second electronic device, the first electronic device can also send the data block of the target function body and the kernel function name to the second electronic device.

[0089] In this way, the second electronic device can obtain the content pointed to by the typeless pointer parameter in the CUDA kernel function of the first electronic device, so that when the second electronic device performs a remote call, it can make a call in the GPU according to the data block of the target function body and the name of the kernel function.

[0090] Therefore, the GPU resource remote invocation method provided in this application parses the CUDA kernel function of the first electronic device, obtains the content pointed to by the typeless parameter in the CUDA kernel function, and sends this content to the second electronic device. In this way, the second electronic device can obtain the content pointed to by the typeless parameter, making the new CUDA kernel function generated by the second electronic device more complete, thereby realizing the first electronic device's invocation of the GPU resources of the second electronic device.

[0091] Figure 2 This is a flowchart illustrating a method for remotely invoking GPU resources in a second electronic device, as provided in an embodiment of this application. The method can be executed by software and / or hardware devices; for example, the hardware device can be a GPU resource remote invoking apparatus, which can be a second electronic device or a processing chip within that device.

[0092] For example, such as Figure 2 As shown, this GPU resource remote invocation method may include:

[0093] S201. The second electronic device receives and stores the data block of the target function body and the contents of the kernel function parameters sent by the first electronic device.

[0094] The kernel function parameters include the corresponding numerical values, which can be found in the above embodiments and will not be repeated here.

[0095] For example, when the second electronic device receives and stores the data block of the target function body and the contents of the kernel function parameters sent by the first electronic device, the second electronic device can receive the data block of the target function body and load the data block of the target function body into the third function; the second electronic device can also receive the values ​​corresponding to the kernel function parameters and the data blocks of other function bodies, and store the values ​​corresponding to the kernel function parameters and the data blocks of other function bodies in the target function, wherein the data blocks of other function bodies include the function bodies in the CUDA kernel function other than the target function body.

[0096] The third function can be the _cudaRegisterFunction function in the second electronic device, and the target function can be the cudaLaunchKernel function in the second electronic device.

[0097] In this way, by storing the data blocks of the target function body and the contents of the kernel function parameters, the second electronic device can obtain the stored data blocks of the target function body and the contents of the kernel function parameters when the first electronic device makes a remote call. In addition, the second electronic device also receives data blocks of other function bodies, which makes the contents of the new CUDA kernel function generated by the second electronic device more complete and can improve the success rate of the GPU resources of the second electronic device executing the new CUDA kernel function call.

[0098] S202, the second electronic device generates a new CUDA kernel function based on the data block of the target function body and the contents of the kernel function parameters.

[0099] For example, when the second electronic device generates a new CUDA kernel function based on the data block of the target function body and the content of the kernel function parameters, the second electronic device can generate a new CUDA kernel function based on the third function and the target function.

[0100] For example, the second electronic device can obtain the third function and the target function, and encode the third function and the target function into the form of a CUDA kernel function to generate a new CUDA kernel function.

[0101] In this way, the new CUDA kernel function generated by the second electronic device not only includes the content pointed to by the typeless pointer parameter, but also the data blocks of other function bodies, making the generated new CUDA kernel function more complete. This allows the CUDA kernel function of the first electronic device to be reproduced in the second electronic device, enabling the second electronic device to successfully call GPU resources.

[0102] S203. The second electronic device executes a new CUDA kernel function so that the first electronic device can remotely call the GPU resources in the second electronic device.

[0103] For example, when the second electronic device executes a new CUDA kernel function to enable the first electronic device to remotely call the GPU resources in the second electronic device, the second electronic device can receive a call request from the first electronic device. The call request is sent when the first electronic device is processing a target service, which is a service that requires calling the GPU resources of the second electronic device. Based on the call request, the second electronic device calls the new CUDA kernel function to enable the first electronic device to remotely call the GPU resources in the second electronic device.

[0104] The target business can be AI business or other business, and this application embodiment does not limit it.

[0105] For example, when the first electronic device is processing the target service, the first electronic device can remotely call the remote call module in the second electronic device through the hijacked interface, namely the CUDA kernel function. The remote call module can execute a new CUDA kernel function to call the GPU resources in the second electronic device, thereby enabling the first electronic device to remotely call the GPU resources of the second electronic device.

[0106] In this way, the second electronic device can execute new CUDA kernel functions to call upon the GPU resources of the second electronic device, thereby enabling the first electronic device to remotely call upon the GPU resources of the second electronic device.

[0107] Therefore, the GPU resource remote invocation method provided in this application embodiment allows the second electronic device to receive the content pointed to by the typeless pointer parameter in the CUDA kernel function of the first electronic device, resulting in a higher degree of completeness in the new CUDA kernel function generated by the second electronic device based on this content. Thus, when the second electronic device executes the new CUDA kernel function, it can invoke GPU resources.

[0108] To facilitate understanding of the GPU resource remote invocation method described in the embodiments of this application, the method will be described in detail below with specific examples. For details, please refer to... Figure 3 As shown, Figure 3 This is a schematic diagram of a CUDA kernel function in a first electronic device and a new CUDA kernel function in a second electronic device, provided as embodiments of this application.

[0109] like Figure 3 As shown, the CUDA kernel functions in the first electronic device (client side) include the functions _cudaRegisterFatBinary, _cudaRegisterFunction, _cudaSetupArgSimple, and cudaLaunchKernel. The new CUDA kernel functions in the second electronic device (server side) can also include the functions _cudaRegisterFatBinary, _cudaRegisterFunction, _cudaSetupArgSimple, and cudaLaunchKernel.

[0110] The `cudaLaunchKernel` function is an API that the CUDA kernel functions need to call to run. The `_cudaRegisterFatBinary` and `_cudaRegisterFunction` functions are implicitly called by the CUDA compiler `nvcc` and can be used to prepare the parameters `func` for the `cudaLaunchKernel` function. The `_cudaSetupArgSimple` function is a macro in a header file implicitly called by the `nvcc` compiler. It is called every time kernel function parameters are set, recording all parameter addresses and passing them as `args` to `cudaLaunchKernel`.

[0111] It's important to note that when remotely executing CUDA kernel functions, the `gridDim`, `blockDim`, `sharedMem`, and `stream` methods in `cudaLaunchKernel` are concrete structures that can be directly converted into data blocks and passed to the second electronic device. However, `func` and `args` are void pointers. Directly passing the local address (i.e., the address of the first electronic device) to the remote machine (i.e., the second electronic device) might fail due to the void pointers of `func` and `args`. Therefore, the key to remote execution via `cudaLaunchKernel` is obtaining the data blocks pointed to by `func` and `args` and then reconstructing them on the remote end (the second electronic device).

[0112] In this embodiment, specific structures such as gridDim, blockDim, sharedMem, and stream in the first electronic device can be directly sent to the second electronic device via Transmission Control Protocol / Internet Protocol (TCP / IP) or Remote Direct Memory Access (RDMA). The first electronic device can parse func and args to obtain the data block pointed to by func and args, and send the data block pointed to by func and args to the second electronic device via TCP / IP or RDMA to generate a new CUDA kernel function in the second electronic device.

[0113] For example, when the server calls the GPU, it can do so through the CUDA driver API and the NVIDIA driver API. This application embodiment does not limit this.

[0114] For example, the method by which the first electronic device parses the data block and the second electronic device generates and executes the new CUDA kernel function may include the following steps:

[0115] Step 1: By analyzing the intermediate process files compiled by nvcc, obtain the referenced CUDA header files from the intermediate process files, obtain the function declaration information of _cudaRegisterFaBinary and _cudaRegisterFunction from the header files, and reimplement these two functions in the client-side code.

[0116] Step 2: Analyze the header file referenced in the intermediate compilation file corresponding to the _cudaRegisterFatBinary function. The data type is _fatBinC_Wrapper_t, and the data in _fatBinC_Wrapper_t corresponds to the kernel function body.

[0117] For example, the function declaration for `_cudaRegisterFatBinary` is `void**_cudaRegisterFatBinary(void*fatCubin)`. Here, `void*fatCubin` is the location where the compiled kernel function's overall information is placed. The data type is `_fatBinC_Wrapper_t`, and the `data` field within `_fatBinC_Wrapper_t` corresponds to the kernel function body structure `computeFatBinaryFormat_t`.

[0118] During the reimplementation of the _cudaRegisterFatBinary function, after setting fatCubinHandle = (_fatBinC_Wrapper_t*)fatCubin->data, it returns (void*)fatCubinHandle, which is a pointer to the block data.

[0119] Step 3: During the reimplementation of the _cudaRegisterFunction function, you can record the compiled kernel function name, i.e., deviceFun. You can also pass fatBinData = (computeFatBinaryFormat_t)(*fatCubinHandle) to the server-side _cudaRegisterFunction function. The server-side _cudaRegisterFunction function receives the data block, saves it, and records it as S_fatBinData.

[0120] For example, the function declaration for _cudaRegisterFunction is void __cudaRegisterFunction(void**fatCubinHandle,constchar*hostFun,char*deviceFun,constchar*deviceName,intthread_limit,uint3*tid,uint3*bid,dim3*bDim,dim3*gDim,int*wSize). The parameter fatCubinHandle is the content returned in step 2.

[0121] Step 4: Reimplement the `cudaLaunchKernel` function on the client side. Inside the `cudaLaunchKernel` function, all kernel function parameters represented by `void**args` can be parsed to obtain their specific values, which are then passed to the server side. By analyzing the header files referenced in the nvcc compiler intermediate file, we can determine that `void**args` is `*char[]`, thus obtaining the number of kernel function parameters. Then, using the `c++filt` tool to parse the kernel function name `deviceFun` recorded in Step 3, we can obtain the kernel function declaration and parameter types, and thus the corresponding values ​​of the kernel function parameters. The client side can pass all values ​​to the `cudaLaunchKernel` function on the server side. The server side receives and saves these values, recording them as `S_kfargs`. Other structure parameters are also passed to the server side as data blocks, respectively recorded as `S_gridDim`, `S_blockDim`, `S_sharedMem`, and `S_stream`.

[0122] Step 5: In the cudaLaunckKernel function on the server side, load the kernel function information obtained from S_fatBinData in step 3 through cuda driver api cuModuleLoadData and record it as S_CUfunction.

[0123] Step 6: The server can obtain the new CUDA kernel function. The server can also call cuLaunchKernel(S_CUfunction,S_gridDim.x,S_gridDim.y,S_gridDim.z,S_blockDim.x,S_blockDim.y,S_blockDim.z,(unsignedint)S_sharedMem,(CUstream)S_str eam,S_kfargs,NULL) to complete the remote execution of the CUDA kernel function.

[0124] In summary, because `func` and `args` in the CUDA kernel function on the client side are void pointers, directly passing the local address to the remote device may not work on the server side. The method provided in this application allows the client to obtain the data block pointed to by `func` and `args` in the CUDA kernel function and send this data block to the server. This enables the address to be converted to the server's address after recovery on the server side, allowing the `cudaLaunchKernel` function to be reimplemented, thereby achieving remote access to GPU resources.

[0125] Figure 4 This is a schematic diagram of a GPU resource remote access device 40 provided in an embodiment of this application. For an example, please refer to [link to example]. Figure 4 As shown, the GPU resource remote access device 40 may include:

[0126] Processing module 401 is used to reimplement the first and second functions in the CUDA kernel function of the parallel computing architecture. The CUDA kernel function is the kernel function used by the first electronic device to remotely call the GPU resources of the second electronic device.

[0127] The processing module 401 is also used to parse the first function to obtain the data block of the target function body, and the first electronic device parses the second function to obtain the content of the kernel function parameters. The data block of the target function body and the content of the kernel function parameters are both contents pointed to by typeless pointer parameters.

[0128] The sending module 402 is used to send the data block of the target function body and the contents of the kernel function parameters to the second electronic device, so that the second electronic device executes the new CUDA kernel function and calls the GPU resources in the second electronic device. The new CUDA kernel function is generated by the second electronic device based on the contents of the data block of the target function body and the kernel function parameters.

[0129] In one possible implementation, the processing module 401 is specifically used to obtain the location of the first function from the declaration information of the first function, the location of the first function being used to indicate the location of the first function in the header file of the CUDA kernel function; based on the location of the first function, it obtains the data block of the target function body and the first pointer pointing to the data block from the header file of the CUDA kernel function.

[0130] In one possible implementation, the CUDA kernel function further includes a third function; the sending module 402 is also used to obtain the kernel function name in the third function according to the first pointer, the kernel function name corresponding to the data block of the target function body; and send the data block of the target function body and the kernel function name to the second electronic device.

[0131] In one possible implementation, the processing module 401 is specifically used to parse the contents corresponding to the second function in the header file of the CUDA kernel function to obtain the number of kernel function parameters; and to obtain the corresponding values ​​of the kernel function parameters in the header file of the CUDA kernel function based on the kernel function name and the number of kernel function parameters.

[0132] The GPU resource remote invocation device provided in this application embodiment can execute the technical solution of the GPU resource remote invocation method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the GPU resource remote invocation method. Please refer to the implementation principle and beneficial effects of the GPU resource remote invocation method. It will not be repeated here.

[0133] Figure 5 This is a schematic diagram of a GPU resource remote access device 50 provided in an embodiment of this application. For an example, please refer to [link to example]. Figure 5 As shown, the GPU resource remote access device 50 may include:

[0134] The receiving module 501 is used to receive and store the data block of the target function body and the contents of the kernel function parameters sent by the first electronic device.

[0135] The generation module 502 is used to generate a new CUDA kernel function based on the data block of the target function body and the contents of the kernel function parameters.

[0136] Module 503 is invoked to execute a new CUDA kernel function, enabling the first electronic device to remotely access GPU resources in the second electronic device.

[0137] In one possible implementation, the kernel function parameters include the numerical values ​​corresponding to the kernel function parameters. The receiving module 501 is further configured to receive data blocks of the target function body and load these data blocks into the third function; the second electronic device receives the numerical values ​​corresponding to the kernel function parameters and data blocks of other function bodies, and stores these values ​​and data blocks in the target function. The data blocks of other function bodies include function bodies in the CUDA kernel function other than the target function body.

[0138] In one possible implementation, the generation module 502 is specifically used by the second electronic device to generate a new CUDA kernel function based on the third function and the target function.

[0139] In one possible implementation, the receiving module 501 is further configured to receive a call request from the first electronic device, which is sent when the first electronic device is processing a target service, the target service being a service that requires calling the GPU resources of the second electronic device.

[0140] Module 503 is specifically used to invoke a new CUDA kernel function based on the invocation request, so that the first electronic device can remotely invoke the GPU resources in the second electronic device.

[0141] The GPU resource remote invocation device provided in this application embodiment can execute the technical solution of the GPU resource remote invocation method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the GPU resource remote invocation method. Please refer to the implementation principle and beneficial effects of the GPU resource remote invocation method. It will not be repeated here.

[0142] Figure 6 This is a schematic diagram of an electronic device structure provided in this application. Figure 6 As shown, the electronic device 600 may include at least one processor 601 and a memory 602.

[0143] The memory 602 is used to store programs. Specifically, the program may include program code, which includes computer operation instructions.

[0144] The memory 602 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0145] Processor 601 executes computer execution instructions stored in memory 602 to implement the GPU resource remote invocation method described in the foregoing method embodiments. Processor 601 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. Specifically, when implementing the GPU resource remote invocation method described in the foregoing method embodiments, the electronic device may be, for example, a terminal, a server, or other electronic device with processing capabilities.

[0146] Optionally, the electronic device 600 may also include a communication interface 603. In specific implementations, if the communication interface 603, memory 602, and processor 601 are implemented independently, they can be interconnected via a bus to complete communication. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc., but this does not imply that there is only one bus or one type of bus.

[0147] Optionally, in a specific implementation, if the communication interface 603, memory 602, and processor 601 are integrated on a single chip, then the communication interface 603, memory 602, and processor 601 can communicate through an internal interface.

[0148] This application also provides a computer-readable storage medium, which may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk. Specifically, the computer-readable storage medium stores program instructions, which are used in the methods described in the above embodiments.

[0149] This application also provides a program product including executable instructions stored in a readable storage medium. At least one processor of an electronic device can read the executable instructions from the readable storage medium, and the execution of the executable instructions by the at least one processor causes the electronic device to implement the GPU resource remote invocation methods provided in the various embodiments described above.

[0150] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A GPU resource remote invocation method, applied to a first electronic device, and having the steps of: include: The first electronic device reimplements the first and second functions in the CUDA kernel function of the parallel computing architecture, wherein the CUDA kernel function is the kernel function used by the first electronic device when remotely calling the GPU resources of the second electronic device; The first electronic device parses the first function to obtain the data block of the target function body. The first electronic device parses the second function to obtain the content of the kernel function parameters. The data block of the target function body and the content of the kernel function parameters are both contents pointed to by typeless pointer parameters. The first electronic device sends the data block of the target function body and the contents of the kernel function parameters to the second electronic device, so that the second electronic device executes a new CUDA kernel function and calls the GPU resources in the second electronic device. The new CUDA kernel function is generated by the second electronic device based on the data block of the target function body and the contents of the kernel function parameters.

2. The method according to claim 1, characterized in that, The first electronic device parses the first function to obtain a data block of the target function body, including: The first electronic device obtains the location of the first function from the declaration information of the first function, and the location of the first function is used to indicate the position of the first function in the header file of the CUDA kernel function; The first electronic device obtains the data block of the target function body and a first pointer to the data block from the header file of the CUDA kernel function, based on the location of the first function.

3. The method according to claim 2, characterized in that, The CUDA kernel function also includes a third function; The first electronic device sends the data block of the target function body to the second electronic device, including: The first electronic device obtains the kernel function name in the third function according to the first pointer, and the kernel function name corresponds to the data block of the target function body; The first electronic device sends the data block of the target function body and the name of the kernel function to the second electronic device.

4. The method according to claim 3, characterized in that, The first electronic device parses the second function to obtain the contents of the kernel function parameters, including: The first electronic device parses the contents of the header file of the second function in the CUDA kernel function to obtain the number of kernel function parameters; The first electronic device obtains the corresponding values ​​of the kernel function parameters from the header file of the CUDA kernel function based on the kernel function name and the number of kernel function parameters.

5. A method for remotely invoking GPU resources, applied to a second electronic device, characterized in that, The method includes: The second electronic device receives and stores the data block of the target function body and the contents of the kernel function parameters sent by the first electronic device; wherein, the data block of the target function body and the contents of the kernel function parameters are obtained by the first electronic device parsing the first function and the second function, and the data block of the target function body and the contents of the kernel function parameters are both contents pointed to by typeless pointer parameters; The second electronic device generates a new CUDA kernel function based on the data block of the target function body and the contents of the kernel function parameters; The second electronic device executes the new CUDA kernel function to enable the first electronic device to remotely invoke the GPU resources in the second electronic device.

6. The method according to claim 5, characterized in that, The content of the kernel function parameters includes the numerical values ​​corresponding to the kernel function parameters; The second electronic device receives and stores the data block of the target function body and the contents of the kernel function parameters sent by the first electronic device, including: The second electronic device receives the data block of the target function body and loads the data block of the target function body into the third function; The second electronic device receives the numerical values ​​corresponding to the kernel function parameters and the data blocks of other function bodies, and stores the numerical values ​​corresponding to the kernel function parameters and the data blocks of other function bodies in the target function. The data blocks of other function bodies include the function bodies in the CUDA kernel function other than the target function body.

7. The method according to claim 6, characterized in that, The second electronic device generates a new CUDA kernel function based on the data block of the target function body and the contents of the kernel function parameters, including: The second electronic device generates the new CUDA kernel function based on the third function and the target function.

8. The method according to claim 6, characterized in that, The second electronic device executes the new CUDA kernel function to enable the first electronic device to remotely access GPU resources in the second electronic device, including: The second electronic device receives a call request from the first electronic device, which is sent when the first electronic device is processing a target service, and the target service is a service that requires calling the GPU resources of the second electronic device; The second electronic device invokes the new CUDA kernel function according to the invocation request, so that the first electronic device can remotely invoke the GPU resources in the second electronic device.

9. A device for remotely accessing GPU resources, characterized in that, include: The processing module is used to reimplement the first and second functions in the CUDA kernel function of the parallel computing architecture, wherein the CUDA kernel function is the kernel function used by the first electronic device to remotely call the GPU resources of the second electronic device; The processing module is further configured to parse the first function to obtain a data block of the target function body, and the first electronic device to parse the second function to obtain the content of the kernel function parameters. The data block of the target function body and the content of the kernel function parameters are both contents pointed to by typeless pointer parameters. The sending module is used to send the data block of the target function body and the contents of the kernel function parameters to the second electronic device, so that the second electronic device executes a new CUDA kernel function and calls the GPU resources in the second electronic device. The new CUDA kernel function is generated by the second electronic device based on the data block of the target function body and the contents of the kernel function parameters.

10. A GPU resource remote access device, characterized in that, include: The receiving module is used to receive and store the contents of the data block of the target function body and the kernel function parameters sent by the first electronic device; wherein, the contents of the data block of the target function body and the kernel function parameters are obtained by the first electronic device parsing the first function and the second function, and the contents of the data block of the target function body and the kernel function parameters are both contents pointed to by typeless pointer parameters; The generation module is used to generate a new CUDA kernel function based on the data block of the target function body and the content of the kernel function parameters; The calling module is used to execute the new CUDA kernel function so that the first electronic device can remotely call the GPU resources in the second electronic device.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement a GPU resource remote invocation method as described in any one of claims 1-8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by the processor, implement the GPU resource remote invocation method according to any one of claims 1-8.

Images