Method, device, storage medium and program product for remotely invoking an acceleration device

Abstract

The application discloses a method, device, storage medium and program product for remotely calling an acceleration device, and relates to the technical field of artificial intelligence. The method sets acceleration devices and device codes on a server side, and a client sends deep learning framework version information to the server side so that the server side registers and loads the device codes accordingly. The client does not need to send the device codes and data size information of the device codes to the server side, so that the client does not need to parse the file header of the device codes to obtain the data size of the device codes, and the problem that the device codes cannot be correctly loaded to the acceleration device due to the inaccurate acquisition of the data size of the device codes can be avoided. The device codes are directly set on the server side, and in the acceleration device reuse scene, artificial intelligence tasks sharing the same acceleration device do not need to independently perform the operation of registering and loading the device codes once, so that the waste of display memory resources caused by the retention of a large number of repeated device code copies on the acceleration device can be avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a method, device, storage medium and program product for remotely calling an acceleration device. BACKGROUND

[0002] Remote calling of an acceleration device has become a main acceleration strategy adopted by an artificial intelligence platform to accelerate an artificial intelligence task by means of the acceleration device. Remote calling of the acceleration device requires that device code of a deep learning framework be loaded to the acceleration device first, and then the device code can be called to carry out accelerated operation. However, the related technology of remote calling of the acceleration device has the following technical problems: first, remote calling of the acceleration device requires that device code and data size information of the device code be sent to a device where the acceleration device is located, and it is difficult to obtain the data size of the device code, and if the data size of the device code cannot be accurately obtained, the device code cannot be correctly loaded to the acceleration device. Second, in an acceleration device reuse scenario, artificial intelligence tasks sharing the same acceleration device all need to independently perform a registration operation of loading device code, so that a large number of repeated device code copies are left on the acceleration device, resulting in waste of video memory resources. Therefore, how to solve the above technical problems has become a technical problem to be solved by the person skilled in the art. SUMMARY

[0003] The present application provides a method, device, storage medium and program product for remotely calling an acceleration device, to at least solve the problems in the related art that device code cannot be correctly loaded to the acceleration device due to the inability to accurately obtain the data size of the device code, and that a large number of repeated device code copies are left on the acceleration device, resulting in waste of video memory resources.

[0004] The present application provides a method for remotely calling an acceleration device, applied to a server, wherein the server is provided with an acceleration device and device code; the method for remotely calling the acceleration device comprises the following steps:

[0005] receiving a first request sent by a client; the first request carries deep learning framework version information;

[0006] registering device code corresponding to the deep learning framework version information;

[0007] loading the device code corresponding to the deep learning framework version information to the acceleration device according to the registered information;

[0008] establishing a mapping relationship between a server function stub and a name of a target device kernel function, so as to call the target device kernel function according to the server function stub; the target device kernel function is a device kernel function contained in the device code corresponding to the deep learning framework version information.

[0009] The application further provides a method for remotely calling an acceleration device, applied to a client, comprising:

[0010] obtaining deep learning framework version information;

[0011] sending a first request to a server, wherein the first request carries the deep learning framework version information, so that the server registers device code corresponding to the deep learning framework version information, loads the device code corresponding to the deep learning framework version information to an acceleration device according to the registered information, and establishes a mapping relationship between a server function stub and a name of a target device kernel function, so as to call the target device kernel function according to the server function stub; the target device kernel function is a device kernel function contained in the device code corresponding to the deep learning framework version information; the server is provided with an acceleration device and device code.

[0012] The application further provides a device for remotely calling an acceleration device, applied to a server, wherein the server is provided with an acceleration device and device code; the device for remotely calling an acceleration device comprises:

[0013] a first receiving module, configured to receive a first request sent by a client; the first request carries deep learning framework version information;

[0014] a registering module, configured to register device code corresponding to the deep learning framework version information;

[0015] a loading module, configured to load the device code corresponding to the deep learning framework version information to an acceleration device according to the registered information;

[0016] a first establishing module, configured to establish a mapping relationship between a server function stub and a name of a target device kernel function, so as to call the target device kernel function according to the server function stub; the target device kernel function is a device kernel function contained in the device code corresponding to the deep learning framework version information.

[0017] The application further provides a device for remotely calling an acceleration device, applied to a client, comprising:

[0018] a obtaining module, configured to obtain deep learning framework version information;

[0019] The sending module is configured to send a first request to a server, the first request carrying deep learning framework version information, so that the server registers device code corresponding to the deep learning framework version information, and loads the device code corresponding to the deep learning framework version information to an acceleration device according to the registered information; and establish a mapping relationship between a server function stub and a name of a target device kernel function, so as to call the target device kernel function according to the server function stub; the target device kernel function is a device kernel function contained in the device code corresponding to the deep learning framework version information; and the server is provided with an acceleration device and device code.

[0020] The application further provides an electronic device, comprising a memory for storing a computer program and a processor for executing the computer program to implement the steps of the method for remotely calling an acceleration device.

[0021] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the steps of the method for remotely calling an acceleration device.

[0022] The application further provides a computer program product, which comprises a computer program, and the computer program is executed by a processor to implement the steps of the method for remotely calling an acceleration device.

[0023] The application has the beneficial effects that: the device code is set on the server, the server is further provided with an acceleration device, the client remotely calls the acceleration device, the client does not need to send the device code and the data size information of the device code to the server, so that the client does not need to parse the file header of the device code to obtain the data size of the device code, and the problem that the device code cannot be correctly loaded to the acceleration device due to inaccurate acquisition of the data size of the device code can be avoided. Moreover, since the device code is directly set on the server, the client does not need to send the device code to the server for registration and loading, so that in the acceleration device reuse scene, the artificial intelligence task shared by the same acceleration device does not need to independently perform the operation of registering and loading the device code once, so that the waste of video memory resources caused by leaving a large number of repeated device code copies on the acceleration device can be avoided. Moreover, the application avoids the process of transmitting the device code from the client to the server, and can reduce the cold start time. BRIEF DESCRIPTION OF DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0025] Figure 1 A flow chart of a method for remotely calling an acceleration device is provided for an embodiment of the present application.

[0026] Figure 2 A schematic diagram of a server and a client is provided for an embodiment of the present application.

[0027] Figure 3 A schematic diagram of establishing a mapping relationship is provided for an embodiment of the present application.

[0028] Figure 4 An interaction schematic diagram of a server and a client is provided for an embodiment of the present application.

[0029] Figure 5 Another schematic diagram of establishing a mapping relationship is provided for an embodiment of the present application.

[0030] Figure 6 A flow chart of another method for remotely calling an acceleration device is provided for an embodiment of the present application.

[0031] Figure 7 A schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, any other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0033] It should be noted that, in the description of the present application, the terms “comprise”, “contain” or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. The terms “first”, “second” and the like in the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0034] In order for those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0035] Embodiments of the present application provide a method for remotely calling an acceleration card, and the method is described in detail in combination with the execution flow of the method.

[0036] Reference Figure 1As shown, the embodiment of the application provides a method for remotely calling an acceleration device, which is applied to a server, and the server is provided with an acceleration device and device code.

[0037] S101: receiving a first request sent by a client; the first request carries deep learning framework version information;

[0038] S102: registering device code corresponding to the deep learning framework version information;

[0039] S103: loading the device code corresponding to the deep learning framework version information to the acceleration device according to the registered information;

[0040] S104: establishing a mapping relationship between a server function stub and a target device kernel function name, so as to call the target device kernel function according to the server function stub; the target device kernel function is a device kernel function contained in the device code corresponding to the deep learning framework version information.

[0041] Reference Figure 2 As shown, the client runs an artificial intelligence task and is provided with a hijacking library. The hijacking library is used to identify and intercept API calling requests of the artificial intelligence task related to the acceleration device. The client is not provided with the acceleration device and the device code, and the server is provided with the acceleration device and the device code. The device code is generated by compiling a deep learning framework (such as Pytorch, TensorFlow, etc.) used by the artificial intelligence task and is used to execute operation tasks on the acceleration device. The version of the device code corresponds to the version of the deep learning framework. The device code can be obtained from a deep learning framework software distribution package, or the device code can be directly generated by compiling the deep learning framework. The device code is adapted to the computing architecture of the acceleration device.

[0042] In some embodiments, the server is provided with multiple versions of device code. In this way, it can be ensured that the server can adapt to different deep learning framework version requirements.

[0043] When the client runs the artificial intelligence task to generate the use demand of the acceleration device, the artificial intelligence task triggers the API (Application Programming Interface, application programming interface) calling related to the acceleration device. The hijacking library of the client identifies and intercepts the calling request of the artificial intelligence task to the API related to the acceleration device, and forwards it to the server for processing.

[0044] When the hijacking library of the client intercepts the request of the artificial intelligence task to call the xxRegisterFatBinary function, the version information of the deep learning framework in the current environment is obtained, and a first request is sent to the server; the first request carries the version information of the deep learning framework. The xx in the xxRegisterFatBinary function represents the specific name prefix of the API by the manufacturer of the acceleration device.

[0045] The server runs a server main program, receives the first request of the client, and determines the corresponding device code according to the version information of the deep learning framework carried in the first request. The device code corresponding to the version information of the deep learning framework is registered to the acceleration device software runtime, that is, the acceleration device software runtime. Registering the device code corresponding to the version information of the deep learning framework to the acceleration device software runtime includes recording the address and other information of the device code corresponding to the version information of the deep learning framework. After the acceleration device software runtime completes context initialization, the acceleration device software runtime loads the device code corresponding to the version information of the deep learning framework to the acceleration device according to the registered information.

[0046] The device code is set on the server. In order to accurately call the device code, the corresponding server is provided with a host function stub. The host function stub of the server is called a server function stub. After receiving the first request, the server establishes a mapping relationship between the server function stub and the name of the target device kernel function, which is a device kernel function included in the device code corresponding to the version information of the deep learning framework. The server function stub and the device kernel function are one-to-one mapped. The client can specify the server function stub to enable the server to accurately call the corresponding device kernel function.

[0047] In some embodiments, receiving the first request sent by the client includes:

[0048] Receiving the first request sent by the client after the client's hijacking library intercepts the request of the artificial intelligence task to call the registration fat binary function for the first time.

[0049] When the hijacking library of the client intercepts the request of the artificial intelligence task to call the xxRegisterFatBinary function for the first time, the version information of the deep learning framework in the current environment is obtained, and a first request is sent to the server. Subsequently, even if the hijacking library intercepts the request of the artificial intelligence task to call the xxRegisterFatBinary function, the first request does not need to be sent again, and the server does not need to register and load the device code corresponding to the same version information of the deep learning framework again, thereby avoiding redundant operations.

[0050] When the version of the deep learning framework of the client changes, the client again intercepts the calling request of the artificial intelligence task to the xxRegisterFatBinary function, and sends the first request again to make the server register and load the device code corresponding to the changed deep learning framework version information, so as to ensure that the device kernel function can be correctly called subsequently.

[0051] In some embodiments, the registering the device code corresponding to the deep learning framework version information comprises:

[0052] The xxRegisterFatBinary function is called to register the device code corresponding to the deep learning framework version information.

[0053] After receiving the first request, the server calls the xxRegisterFatBinary function to register the device code corresponding to the deep learning framework version information to the acceleration device software runtime.

[0054] In some embodiments, the establishing the mapping relationship between the server function stub and the name of the target device kernel function comprises:

[0055] The xxRegisterFunction function is called to establish the mapping relationship between the server function stub and the name of the target device kernel function.

[0056] For each device kernel function included in the device code, the server calls the xxRegisterFunction function to establish the mapping relationship between the server function stub and the name of the device kernel function.

[0057] In some embodiments, the method further comprises:

[0058] The second request sent by the client is received; the second request carries the name of the device kernel function and the client function stub;

[0059] The mapping relationship between the client function stub and the first target server function stub is established; the first target server function stub is the server function stub corresponding to the name of the device kernel function carried by the second request.

[0060] In the embodiment, the client sets a host function stub, and the host function stub of the client is referred to as a client host function stub. When the hijacking library of the client intercepts a request of calling the xxRegisterFunction function by the artificial intelligence task, the second request is sent to the server. The second request carries the name of the device kernel function and the client function stub. The server receives the second request, and establishes a mapping relationship between the client function stub and the first target server function stub. The first target server function stub is the server function stub corresponding to the name of the device kernel function carried by the second request. The client sends the client function stub to the server, so as to specify the server to call the corresponding device kernel function.

[0061] Reference Figure 3 As shown, Figure 3 In process one, the mapping relationship between the name of the device kernel function and the server function stub is established. In process two, the mapping relationship between the client function stub and the server function stub is established. The server calls the xxRegisterFunction function, and the calling parameter includes the name of the device kernel function and the server function stub. The mapping relationship between the server function stub and the name of the device kernel function is established and recorded. In process two, the client remotely calls the xxRegisterFunction function, and the calling parameter includes the name of the device kernel function and the client function stub. When the server receives the request of remotely calling the xxRegisterFunction function by the client, that is, the second request, the corresponding server function stub is determined according to the mapping relationship established in process one according to the name of the device kernel function, and then the mapping relationship between the client function stub and the server function stub is established.

[0062] The mapping relationship between the client function stub and the server function stub is established, the client sends the request carrying the client function stub to the server, so as to specify the server to call the corresponding device kernel function, and the correct start of the device kernel function can be ensured.

[0063] In some embodiments, further comprising:

[0064] The third request sent by the client is received, and the third request carries the client function stub;

[0065] According to the mapping relationship between the client function stub and the server function stub, the second target server function stub is determined, and the device kernel function is called according to the second target server function stub. The second target server function stub is the server function stub corresponding to the client function stub carried by the third request.

[0066] The embodiment aims to realize remote calling of a device kernel function. After the hijacking library of the client intercepts a calling request of an xxLaunchKernel function by an artificial intelligence task, the client initiates remote calling with a client function stub as a parameter, and sends a third request carrying the client function stub. The server receives the third request, determines a server function stub corresponding to the client function stub according to a mapping relationship between the client function stub and the server function stub, and uses the server function stub as a parameter to call an underlying xxLaunchKernel function, i.e., a device kernel function, to realize starting of the device kernel function.

[0067] The server determines a corresponding server function stub according to the request carrying the client function stub sent by the client according to the mapping relationship, and then correctly calls the device kernel function with the server function stub as a parameter.

[0068] In some embodiments, further comprising:

[0069] After loading the device code, returning a flag representing successful loading of the device code to the client;

[0070] After establishing the mapping relationship between the client function stub and the server function stub, returning a flag representing successful establishment of the mapping relationship between the client function stub and the server function stub to the client;

[0071] After calling the device kernel function, returning a flag representing successful starting of the device kernel function to the client.

[0072] Reference Figure 4 As shown, the server completes starting and enters a ready state. When the artificial intelligence task of the client calls an acceleration device related API, the hijacking library intercepts. The first calling request of the API by the artificial intelligence task is intercepted, triggering initialization of the hijacking library. In the initialization process of the hijacking library, the hijacking library establishes a connection between the client and the server to provide a communication basis for subsequent remote calling. After initialization, the hijacking library intercepts the first calling request of the xxRegisterFatBinary function by the client, obtains deep learning framework version information in the current environment, and initiates a remote calling request to the server, i.e., sends a first request carrying the deep learning framework version information to the server.

[0073] The server loads the device code of the corresponding version through dlopen according to the deep learning framework version information of the client, which automatically triggers the registration and loading process of the device code, and establishes a mapping relationship between the name of the device kernel function and the server function stub. After the server registers and loads the device code, the server returns a success flag to the client.

[0074] When the client intercepts a request to call the xxRegisterFunction function, the request to call the xxRegisterFunction function is forwarded to the server, i.e., a second request carrying the name of the client function stub and the device kernel function is sent to the server. The server establishes a mapping between the client function stub and the server function stub accordingly, and returns a success flag to the client after establishing the mapping between the client function stub and the server function stub. The server no longer repeatedly executes the underlying xxRegisterFunction function.

[0075] The client continues to perform the preparation work of the device kernel function and forwards according to the conventional remote calling mode. When the client intercepts a request to call the xxLaunchKernel function, i.e., the device kernel function, the client initiates a remote call with the client function stub as a parameter, and sends a third request carrying the client function stub to the server. The server maps the client function stub to the server function stub according to the mapping relationship between the client function stub and the server function stub that has been established, and uses the server function stub as a parameter to call the underlying device kernel function, thereby starting the device kernel function. After calling the device kernel function, a start success flag is returned to the client.

[0076] In some embodiments, before registering the device code corresponding to the deep learning framework version information, the method further includes:

[0077] determining whether the device code corresponding to the deep learning framework version information has been registered;

[0078] if the device code corresponding to the deep learning framework version information has been registered, the device code corresponding to the deep learning framework version information is not registered again;

[0079] if the device code corresponding to the deep learning framework version information has not been registered, the device code corresponding to the deep learning framework version information is registered.

[0080] In this embodiment, before registering the device code, the server first determines whether the device code corresponding to the deep learning framework version information carried in the first request sent by the client has already been registered. If the device code corresponding to the deep learning framework version information has already been registered, it will not be registered again. If the device code corresponding to the deep learning framework version information has not been registered, it will be registered. This avoids duplicate registration of the same device code. In the scenario of accelerated device reuse, i.e., multiple artificial intelligence tasks sharing the same accelerated device, each artificial intelligence task uses the same deep learning framework, and the corresponding device code is the same. Clients running each artificial intelligence task connect to the same server. When the server executes the operation of loading the corresponding version of the device code using dlopen according to the deep learning framework version information, the device code for the same deep learning framework version is loaded only once, avoiding duplicate loading. Since there are multiple clients, each client has its own client function stub for the same device function. When the client remotely calls xxRegisterFunction, the server establishes a mapping relationship between the client function stub of each client and the same server function stub, thereby achieving efficient reuse of device code.

[0081] Before registering a device code, first check if the device code has already been registered. This way, the device code for the same deep learning framework version is only registered and loaded once, avoiding duplicate registration and loading and improving server efficiency.

[0082] For example, refer to Figure 5 As shown, the client function stub of client one is called the client one function stub, and the client function stub of client two is called the client two function stub. Client one and client two use the same deep learning framework version; therefore, the client one function stub and the client two function stub are mapped to the same server function stub. When client one sends a first request to the server, the server determines whether the device code corresponding to the deep learning framework version information carried in the first request has already been registered. If it has already been registered, it will not be registered again. If it has not been registered (for example, client two sent a first request beforehand, causing the server to register the corresponding device code), then the server registers the device code corresponding to the deep learning framework version information. When client two sends a first request to the server, the server determines whether the device code corresponding to the deep learning framework version information carried in the first request has already been registered. If it has already been registered (client one sent a first request beforehand, causing the server to register the corresponding device code), it will not be registered again. If it has not been registered, then the server registers the device code corresponding to the deep learning framework version information.

[0083] In summary, the application sets the device code on the server side, and the server side is also provided with an acceleration device, the client remotely calls the acceleration device, the client does not need to send the device code and the data size information of the device code to the server side, so that the client does not need to parse the file header of the device code to obtain the data size of the device code, and the problem that the device code cannot be correctly loaded to the acceleration device due to inaccurate acquisition of the data size of the device code can be avoided. Moreover, since the device code is directly set on the server side, the client does not need to send the device code to the server side for registration and loading, so in the acceleration device reuse scene, the artificial intelligence task sharing the same acceleration device does not need to independently perform the operation of registering and loading the device code once, so that the waste of video memory resources caused by retaining a large number of repeated device code copies on the acceleration device is avoided. Moreover, the application avoids the process of transmitting the device code from the client to the server, and can reduce the cold start time.

[0084] Reference Figure 6 As shown in the figure, the application embodiment provides a method for remotely calling an acceleration device, applied to a client, comprising:

[0085] S201: Obtain deep learning framework version information;

[0086] S202: Send a first request to the server, the first request carrying the deep learning framework version information, so that the server registers the device code corresponding to the deep learning framework version information, loads the device code corresponding to the deep learning framework version information to the acceleration device according to the registered information, establishes a mapping relationship between the server function stub and the name of the target device kernel function, so as to call the target device kernel function according to the server function stub; the target device kernel function is a device kernel function contained in the device code corresponding to the deep learning framework version information; the server is provided with an acceleration device and device code.

[0087] The client runs an artificial intelligence task and is provided with a hijacking library. The hijacking library is used to identify and intercept API call requests related to the acceleration device. The client is not provided with an acceleration device and device code, and the server is provided with an acceleration device and device code. The device code is generated by compiling the deep learning framework used by the artificial intelligence task, and is used to perform operation tasks on the acceleration device. The version of the device code corresponds to the version of the deep learning framework. The device code can be obtained from the deep learning framework software distribution package, or the device code can be directly generated by compiling the deep learning framework. The device code adapts to the computing architecture of the acceleration device. The server can be provided with multiple versions of device code, so that the server can adapt to different deep learning framework version requirements.

[0088] When the client runs an artificial intelligence task to generate a use demand of an acceleration device, the artificial intelligence task triggers an API (Application Programming Interface) call related to the acceleration device. The hijacking library of the client identifies and intercepts the call request of the artificial intelligence task to the API related to the acceleration device, and forwards the call request to the server for processing.

[0089] When the hijacking library of the client intercepts the call request of the artificial intelligence task to the xxRegisterFatBinary function, the deep learning framework version information in the current environment is obtained, and a first request is sent to the server. The first request carries the deep learning framework version information. The server runs a server main program, receives the first request of the client, and determines the corresponding device code according to the deep learning framework version information carried in the first request. The deep learning framework version information corresponding device code is registered to the acceleration device software runtime, that is, the acceleration device software runtime. Registering the deep learning framework version information corresponding device code to the acceleration device software runtime includes recording the address and size of the deep learning framework version information corresponding device code. After the acceleration device software runtime completes context initialization, the acceleration device software runtime loads the deep learning framework version information corresponding device code to the acceleration device according to the registered information. The device code is set in the server, and in order to accurately call the device code, the corresponding server is provided with a host function stub. The server function stub is called a server function stub. After receiving the first request, the server establishes a mapping relationship between the server function stub and the name of the target device kernel function, which is a device kernel function included in the deep learning framework version information corresponding device code. The server function stub and the device kernel function are one-to-one mapped. The client can specify the server function stub to enable the server to accurately call the corresponding device kernel function.

[0090] In some embodiments, sending the first request to the server includes:

[0091] The first request sent to the server after hijacking the request of the artificial intelligence task to call the registration fat binary function for the first time.

[0092] When the hijacking library of the client intercepts the call request of the artificial intelligence task to the xxRegisterFatBinary function for the first time, the deep learning framework version information in the current environment is obtained, and a first request is sent to the server. Even if the hijacking library intercepts the call request of the artificial intelligence task to the xxRegisterFatBinary function again, the first request does not need to be sent again, and the server does not need to register and load the device code corresponding to the same deep learning framework version information again, thereby avoiding redundant operations.

[0093] In some embodiments, further comprising:

[0094] sending a second request to the server, the second request carrying a name of the device kernel function and the client function stub, so that the server establishes a mapping relationship between the client function stub and a first target server function stub; the first target server function stub is a server function stub corresponding to the name of the device kernel function carried by the second request.

[0095] The client sets a host function stub, and the host function stub of the client is referred to as a client host function stub. When the hijacking library of the client intercepts a request of calling the xxRegisterFunction function of the artificial intelligence task, a second request is sent to the server. The second request carries the name of the device kernel function and the client function stub. The server receives the second request and establishes a mapping relationship between the client function stub and a first target server function stub; the first target server function stub is a server function stub corresponding to the name of the device kernel function carried by the second request. The client sends the client function stub to the server to specify the server to call the corresponding device kernel function.

[0096] In some embodiments, further comprising:

[0097] sending a third request to the server, the third request carrying the client function stub, so that the server determines a second target server function stub according to the mapping relationship between the client function stub and the server function stub, and calls the device kernel function according to the second target server function stub; the second target server function stub is a server function stub corresponding to the client function stub carried by the third request.

[0098] When the hijacking library of the client intercepts a request of calling the xxLaunchKernel function of the artificial intelligence task, the client initiates a remote call with the client function stub as a parameter, and sends a third request carrying the client function stub. The server receives the third request, determines the server function stub corresponding to the client function stub carried by the third request according to the mapping relationship between the client function stub and the server function stub, and uses the server function stub as a parameter to call the underlying xxLaunchKernel function, i.e., the device kernel function, to implement the start of the device kernel function.

[0099] In some embodiments, further comprising:

[0100] receiving a flag indicating that the device code is successfully loaded from the server after the server loads the device code;

[0101] The flag returned after the receiving side calls the device kernel function, indicating that the device kernel function is successfully started.

[0102] The flag returned after the receiving side calls the device kernel function, indicating that the device kernel function is successfully started.

[0103] The receiving side completes the start and enters the ready state. When the client artificial intelligence task calls the acceleration device related API, the hijacking library intercepts. The first request of the artificial intelligence task to call the API is intercepted, triggering the initialization of the hijacking library. In the initialization process of the hijacking library, the hijacking library establishes a connection between the client and the server to provide a communication basis for subsequent remote calls. After initialization, the hijacking library intercepts the first request of the client to call the xxRegisterFatBinary function, obtains the deep learning framework version information in the current environment, and initiates a remote call request to the server, i.e., sends the first request carrying the deep learning framework version information to the server.

[0104] The server loads the device code of the corresponding version according to the deep learning framework version information of the client through dlopen, which automatically triggers the registration and loading process of the device code, and establishes the mapping relationship between the device kernel function name and the server function stub. After the server registers and loads the device code, the server returns a success flag to the client.

[0105] When the client intercepts the request of the artificial intelligence task to call the xxRegisterFunction function, the request to call the xxRegisterFunction function is forwarded to the server, i.e., the second request carrying the client function stub and the device kernel function name is sent to the server. The server establishes the mapping between the client function stub and the server function stub accordingly, and returns a success flag to the client after establishing the mapping relationship between the client function stub and the server function stub. The server no longer repeatedly executes the underlying xxRegisterFunction function.

[0106] The client continues to perform the preparation work for starting the device kernel function, and forwards according to the conventional remote call mode. When the client intercepts the call request of the xxLaunchKernel function, i.e., the device kernel function, the client function stub is used as a parameter to initiate a remote call, and the third request carrying the client function stub is sent to the server. The server maps the client function stub to the server function stub according to the mapping relationship between the client function stub and the server function stub, and uses the server function stub as a parameter to call the underlying device kernel function, thereby starting the device kernel function. After calling the device kernel function, a success flag is returned to the client.

[0107] In summary, this invention sets the device code on the server side, which also houses an acceleration device. The client remotely invokes the acceleration device without needing to send the device code and its size information to the server. This eliminates the need for the client to parse the device code header to obtain its size, preventing issues caused by inaccurate size information leading to incorrect loading of the device code onto the acceleration device. Furthermore, since the device code is directly on the server, the client doesn't need to send the device code to the server for registration and loading. Therefore, in acceleration device reuse scenarios, AI tasks sharing the same acceleration device don't need to independently perform a registration and loading operation, avoiding the waste of GPU memory resources caused by numerous duplicate device code copies on the acceleration device. Moreover, this invention avoids the process of transmitting device code from the client to the server, reducing cold start time.

[0108] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0109] Embodiments of the present invention also provide a device for remotely invoking an acceleration device. This device is applied to a server, which has an acceleration device and device code configured on it. The device includes:

[0110] The first receiving module is used to receive a first request sent by the client; the first request carries deep learning framework version information.

[0111] The registration module is used to register the device code corresponding to the version information of the deep learning framework.

[0112] The loading module is used to load the device code corresponding to the deep learning framework version information into the acceleration device based on the registered information.

[0113] The first establishment module is used to establish a mapping relationship between the names of server-side function stubs and target device kernel functions, so as to call the target device kernel function according to the server-side function stubs; the target device kernel function is the device kernel function contained in the device code corresponding to the deep learning framework version information.

[0114] Based on the above embodiments, as a specific implementation method, it further includes:

[0115] The second receiving module is used to receive a second request sent by the client; the second request carries the name of the device kernel function and the client function stub.

[0116] a second establishing module, configured to establish a mapping relationship between the client function stub and a first target server function stub; the first target server function stub is a server function stub corresponding to a name of the device kernel function carried by the second request.

[0117] In the above embodiment, as a specific implementation, the method further includes:

[0118] a third receiving module, configured to receive a third request sent by the client, the third request carrying the client function stub;

[0119] a calling module, configured to determine a second target server function stub according to the mapping relationship between the client function stub and the server function stub, and call the device kernel function according to the second target server function stub; the second target server function stub is a server function stub corresponding to the client function stub carried by the third request.

[0120] In the above embodiment, as a specific implementation, the method further includes:

[0121] a judging module, configured to judge whether the device code corresponding to the deep learning framework version information has been registered; if the device code corresponding to the deep learning framework version information has been registered, the device code corresponding to the deep learning framework version information is not registered again; if the device code corresponding to the deep learning framework version information has not been registered, the registering module registers the device code corresponding to the deep learning framework version information.

[0122] In the above embodiment, as a specific implementation, the registering module is configured to:

[0123] call the registration fat binary function to register the device code corresponding to the deep learning framework version information.

[0124] In the above embodiment, as a specific implementation, the first establishing module is configured to:

[0125] call the registration function function to establish the mapping relationship between the server function stub and the name of the target device kernel function.

[0126] In the above embodiment, as a specific implementation, the first receiving module is configured to:

[0127] receive the first request sent by the client after the client hijacks the request of the artificial intelligence task to call the registration fat binary function for the first time.

[0128] In the above embodiment, as a specific implementation, the method further includes:

[0129] The first returning module is configured to return a flag representing successful loading of the device code to the client after the device code is loaded.

[0130] The second returning module is configured to return a flag representing successful establishment of the mapping relationship between the client function stub and the server function stub to the client after the mapping relationship between the client function stub and the server function stub is established.

[0131] The third returning module is configured to return a flag representing successful starting of the device kernel function to the client after the device kernel function is called.

[0132] As a specific implementation, the server is provided with a plurality of versions of device code.

[0133] Embodiments of the present application also provide a device for remotely calling an acceleration device, which is applied to a client and comprises:

[0134] The obtaining module is configured to obtain deep learning framework version information.

[0135] The sending module is configured to send a first request to the server, the first request carrying the deep learning framework version information, so that the server registers device code corresponding to the deep learning framework version information, loads the device code corresponding to the deep learning framework version information to the acceleration device according to the registered information, establishes a mapping relationship between a server function stub and the name of a target device kernel function, so as to call the target device kernel function according to the server function stub; the target device kernel function is a device kernel function contained in the device code corresponding to the deep learning framework version information; and the server is provided with the acceleration device and the device code.

[0136] As a specific implementation, the device further comprises:

[0137] The second sending module is configured to send a second request to the server, the second request carrying the name of the device kernel function and a client function stub, so that the server establishes a mapping relationship between the client function stub and a first target server function stub; the first target server function stub is a server function stub corresponding to the name of the device kernel function carried by the second request.

[0138] As a specific implementation, the device further comprises:

[0139] The third sending module is configured to send a third request to the server, the third request carrying a client function stub, so that the server determines a second target server function stub according to a mapping relationship between the client function stub and the server function stub, and calls a device kernel function according to the second target server function stub; the second target server function stub is a server function stub corresponding to the client function stub carried by the third request.

[0140] The description of the features in the embodiments of the device for remotely calling an acceleration device can refer to the related description of the embodiments of the method, which will not be repeated here.

[0141] Embodiments of the present application also provide an electronic device, as shown in the accompanying drawings, Figure 7 The electronic device includes a memory 1 and a processor 2, the memory 1 stores a computer program, and the processor 2 is configured to run the computer program to perform the steps in any of the above method embodiments for remotely calling an acceleration device.

[0142] Embodiments of the present application also provide a computer readable storage medium, which stores a computer program, wherein the computer program is configured to perform the steps in any of the above method embodiments for remotely calling an acceleration device when running.

[0143] In an exemplary embodiment, the above computer readable storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store computer programs.

[0144] Embodiments of the present application also provide a computer program product, which includes a computer program, and the computer program is executed by a processor to implement the steps in any of the above method embodiments for remotely calling an acceleration device.

[0145] Embodiments of the present application also provide another computer program product, which includes a non-volatile computer readable storage medium, and the non-volatile computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps in any of the above method embodiments for remotely calling an acceleration device.

[0146] Those skilled in the art will further realize that the mere concepts, teachings, and embodiments described herein are merely meant to provide an enabling description of the claimed invention and are not intended to limit the scope of the claimed invention to these embodiments. Therefore, embodiments described herein are not meant to be limiting, but merely representative. Further, the routines executed to implement the embodiments of the invention, individually or collectively, need not be limited to any specific combination of hardware and software. Various embodiments can also be implemented using more conventional components, as well as hardware and software that are not expressly identified herein. To clearly illustrate this interchangeability of hardware and software, various embodiments have been depicted in the form of specific examples. Those skilled in the art will appreciate that the concepts and teachings described herein are applicable to a variety of other platforms and applications. Accordingly, the present invention is not intended to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the claims, the principles and the practices described herein.

[0147] The method, device, storage medium and program product for remote call acceleration device provided by the present application are described in detail above. The principles and implementation manners of the present application are described by applying specific examples in the present application. The above description of the embodiments is only applicable to help understand the method of the present application and its core idea. It should be pointed out that, for those skilled in the art, without departing from the principles of the present application, some improvements and modifications can be made to the present application. These improvements and modifications also fall within the protection scope of the present application.

Claims

1. A method for remotely invoking an acceleration device, characterized in that, Applied to the server side, the server side is equipped with acceleration devices and device code; methods for remotely invoking the acceleration devices include: Receive the first request sent by the client; the first request carries the deep learning framework version information; Register the device code corresponding to the version information of the deep learning framework; Based on the registration information, the device code corresponding to the deep learning framework version information is loaded into the acceleration device; Establish a mapping relationship between the names of server-side function stubs and target device kernel functions, so as to call the target device kernel function according to the server-side function stubs; the target device kernel function is the device kernel function contained in the device code corresponding to the deep learning framework version information.

2. The method for remotely invoking an acceleration device according to claim 1, characterized in that, Also includes: Receive a second request sent by the client; the second request carries the name of the device kernel function and the client function stub. Establish a mapping relationship between the client function stub and the first target server function stub; the first target server function stub is a server function stub that maps to the name of the device kernel function carried in the second request.

3. The method for remotely invoking an acceleration device according to claim 2, characterized in that, Also includes: Receive a third request sent by the client, the third request carrying a client function stub; Based on the mapping relationship between client function stubs and server function stubs, a second target server function stub is determined, and the device core function is called based on the second target server function stub; the second target server function stub is the server function stub mapped to the client function stub carried in the third request.

4. The method for remotely invoking an acceleration device according to claim 1, characterized in that, Before registering the device code corresponding to the deep learning framework version information, the following is also included: Determine whether the device code corresponding to the deep learning framework version information has been registered; If the device code corresponding to the deep learning framework version information has already been registered, then the device code corresponding to the deep learning framework version information will not be registered again. If the device code corresponding to the deep learning framework version information is not registered, then register the device code corresponding to the deep learning framework version information.

5. The method for remotely invoking an acceleration device according to claim 1, characterized in that, The device code corresponding to the registered deep learning framework version information includes: The registration fat binary function is called to register the device code corresponding to the version information of the deep learning framework.

6. The method for remotely invoking an acceleration device according to claim 1, characterized in that, Establishing a mapping between the names of server-side function stubs and target device kernel functions includes: The registration function is called to establish a mapping relationship between the names of the server-side function stubs and the target device kernel functions.

7. The method for remotely invoking an acceleration device according to claim 1, characterized in that, The first request sent by the receiving client includes: The client sends its first request after it has hijacked the request to register the fat binary function for the first time in an AI task.

8. The method for remotely invoking an acceleration device according to claim 3, characterized in that, Also includes: After loading the device code, a flag indicating successful loading of the device code is returned to the client; After establishing the mapping relationship between client function stubs and server function stubs, a flag indicating that the mapping relationship between client function stubs and server function stubs has been successfully established is returned to the client. After invoking the device kernel function, a flag indicating successful startup of the device kernel function is returned to the client.

9. The method for remotely invoking an acceleration device according to claim 1, characterized in that, The server has multiple versions of device codes.

10. A method for remotely invoking an acceleration device, characterized in that, Applied to the client side, including: Get deep learning framework version information; A first request is sent to the server, the first request carrying the deep learning framework version information, so that the server registers the device code corresponding to the deep learning framework version information. Based on the registered information, the device code corresponding to the deep learning framework version information is loaded into the acceleration device. A mapping relationship is established between the names of server-side function stubs and target device kernel functions, so that the target device kernel function can be called according to the server-side function stubs. The target device kernel function is the device kernel function contained in the device code corresponding to the deep learning framework version information. The server is configured with acceleration devices and device code.

11. The method for remotely invoking an acceleration device according to claim 10, characterized in that, Also includes: A second request is sent to the server, the second request carrying the name of the device kernel function and the client function stub, so that the server establishes a mapping relationship between the client function stub and the first target server function stub; the first target server function stub is the server function stub that maps to the name of the device kernel function carried in the second request.

12. The method for remotely invoking an acceleration device according to claim 11, characterized in that, Also includes: A third request is sent to the server, the third request carrying a client function stub, so that the server determines a second target server function stub based on the mapping relationship between the client function stub and the server function stub, and calls the device kernel function based on the second target server function stub; the second target server function stub is the server function stub mapped to the client function stub carried in the third request.

13. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the method for remotely invoking an acceleration device as claimed in any one of claims 1 to 9, or to implement the steps of the method for remotely invoking an acceleration device as claimed in any one of claims 10 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the method for remotely invoking an acceleration device as claimed in any one of claims 1 to 9 or the steps of the method for remotely invoking an acceleration device as claimed in any one of claims 10 to 12.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method for remotely invoking an acceleration device as described in any one of claims 1 to 9, or the steps of the method for remotely invoking an acceleration device as described in any one of claims 10 to 12.

Images