Implementation methods, devices, electronic equipment, and storage media of the OpenCL CUB library

Abstract

The present disclosure discloses an implementation method and device of OpenCL CUB library, electronic equipment and storage medium, relates to the technical field of artificial intelligence, designs CUB library related algorithms based on the OpenCL programming standard, and redesigns the encapsulation, calling and compiling of the CUB library, realizes the CUB library based on the OpenCL standard, and can provide efficient support for the OpenCL version deep learning framework and OpenCL application development.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to an implementation method and apparatus, electronic device and storage medium for the OpenCL CUB library. Background Technology

[0002] Currently, deep learning frameworks mainly utilize various high-performance computing libraries to further optimize computation speed.

[0003] The computation process of deep learning frameworks typically involves basic computational functions such as reduction, scanning, and deduplication. By leveraging the parallel acceleration capabilities of heterogeneous devices, the efficiency of these basic computational functions can be significantly optimized. NVIDIA's high-performance computing library provides a large number of efficient basic computations, which significantly accelerates the implementation of operators in deep learning frameworks.

[0004] The aforementioned high-performance computing library from NVIDIA can only be used on proprietary acceleration devices from manufacturers such as NVIDIA, which prevents many OpenCL-compatible acceleration devices from playing a greater role in the field of deep learning. Summary of the Invention

[0005] This disclosure provides a method, apparatus, electronic device, and storage medium for implementing the OpenCL CUB library. Its main purpose is to enable the CUB library to provide efficient support for the development of OpenCL versions of deep learning frameworks and OpenCL applications.

[0006] According to a first aspect of this disclosure, an implementation method for the OpenCL CUB library is provided, comprising:

[0007] Based on the OpenCL programming standard, design related algorithms of the OpenCL CUB library and implement the corresponding OpenCL kernel functions in the OpenCL CUB library;

[0008] The OpenCL CUB library is compiled using a compilation module to generate header files and dynamic link libraries. The header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithmic encapsulations for calling these encapsulated APIs.

[0009] The OpenCL API encapsulation and cache management module encapsulates the platform API and runtime API in the OpenCL API and manages the platform API and runtime API on the host side.

[0010] The interface module provides a first calling interface to the host side; the first calling interface is used to schedule the kernel function in the kernel function algorithm module for calculation in the host side code;

[0011] The kernel function algorithm module provides a second calling interface oriented towards the device side; the second calling interface is used to call the kernel function in the kernel function algorithm module from other device side code.

[0012] Optionally, the compilation of OpenCL CUB based on the compilation module includes:

[0013] The compilation module compiles the OpenCL CUB library according to the MakeFile file; the MakeFile file contains CMake syntax and OpenCL CUB library compilation rules.

[0014] Optionally, the encapsulation of the platform API and runtime API in the OpenCL API based on the OpenCL API encapsulation and cache management module includes:

[0015] Based on the platform API, at least one of the following is used for the allocation of resources: device, platform, context, command queue, etc.

[0016] At least one of the following is performed based on the runtime API: compiling and executing a Program or a kernel function, and caching the compilation result in a container when the kernel function is compiled for the first time.

[0017] Optionally, the compilation and execution of the Program and kernel functions based on the runtime API interface further includes:

[0018] The device performs calculations by calling the compiled kernel function based on the runtime API.

[0019] Optionally, before providing the first call interface to the host based on the interface module, the method further includes:

[0020] It provides the header file definition for external calls, the specific implementation of the interface, and the scheduling logic between the interface and the kernel function algorithm module.

[0021] Optionally, the device-side calculation based on the runtime API call to the compiled kernel function further includes:

[0022] The data to be calculated is divided into at least one group, and the device calls at least one kernel function to perform calculations using the at least one group;

[0023] If the device calls two or more kernel functions, data is transferred between the two or more kernel functions based on OpenCL memory objects.

[0024] Optionally, the device-side calculation based on the runtime API call to the compiled kernel function further includes:

[0025] Reduce the span of data I / O by using auxiliary space;

[0026] Incorporating shared memory into WARP-level algorithms improves data loading speed;

[0027] The data storage of the algorithm is optimized based on OpenCL's vector type.

[0028] Optionally, the method further includes:

[0029] Integrate and link the header files and dynamic link libraries in the OpenCL CUB library with the deep learning framework;

[0030] Introduce specific algorithm encapsulation files for kernel functions into the deep learning framework, enabling the deep learning framework to call device-side algorithms provided by the OpenCL CUB library;

[0031] On the host side, import the singleton encapsulation and the file containing the algorithm interface, and call the specific algorithm interface through the namespace;

[0032] At the device level, the algorithm is encapsulated using a procedural approach, and necessary kernel function compilation options are added to call the corresponding kernel functions in the device code; the introduction of the OpenCL CUB library into the deep learning framework is completed.

[0033] According to a second aspect of this disclosure, an implementation apparatus for the OpenCL CUB library is provided, comprising:

[0034] The design unit is used to design OpenCL CUB library related algorithms based on the OpenCL programming standard, and to implement the OpenCL kernel functions corresponding to the related algorithms in the OpenCL CUB library.

[0035] The compilation unit is used to compile the OpenCL CUB library based on the compilation module, generating header files and dynamic link libraries; the header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithmic encapsulations for the process of calling the encapsulated APIs.

[0036] The first encapsulation unit is used to encapsulate the platform API and runtime API in the OpenCL API based on the OpenCL API encapsulation and cache management module, and to manage the platform API and runtime API based on the host side.

[0037] The first providing unit is used to provide a first calling interface to the host based on the interface module; the first calling interface is used to schedule the kernel function in the kernel function algorithm module for calculation in the host code;

[0038] The second providing unit is used to provide a second calling interface for the device-side layer based on the kernel function algorithm module; the second calling interface is used to call the kernel function in the kernel function algorithm module in other device-side code.

[0039] Optionally, the compilation unit is further configured to compile the OpenCL CUB library according to the MakeFile file; the MakeFile file contains CMake syntax and OpenCL CUB library compilation rules.

[0040] Optionally, the first packaging unit is further used for

[0041] Based on the platform API, at least one of the following is used for the allocation of resources: device, platform, context, command queue, etc.

[0042] At least one of the following is performed based on the runtime API: compiling and executing a Program or a kernel function, and caching the compilation result in a container when the kernel function is compiled for the first time.

[0043] Optionally, the first packaging unit is further configured to:

[0044] The device performs calculations by calling the compiled kernel function based on the runtime API.

[0045] Optionally, the device further includes:

[0046] The third providing unit is used to provide the interface header file definition, interface implementation and scheduling logic between the interface and kernel function algorithm module before the first providing unit provides the first calling interface to the host based on the interface module.

[0047] Optionally, the first packaging unit is further configured to:

[0048] The data to be calculated is divided into at least one group, and the device calls at least one kernel function to perform calculations using the at least one group;

[0049] When the device calls two or more kernel functions, data is transferred between the two or more kernel functions based on OpenCL memory objects.

[0050] Optionally, the first packaging unit is further configured to:

[0051] Reduce the span of data I / O by using auxiliary space;

[0052] Incorporating shared memory into WARP-level algorithms improves data loading speed;

[0053] The data storage of the algorithm is optimized based on OpenCL's vector type.

[0054] Optionally, the device further includes:

[0055] An integration unit is used to integrate and link the header files and dynamic link libraries in the OpenCL CUB library with the deep learning framework before the first providing unit provides the first calling interface to the host based on the interface module;

[0056] The first introduction unit is used to introduce the specific algorithm encapsulation file of the kernel function into the deep learning framework, so that the deep learning framework can call the device-side algorithm provided by the OpenCL CUB library.

[0057] The second import unit is used to import the file containing the singleton encapsulation and algorithm interface on the host side, and call the specific algorithm interface through the namespace;

[0058] The second encapsulation unit is used to encapsulate the algorithm at the device level using a procedural approach, while adding necessary kernel function compilation options to call the corresponding kernel functions in the device code; thus completing the introduction of the OpenCL CUB library into the deep learning framework.

[0059] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0060] At least one processor; and

[0061] A memory communicatively connected to the at least one processor; wherein,

[0062] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0063] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect above.

[0064] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0065] This disclosure discloses an implementation method, apparatus, electronic device, and storage medium for the OpenCL CUB library. Based on the OpenCL programming standard, it designs OpenCL CUB library-related algorithms and implements the corresponding OpenCL kernel functions. The OpenCL CUB library is compiled using a compilation module to generate header files and dynamic link libraries. The header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithm encapsulations for calling these encapsulated APIs. An OpenCL API encapsulation and cache management module encapsulates the platform APIs and runtime APIs in the OpenCL API and manages these APIs on the host side. An interface module provides a first calling interface to the host side, used to schedule kernel functions in the kernel function algorithm module for computation within the host-side code. A second calling interface is provided based on the kernel function algorithm module, oriented towards the device level, used to call kernel functions in the kernel function algorithm module from other device-side code. Compared with related technologies, the embodiments of this application are based on the OpenCL programming standard, design the CUB library related algorithms, and redesign the encapsulation, calling and compilation of the CUB library, thus realizing a CUB library based on the OpenCL standard, which can provide efficient support for the development of OpenCL version deep learning frameworks and OpenCL applications.

[0066] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0067] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0068] Figure 1 A flowchart illustrating an implementation method of the OpenCL CUB library provided in this embodiment of the disclosure;

[0069] Figure 2 This is an overall architecture diagram of an OpenCL CUB library provided in an embodiment of the present disclosure;

[0070] Figure 3 A flowchart illustrating a device-side algorithm optimization method provided in this embodiment of the disclosure;

[0071] Figure 4 A schematic diagram of the structure of an implementation device for the OpenCL CUB library provided in this embodiment of the disclosure;

[0072] Figure 5A schematic diagram of the structure of another implementation device for the OpenCL CUB library provided in this embodiment of the disclosure;

[0073] Figure 6 A schematic block diagram of an example electronic device 500 provided for embodiments of this disclosure. Detailed Implementation

[0074] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0075] The following description, with reference to the accompanying drawings, outlines methods, apparatuses, electronic devices, and storage media for implementing the OpenCL CUB library according to embodiments of this disclosure.

[0076] Figure 1 This is a flowchart illustrating an implementation method of the OpenCL CUB library provided in an embodiment of this disclosure.

[0077] like Figure 1 As shown, the method includes the following steps:

[0078] Step 101: Based on the OpenCL programming standard, design relevant algorithms for the OpenCL CUB library and implement the OpenCL kernel functions corresponding to the relevant algorithms in the OpenCL CUB library.

[0079] OpenCL kernel functions, based on the OpenCL programming standard, are the foundation upon which the CUB library enables the running of OpenCL deep learning frameworks and OpenCL applications.

[0080] Step 102: Compile the OpenCL CUB library based on the compilation module to generate header files and dynamic link libraries; the header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithmic encapsulations for calling the encapsulated APIs.

[0081] Please see Figure 2 , Figure 2This is an overall architecture diagram of an OpenCL CUB library provided in this application embodiment. Based on the compilation characteristics of OpenCL, oclCUB (OpenCL CUB, a high-performance computing library (CUB library) developed and implemented by the School of Software at Nankai University, containing basic operations such as prefix scanning, reduction, deduplication, and sorting) is designed to completely decouple host-side code and device-side code. OpenCL kernel functions are stored separately in a series of kernel function files and header files based on the C99 standard. However, when compiling a Program, string-based kernel function information is required. Therefore, this application embodiment calls a Python regular expression script in the compilation file to scan and match the device-side code in all files of the kernel function algorithm module, generating a corresponding string for each device-side function. The target result is saved using syntax such as add_custom_command. The target result is compiled together with other host-side code, ultimately generating the oclCUB header file and dynamic link library.

[0082] Step 103: Based on the OpenCL API encapsulation and cache management module, the platform API and runtime API in the OpenCL API are encapsulated, and the platform API and runtime API are managed on the host side.

[0083] Please continue reading. Figure 2 In large OpenCL projects, to avoid the performance overhead caused by repeated operations, the initialization process only needs to be performed once. Therefore, an effective way is to use the singleton pattern to encapsulate the initialization process. oclCUB's standardized encapsulation of the OpenCL API is mainly used for host interface design and integration with application layer development environments.

[0084] Step 104: Provide a first calling interface to the host based on the interface module; the first calling interface is used to schedule the kernel function in the kernel function algorithm module for calculation in the host code.

[0085] Please continue reading. Figure 2 The first interface module of oclCUB is mainly used at the host level, providing the corresponding host first call interface, such as oclCUB::core::Sum. The definitions of different algorithm interfaces are managed by different hpp header files. The calculation logic is implemented in the relevant cpp source files using the OpenCL API singleton class encapsulated by oclCUB, and the device-side algorithm in the kernel function algorithm module is scheduled to complete the calculation.

[0086] Step 105: Provide a second calling interface for the device-side layer based on the kernel function algorithm module; the second calling interface is used to call the kernel function in the kernel function algorithm module in other device-side code.

[0087] Please continue reading. Figure 2 The kernel function algorithm module is the computational core of oclCUB, and its device-side algorithms are key to improving overall performance. Primarily targeting kernel-level applications, this module provides the same algorithmic functionality as CUB and hipCUB, with implementations at both Warp and Block levels. Additionally, oclCUB extends this with API implementations for several device-side algorithms. These APIs, provided by CUDA, are frequently used in deep learning frameworks, such as Atomic, shfl_down_sync, shfl_xor_sync, ballot_sync, and Welford. Expanding these APIs facilitates the design of OpenCL kernel function algorithms based on oclCUB.

[0088] The implementation method of the OpenCL CUB library disclosed herein is based on the OpenCL programming standard, designs related algorithms for the OpenCL CUB library, and implements the corresponding OpenCL kernel functions in the OpenCL CUB library. The OpenCL CUB library is compiled using a compilation module to generate header files and dynamic link libraries. The header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithm encapsulations for calling these encapsulated APIs. The OpenCL API encapsulation and cache management module encapsulates the platform APIs and runtime APIs in the OpenCL API, and manages these APIs on the host side. An interface module provides a first calling interface to the host side; this first calling interface is used to schedule kernel functions in the kernel function algorithm module for computation within the host-side code. A second calling interface is provided based on the kernel function algorithm module, targeting the device side; this second calling interface is used to call kernel functions in the kernel function algorithm module from other device-side code. Compared with related technologies, the embodiments of this application are based on the OpenCL programming standard, design the CUB library related algorithms, and redesign the encapsulation, calling and compilation of the CUB library, thus realizing a CUB library based on the OpenCL standard, which can provide efficient support for the development of OpenCL version deep learning frameworks and OpenCL applications.

[0089] As an extension to the above-mentioned embodiments, when compiling OpenCL CUB based on the compilation module, the following is also included: the compilation module compiles the OpenCL CUB library according to the MakeFile file; the MakeFile file contains CMake syntax and OpenCL CUB library compilation rules.

[0090] As an extension of the above-mentioned application embodiments, step 103, which encapsulates the platform API and runtime API in the OpenCL API based on the OpenCL API encapsulation and cache management module, further includes: allocating at least one of the following resources based on the platform API: device, platform, context, command queue, etc.; compiling and executing at least one of the following based on the runtime API: program, kernel function, etc., and caching the compilation result in a container when the kernel function is compiled for the first time; if the device needs to call the kernel function for calculation, the kernel function needs to be compiled into binary or other forms before the device can call it. To reduce performance loss, unordered_map is used to bind the program and the kernel function, and the kernel function compilation result is cached in a container when it is compiled for the first time. The device calls the compiled kernel function for calculation based on the runtime API, and when the same kernel function is subsequently called, it is directly queried and read through the red-black tree in the Map during compilation.

[0091] As one possible implementation of this application, before step 104 provides the first calling interface to the host based on the interface module, the method further includes: providing the interface header file definition for external calls, the specific implementation of the interface, and the scheduling logic between the interface and the kernel function algorithm module.

[0092] The embodiments described in this application have made some optimizations to the calculation algorithm on the device side. Figure 3 A flowchart illustrating a device-side algorithm optimization method provided in this application embodiment includes:

[0093] Step 201: Divide the data to be calculated into at least one Group, and call at least one kernel function on the device side to perform calculations using the at least one Group.

[0094] To fully utilize the computing resources of the accelerated device, oclCUB divides the data to be processed into at least one group based on the size of the data to be processed when implementing the device-side algorithm. However, multiple groups can cause data synchronization problems. Therefore, the algorithm is divided into multiple kernel functions to ensure that the algorithm is thread-safe and will not cause chaos. For example, the deduplication algorithm involves the call of four kernel functions.

[0095] Step 202: If the device calls two or more kernel functions, data is transferred between the two or more kernel functions based on the OpenCL memory object.

[0096] When calling two or more kernel functions for computation, unnecessary data copying or data conversion between kernel functions may occur, which is time-consuming. Therefore, oclCUB uses OpenCL's native memory objects as parameter variables, so that OpenCL's native memory objects are passed when parameters are passed between kernel functions, which saves unnecessary time-consuming operations and improves the computational performance of oclCUB.

[0097] This application embodiment optimizes the process of processing data to be computed within a Group on the device side, including: reducing the span of data I / O by using auxiliary space; the computation results within a Group often need to be mapped to other Groups globally. Since large-scale data may be far greater than the number of threads that OpenCL can allocate, given the scheduling characteristics of OpenCL, the excess data will be processed in a loop on the available threads. This process is prone to large-span data I / O. Therefore, oclCUB focuses on reducing data I / O outside the cache when designing the algorithm. For example, the deduplication algorithm is optimized for this by using auxiliary space and temporary variables.

[0098] In the WARP-level algorithm, shared memory is used to improve data loading speed. When performing calculations within a Group, oclCUB incorporates the concept of reduction as much as possible to effectively reduce the time complexity of the algorithm and fully utilize the advantages of data synchronization within the Group. For example, when implementing calculations within a Group using a scanning algorithm, two forward and backward reduction passes can efficiently scan all data within the Group while ensuring thread safety.

[0099] The data storage of the algorithm is optimized based on OpenCL vector types. To achieve better performance, oclCUB uses local memory in the warp-level implementation to improve data I / O efficiency, and uses OpenCL vectors in sorting and other algorithms to further improve computation speed.

[0100] As one possible implementation of this application, when integrating the OpenCL CUB library into a deep learning framework, the following method can be used: Integrate and link the header files and dynamic link libraries of the OpenCL CUB library with the deep learning framework; introduce specific algorithm encapsulation files for kernel functions into the deep learning framework, enabling the deep learning framework to call device-side algorithms provided by the OpenCL CUB library; introduce singleton encapsulation and algorithm interface files on the host side, and call specific algorithm interfaces through namespaces; at the device level, encapsulate algorithms using procedural methods, while adding necessary kernel function compilation options, and call the corresponding kernel functions in the device-side code; thus completing the introduction of the OpenCL CUB library into the deep learning framework.

[0101] Corresponding to the aforementioned implementation method of the OpenCL CUB library, this invention also proposes an implementation apparatus for the OpenCL CUB library. Since the apparatus embodiments of this invention correspond to the aforementioned method embodiments, details not disclosed in the apparatus embodiments can be referred to the aforementioned method embodiments, and will not be repeated here.

[0102] Figure 4 This is a schematic diagram of the structure of an implementation device for the OpenCL CUB library provided in an embodiment of the present disclosure, as shown below. Figure 4 As shown, it includes:

[0103] Design unit 41 is used to design OpenCL CUB library related algorithms based on the OpenCL programming standard, and to implement the OpenCL kernel functions corresponding to the related algorithms in the OpenCL CUB library.

[0104] The compilation unit 42 is used to compile the OpenCL CUB library based on the compilation module, and generate header files and dynamic link libraries; the header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithmic encapsulations for the process of calling the encapsulated APIs.

[0105] The first encapsulation unit 43 is used to encapsulate the platform API and runtime API in the OpenCL API based on the OpenCL API encapsulation and cache management module, and to manage the platform API and runtime API based on the host side.

[0106] The first providing unit 44 is used to provide a first calling interface to the host based on the interface module; the first calling interface is used to schedule the kernel function in the kernel function algorithm module for calculation in the host code;

[0107] The second providing unit 45 is used to provide a second calling interface for the device-side layer based on the kernel function algorithm module; the second calling interface is used to call the kernel function in the kernel function algorithm module in other device-side code.

[0108] The implementation apparatus for the OpenCL CUB library provided in this disclosure is based on the OpenCL programming standard, designs OpenCL CUB library-related algorithms, and implements the OpenCL kernel functions corresponding to the related algorithms in the OpenCL CUB library. It compiles the OpenCL CUB library using a compilation module to generate header files and dynamic link libraries. The header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithm encapsulations for calling these encapsulated APIs. An OpenCL API encapsulation and cache management module encapsulates the platform APIs and runtime APIs in the OpenCL API, and manages these APIs on the host side. An interface module provides a first calling interface to the host side; this first calling interface is used to schedule kernel functions in the kernel function algorithm module for computation within the host-side code. A second calling interface is provided based on the kernel function algorithm module, targeting the device side; this second calling interface is used to call kernel functions in the kernel function algorithm module from other device-side code. Compared with related technologies, the embodiments of this application are based on the OpenCL programming standard, design the CUB library related algorithms, and redesign the encapsulation, calling and compilation of the CUB library, thus realizing a CUB library based on the OpenCL standard, which can provide efficient support for the development of OpenCL version deep learning frameworks and OpenCL applications.

[0109] Furthermore, in one possible implementation of this embodiment, the compiler unit 42 is also used for:

[0110] The compilation module compiles the OpenCL CUB library according to the MakeFile file; the MakeFile file contains CMake syntax and OpenCL CUB library compilation rules.

[0111] Furthermore, in one possible implementation of this embodiment, the first packaging unit 43 is also used for

[0112] Based on the platform API, at least one of the following is used for the allocation of resources: device, platform, context, command queue, etc.

[0113] At least one of the following is performed based on the runtime API: compiling and executing a Program or a kernel function, and caching the compilation result in a container when the kernel function is compiled for the first time.

[0114] Furthermore, in one possible implementation of this embodiment, the first encapsulation unit 43 is further configured to:

[0115] The device performs calculations by calling the compiled kernel function based on the runtime API.

[0116] Furthermore, in one possible implementation of this embodiment, such as Figure 5 As shown, the device further includes:

[0117] The third providing unit 46 is used to provide the interface header file definition, interface implementation and scheduling logic between the interface and kernel function algorithm module before the first providing unit 44 provides the first calling interface to the host based on the interface module.

[0118] Furthermore, in one possible implementation of this embodiment, the first encapsulation unit 43 is further configured to:

[0119] The data to be calculated is divided into at least one group, and the device calls at least one kernel function to perform calculations using the at least one group;

[0120] When the device calls two or more kernel functions, data is transferred between the two or more kernel functions based on OpenCL memory objects.

[0121] Furthermore, in one possible implementation of this embodiment, the first encapsulation unit 43 is further configured to:

[0122] Reduce the span of data I / O by using auxiliary space;

[0123] Incorporating shared memory into WARP-level algorithms improves data loading speed;

[0124] The data storage of the algorithm is optimized based on OpenCL's vector type.

[0125] Furthermore, in one possible implementation of this embodiment, such as Figure 5 As shown, the device further includes:

[0126] Integration unit 47 is used to integrate and link the header files and dynamic link libraries in the OpenCL CUB library with the deep learning framework;

[0127] The first introduction unit 48 is used to introduce the specific algorithm encapsulation file of the kernel function into the deep learning framework, so that the deep learning framework can call the device-side algorithm provided by the OpenCL CUB library.

[0128] The second import unit 49 is used to import the file containing the singleton encapsulation and algorithm interface on the host side, and call the specific algorithm interface through the namespace;

[0129] The second encapsulation unit 410 is used to encapsulate the algorithm at the device level using a procedural approach, while adding necessary kernel function compilation options to call the corresponding kernel function in the device code; thus completing the introduction of the OpenCL CUB library into the deep learning framework.

[0130] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of this embodiment, and the principle is the same, so it is not limited in this embodiment.

[0131] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0132] Figure 6 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0133] like Figure 6 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 502 or a computer program loaded from storage unit 508 into RAM (Random Access Memory) 503. RAM 503 can also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. I / O (Input / Output) interface 505 is also connected to bus 504.

[0134] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] The computing unit 501 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the implementation methods of the OpenCL CUB library. For example, in some embodiments, the implementation methods of the OpenCL CUB library may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, computing unit 501 may be configured to execute the aforementioned implementation of the OpenCLCUB library by any other suitable means (e.g., by means of firmware).

[0136] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0137] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0138] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0140] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0141] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0142] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0143] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0144] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An implementation method for the OpenCL CUB library, characterized in that, include: Based on the OpenCL programming standard, design related algorithms of the OpenCL CUB library and implement the corresponding OpenCL kernel functions in the OpenCL CUB library; The OpenCL CUB library is compiled using a compilation module to generate header files and dynamic link libraries. The header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithmic encapsulations for calling these encapsulated APIs. The OpenCL API encapsulation and cache management module encapsulates the platform API and runtime API in the OpenCL API and manages the platform API and runtime API on the host side. The interface module provides the first calling interface to the host. The first calling interface is used to schedule the kernel function in the kernel function algorithm module for computation in the host-side code; The kernel function algorithm module provides a second calling interface oriented towards the device side; the second calling interface is used to call the kernel function in the kernel function algorithm module from other device side code.

2. The method according to claim 1, characterized in that, The compilation of OpenCL CUB based on the compilation module includes: The compilation module compiles the OpenCL CUB library according to the MakeFile file; the MakeFile file contains CMake syntax and OpenCL CUB library compilation rules.

3. The method according to claim 1, characterized in that, The encapsulation of platform APIs and runtime APIs in the OpenCL API based on the OpenCL API encapsulation and cache management module includes: Based on the platform API, at least one of the following is used for the allocation of resources: device, platform, context, command queue, etc. At least one of the following is performed based on the runtime API: compiling and executing a Program or a kernel function, and caching the compilation result in a container when the kernel function is compiled for the first time.

4. The method according to claim 3, characterized in that, The compilation and execution of Program and kernel functions based on the runtime API interface also includes: The device performs calculations by calling the compiled kernel function based on the runtime API.

5. The method according to claim 1, characterized in that, Before providing the first call interface to the host based on the interface module, the method further includes: It provides the header file definition for external calls, the specific implementation of the interface, and the scheduling logic between the interface and the kernel function algorithm module.

6. The method according to claim 4, characterized in that, The device-side calculation based on the runtime API call to the compiled kernel function also includes: The data to be calculated is divided into at least one group, and the device calls at least one kernel function to perform calculations using the at least one group; If the device calls two or more kernel functions, data is transferred between the two or more kernel functions based on OpenCL memory objects.

7. The method according to claim 6, characterized in that, The device-side calculation based on the runtime API call to the compiled kernel function also includes: Reduce the span of data I / O by using auxiliary space; Incorporating shared memory into WARP-level algorithms improves data loading speed; The data storage of the algorithm is optimized based on OpenCL's vector type.

8. The method according to claims 1-7, characterized in that, The method further includes: Integrate and link the header files and dynamic link libraries in the OpenCL CUB library with the deep learning framework; Introduce specific algorithm encapsulation files for kernel functions into the deep learning framework, enabling the deep learning framework to call device-side algorithms provided by the OpenCL CUB library; On the host side, import the singleton encapsulation and the file containing the algorithm interface, and call the specific algorithm interface through the namespace; At the device level, the algorithm is encapsulated using a procedural approach, and necessary kernel function compilation options are added to call the corresponding kernel functions in the device code; the introduction of the OpenCL CUB library into the deep learning framework is completed.

9. An implementation apparatus for the OpenCL CUB library, characterized in that, include: The design unit is used to design OpenCLCUB library-related algorithms based on the OpenCL programming standard and implement the OpenCL kernel functions corresponding to the related algorithms in the OpenCLCUB library. The compilation unit is used to compile the OpenCL CUB library based on the compilation module, generating header files and dynamic link libraries; the header files contain encapsulated APIs and their names, and the dynamic link libraries contain algorithmic encapsulations for the process of calling the encapsulated APIs. The first encapsulation unit is used to encapsulate the platform API and runtime API in the OpenCL API based on the OpenCL API encapsulation and cache management module, and to manage the platform API and runtime API based on the host side. The first providing unit is used to provide a first calling interface to the host based on the interface module; The first calling interface is used to schedule the kernel function in the kernel function algorithm module for computation in the host-side code; The second providing unit is used to provide a second calling interface for the device-side layer based on the kernel function algorithm module; the second calling interface is used to call the kernel function in the kernel function algorithm module in other device-side code.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images