A performance optimization method and device for a graphics processor set communication operator

Abstract

This invention relates to the field of high-performance computing and provides a method and apparatus for optimizing the performance of a graphics processing unit (GPU) aggregate communication operator. The method includes: allocating device-side storage space on the host side; copying the structured data to be communicated to the device-side storage space to obtain a target data pointer; adjusting the kernel function parameter of the aggregate communication operator from a structure type to a pointer type and passing the target data pointer; executing the kernel function on the GPU side, directly accessing the device-side storage space based on the target data pointer to complete the aggregate communication operation; and releasing the device-side storage space after the aggregate communication operation is completed. This invention solves the problem of excessive GPU register pressure and decreased cache hit rate caused by inefficient memory access instructions generated by the compiler to pass structure parameters in existing technologies. It avoids the generation of inefficient instructions, reduces register pressure, and improves execution efficiency.

Description

Technical Field

[0001] This invention relates to the field of high-performance computing, and in particular to a method and apparatus for performance optimization of a graphics processor aggregation communication operator. Background Technology

[0002] High-performance computing and deep learning training rely heavily on parallel architectures with multiple graphics processing units (GPUs), where ensemble communication operations are a critical factor affecting overall system performance. Ensemble communication libraries provide highly optimized implementations for this type of communication and have become an industry standard.

[0003] Currently, for small-scale data collection communication scenarios, existing technologies typically pass structured data directly as parameters to GPU kernel functions. In this implementation, the compiler often splits the structure into multiple 128-bit or wider data blocks when generating low-level instructions, and generates specific memory access instructions to complete the data loading and transfer. Although this approach has certain advantages in terms of generality, on certain GPU hardware platforms, these instructions can significantly increase the pressure on thread-local registers and may cause register overflows, thereby negatively impacting the GPU's L2 cache hit rate and overall communication bandwidth.

[0004] Therefore, how to effectively avoid generating low-level instructions that are detrimental to the performance of specific hardware platforms by the compiler, reduce register usage pressure, and improve cache efficiency while maintaining the generality of communication operators has become an urgent technical problem to be solved in the current GPU ensemble communication optimization. Summary of the Invention

[0005] This invention provides a method and apparatus for optimizing the performance of a graphics processor's collective communication operator. It solves the problem of excessive GPU register pressure and decreased cache hit rate caused by the compiler generating inefficient memory access instructions to pass structure parameters in the prior art. It avoids the generation of inefficient instructions, reduces register pressure, and improves execution efficiency.

[0006] This invention provides a performance optimization method for a graphics processor ensemble communication operator, comprising the following steps:

[0007] On the host side, request device-side storage space, copy the structured data to be communicated to the device-side storage space, and obtain the target data pointer;

[0008] Change the kernel function parameter of the set communication operator from a structure type to a pointer type, and pass in the target data pointer;

[0009] The kernel function is executed on the graphics processor side, and the set communication operation is completed by directly accessing the device-side storage space based on the target data pointer.

[0010] Release the device-side storage space after the collection communication operation is completed.

[0011] According to a performance optimization method for a graphics processor set communication operator provided by the present invention, the step of requesting device-side storage space on the host side and copying the structured data to be communicated to the device-side storage space to obtain a target data pointer specifically includes: while asynchronously and parallelly executing the preceding graphics processor kernel on the host side, calling the storage allocation interface to request contiguous device-side storage space for the structured data; continuously writing the structured data into the device-side storage space in blocks of a preset bit width through the host-side storage copy interface to form a data layout aligned with the kernel access mode; and using the returned starting address of the device-side storage space as the target data pointer.

[0012] According to the present invention, a performance optimization method for a graphics processor set communication operator is provided, wherein adjusting the kernel function parameters of the set communication operator from structure type to pointer type and passing in the target data pointer specifically includes: parsing the original kernel function interface of the set communication operator, replacing the structure parameters in its parameter list with device memory pointer parameters; and directly assigning the target data pointer as an actual parameter to the changed memory pointer parameter.

[0013] According to the performance optimization method of the graphics processor (GPU) aggregate communication operator provided by the present invention, the step of executing the kernel function on the GPU side and directly accessing the device-side storage space based on the target data pointer to complete the aggregate communication operation specifically includes: using the target data pointer in the kernel parameters as the only data entry on the GPU side, reading continuous communication data blocks directly from the device-side storage space according to the thread bundle level parallel granularity; executing the specified aggregate communication operation based on the read data blocks, controlling the number of concurrently executed thread bundles during the execution process, and writing the result of the aggregate communication operation back to the device-side storage space or downstream communication buffer.

[0014] According to a performance optimization method for a graphics processor set communication operator provided by the present invention, the step of releasing the device-side storage space after the set communication operation is completed specifically includes: inserting an end event synchronized with the kernel function execution flow on the host side to confirm that all thread bundles have completed read and write operations on the device-side storage space; calling the storage release interface to reclaim the contiguous device-side storage space pointed to by the target data pointer; and setting the target data pointer maintained on the host side to null.

[0015] According to the present invention, a performance optimization method for a graphics processor set communication operator is provided, wherein the set communication operator includes full reduction, reduction, broadcasting and full collection, and the data volume of the structured data does not exceed a preset number of bytes.

[0016] The present invention also provides a performance optimization device for a graphics processor aggregate communication operator, comprising the following modules:

[0017] The space allocation module is used to request device-side storage space on the host side, copy the structured data to be communicated to the device-side storage space, and obtain the target data pointer;

[0018] The type adjustment module is used to adjust the kernel function parameters of the set communication operator from structure type to pointer type and pass in the target data pointer;

[0019] The computation module is used to execute the kernel function on the graphics processor side and directly access the device-side storage space based on the target data pointer to complete the set communication operation;

[0020] The space release module is used to release the device-side storage space after the collection communication operation is completed.

[0021] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a performance optimization method for a graphics processor set communication operator as described above.

[0022] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a performance optimization method for a graphics processor set communication operator as described above.

[0023] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a performance optimization method for a graphics processor set communication operator as described above.

[0024] This invention provides a performance optimization method and apparatus for a graphics processing unit (GPU) collective communication operator, which offers the following advantages: By pre-allocating device-side storage space on the host side and copying structured data therein, the structure data that originally needed to be loaded item by item in the GPU kernel through specific memory access instructions is converted into data blocks stored contiguously in the device-side memory. This preprocessing step, combined with the subsequent adjustment of kernel function parameters from structure types to pointer types pointing to this contiguous storage space, fundamentally changes the way the GPU kernel accesses data. During kernel execution, it no longer needs to process scattered structure members, but directly accesses the pre-laid contiguous data blocks in the device-side storage space through the passed target data pointer. This change in data access mode eliminates the need for the compiler to generate the inefficient memory access instructions for structure loading when compiling kernel code, thereby effectively reducing the register usage pressure during GPU kernel execution. The reduction in register pressure not only reduces the performance loss caused by register overflow, but also indirectly improves the utilization and hit rate of the GPU's L2 cache. By optimizing the data flow and instruction set, the overall computational efficiency and performance of the GPU collective communication operator are improved. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the performance optimization method for the graphics processor set communication operator provided by the present invention.

[0027] Figure 2 This is a schematic diagram of the performance optimization device for the graphics processor set communication operator provided by the present invention.

[0028] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] The terminology involved in this invention will be explained below.

[0031] All-Gather is a set communication operation in which all participating processes provide input data, and each process eventually obtains a complete set of all input data.

[0032] All-Reduce is a collective communication operation in which all participating processes provide input data and ultimately obtain the same result after all input data has undergone a specified operation.

[0033] Asynchronous Memory Copy (cudaMemcpyAsync) is a function provided by the CUDA runtime for asynchronously copying data between host memory and device memory. This operation can be performed in a specific stream and can overlap with host computation or other device operations.

[0034] Broadcast is a collection communication operation that sends a copy of data from one process (the root process) to all other participating processes in the collection.

[0035] The Compute Unified Device Architecture (CUDA) consists of a general-purpose parallel computing platform and programming model that allows developers to use languages ​​such as C++ to perform complex computations on GPUs.

[0036] CUDA events (CUDA Event, cudaEventRecord / cudaEventSynchronize) are a mechanism in the CUDA programming model used to mark specific points in a stream and synchronize the execution of the stream. They are often used for precise timing and flow control.

[0037] A CUDA Stream is a concept in the CUDA programming model that represents a sequence of operations on a GPU, executed sequentially. Different streams can be executed concurrently to achieve task-level parallelism.

[0038] Device Memory Allocation (cudaMalloc) is a function provided by the CUDA runtime that allocates a specified amount of linear memory on a GPU device and returns a pointer to that memory.

[0039] Device Memory Free (cudaFree) is a function provided by the CUDA runtime to release memory previously allocated on the GPU device through interfaces such as `cudaMalloc`.

[0040] Direct Memory Access (DMA) is a technique that allows certain hardware subsystems in a computer to directly read and write system memory without the intervention of the central processing unit, which can significantly improve data transfer efficiency.

[0041] Global memory is the largest type of main device memory on the GPU, accessible to all threads, but it has a high access latency.

[0042] A grid is a logical concept in the CUDA programming model that represents the collection of all threads when a kernel function starts, consisting of one or more thread blocks.

[0043] Level 2 cache (L2 cache) is a high-speed memory located between the processor cores and main memory. It is used to store frequently accessed data to reduce access latency. In a GPU, the L2 cache is shared by all streaming multiprocessors.

[0044] Pinned memory (non-paged memory) is a type of host memory that is locked in physical memory and cannot be swapped out to disk, allowing for high-speed DMA data transfers with devices.

[0045] Reduce is a collective communication operation that combines the input data of all participating processes into a single result through a specified operation.

[0046] Register spill occurs when the number of variables allocated by the compiler exceeds the capacity of the hardware's physical registers, causing some variables to "spill" into slower local or global memory, resulting in performance degradation.

[0047] Shared memory is a high-speed, programmable on-chip memory shared within each thread block on the GPU, allowing threads within the same thread block to communicate and collaborate with low latency.

[0048] A Streaming Multiprocessor (SM) is a core computing unit in a GPU architecture, responsible for executing a bundle of threads. A GPU contains multiple Streaming Multiprocessors.

[0049] A thread block is a logical concept in the CUDA programming model that represents a group of cooperating threads that can be scheduled to execute on a streaming multiprocessor and share shared memory within the block.

[0050] A warp is the basic execution unit in the GPU execution model. On a GPU, it typically contains 32 threads that execute the same instructions in a lockstep (SIMT) manner.

[0051] In programming languages ​​such as C / C++, a null pointer (NULL) is a special value that indicates that the pointer does not point to any valid memory address.

[0052] The following is combined Figures 1-3 The embodiments of the present invention are described in detail.

[0053] Figure 1 This is a flowchart illustrating the performance optimization method for the graphics processor set communication operator provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps:

[0054] S110. Apply for device-side storage space on the host side, copy the structured data to be communicated to the device-side storage space, and obtain the target data pointer.

[0055] According to a performance optimization method for a graphics processor set communication operator provided by the present invention, the step of requesting device-side storage space on the host side and copying the structured data to be communicated to the device-side storage space to obtain a target data pointer specifically includes: while asynchronously and parallelly executing the preceding graphics processor kernel on the host side, calling the storage allocation interface to request contiguous device-side storage space for the structured data; continuously writing the structured data into the device-side storage space in blocks of a preset bit width through the host-side storage copy interface to form a data layout aligned with the kernel access mode; and using the returned starting address of the device-side storage space as the target data pointer.

[0056] Specifically, on the host side, when a set communication operator needs to be executed, the system first prepares the structured data to be communicated. This data is typically a structure instance defined in the host memory, and its size is usually small (e.g., no more than 8KB). To optimize data transmission and access, the host CPU asynchronously requests storage space on the device side.

[0057] Specifically, a contiguous block of device-side storage (i.e., video memory) is allocated for the structured data by calling a storage allocation interface similar to cudaMalloc. This allocation operation can be performed asynchronously and in parallel with one or more currently executing preceding graphics processor cores, thus overlapping memory management overhead with computation and hiding some latency. After successful storage allocation, the host immediately moves the structured data from host memory by calling a host-side storage copy interface similar to cudaMemcpyAsync. During this process, instead of a simple byte-to-byte copy, the data members within the structure are intentionally split and reassembled into a series of contiguous, aligned data blocks, which are then continuously written into the previously allocated device-side storage. This layout aims to simulate or directly match the access patterns of threads in subsequent graphics processor cores, ensuring that each thread or thread bundle can achieve merged memory access when accessing data. The starting address of the device-side storage returned by this storage allocation operation is assigned to a pointer variable as the target data pointer used in subsequent operations.

[0058] This embodiment achieves significant technical benefits by completing the allocation of contiguous storage space and the optimized layout of data blocks on the device side during the asynchronous parallel data preparation phase on the host. First, this operation moves the structure parameter parsing and passing process, originally performed during kernel startup, to the host side, eliminating the need for the kernel to process the original structure and fundamentally preventing the compiler from generating inefficient ld.tlm class memory access instructions. Second, splitting data by specific position and width and storing it contiguously ensures that the data layout in device memory highly matches the kernel's access patterns, significantly improving the hit rate of the GPU's L2 cache during subsequent kernel execution. Finally, these measures work together to effectively reduce the pressure on kernel register usage, minimize performance fluctuations caused by register overflows, and improve the overall execution efficiency and stability of the set communication operator.

[0059] In another embodiment of the present invention, when the host side calls the storage allocation interface, it can specify the request for pinned memory as the device-side storage space. This type of memory can support the highest speed data transfer between the host and the device. Subsequently, when transferring data through the storage copy interface, the host can further specify the use of a finer bit width (e.g., 512 bits) to split and organize structured data, and utilize the DMA engine for transfer to maximize bus utilization. This high degree of optimization for the transmission stage is particularly suitable for application scenarios that are extremely sensitive to communication latency.

[0060] In another embodiment of the present invention, the host side can pre-allocate the required contiguous device-side storage space from a dedicated device memory pool for storing small-scale communication data, instead of performing a general storage allocation each time. The block size and address in the memory pool are pre-aligned and optimized. When writing data via the storage copy interface, the system determines the optimal bit width for data splitting based on the cache line size of the target hardware architecture (e.g., strictly aligned to 128 bits), ensuring that each data block falls completely into the cache line. This method reduces the overhead of dynamic allocation and further optimizes data access speed through hardware caching mechanisms.

[0061] S120. Change the kernel function parameter of the set communication operator from a structure type to a pointer type, and pass in the target data pointer.

[0062] According to the present invention, a performance optimization method for a graphics processor set communication operator is provided, wherein adjusting the kernel function parameters of the set communication operator from structure type to pointer type and passing in the target data pointer specifically includes: parsing the original kernel function interface of the set communication operator, replacing the structure parameters in its parameter list with device memory pointer parameters; and directly assigning the target data pointer as an actual parameter to the changed memory pointer parameter.

[0063] According to the present invention, a performance optimization method for a graphics processor set communication operator is provided, wherein the set communication operator includes full reduction, reduction, broadcasting and full collection, and the data volume of the structured data does not exceed a preset number of bytes.

[0064] Specifically, for set communication operators such as All-Reduce, Reduce, Broadcast, or All-Gather, where the size of a single structured data block to be processed is typically designed to be small-scale (e.g., no larger than a preset value, such as 8KB), the kernel function interface needs to be adapted before the target data pointer is passed to the kernel. The system first parses the original kernel function source code or its declaration of the target set communication operator (taking All-Reduce as an example). The parameter list of this function prototype contains one or more structure parameters for input / output. By modifying the source code or using macro definitions for text replacement, the data type of this structure parameter is changed to a pointer to device memory, for example, from `struct DataBlock param` to... or This change ensures that the kernel function interface now expects to receive a memory address, rather than a copy of a structure.

[0065] When this modified kernel function is called on the host side, the "target data pointer," obtained in the previous steps and pointing to the structured data stored in the device's memory space with an optimized layout, is directly assigned as the actual argument to this new pointer parameter. This process does not involve any copying or reassembly of the structure data; it is merely a transfer of an address value. This completes the fundamental shift from transferring "data content" to transferring "data location," eliminating the need for the compiler to generate ld.tlm instructions that load structures item by item.

[0066] This embodiment fundamentally avoids the compiler automatically generating inefficient thread-bound local memory loading instructions due to handling structure parameter passing by changing the kernel function parameter from a structure type to a device memory pointer type. This change in parameter passing directly reduces the register allocation pressure on the GPU kernel during startup and execution, and enables the kernel to directly access contiguous device memory blocks through pointers, thereby improving memory access efficiency and L2 cache hit rate, and ultimately significantly improving the execution performance of small-scale data set communication operators.

[0067] In another embodiment of the present invention, for reduce operations with a data size explicitly less than or equal to 4KB, a more specific pointer type (e.g., ...) can be used when modifying the kernel function interface. or The original generic structure parameters are replaced with vectorized types, which directly correspond to the GPU's efficient vectorized loading instructions. During the call, the target data pointer is appropriately type-casted before being assigned to the new parameter. This method not only achieves parameter type replacement but also provides additional optimization hints to the compiler through vectorized types, potentially leading to more efficient memory transactions at the hardware level.

[0068] In another embodiment of the present invention, when processing operators such as Broadcast, the kernel function may need to receive a small control structure (e.g., containing information such as the root process ID and data size, with a total size not exceeding 256 bytes) and the main data payload. In this scenario, only the control structure parameter can be replaced with a device memory pointer parameter, while the main data payload is still passed through other pointer parameters. During the call, the host needs to allocate a separate block of device memory for this small control structure, copy its contents there, and then pass the address of that memory as an actual parameter. This demonstrates the flexibility of pointerizing some structure parameters in the function parameter list, suitable for scenarios with mixed parameter types.

[0069] S130. Execute the kernel function on the graphics processor side, and directly access the device-side storage space based on the target data pointer to complete the set communication operation.

[0070] According to the performance optimization method of the graphics processor (GPU) aggregate communication operator provided by the present invention, the step of executing the kernel function on the GPU side and directly accessing the device-side storage space based on the target data pointer to complete the aggregate communication operation specifically includes: using the target data pointer in the kernel parameters as the only data entry on the GPU side, reading continuous communication data blocks directly from the device-side storage space according to the thread bundle level parallel granularity; executing the specified aggregate communication operation based on the read data blocks, controlling the number of concurrently executed thread bundles during the execution process, and writing the result of the aggregate communication operation back to the device-side storage space or downstream communication buffer.

[0071] Specifically, when the modified set communication kernel function begins execution on the graphics processor side, all threads within it use the target data pointer passed as a kernel parameter as the sole data source entry point when they need to access raw input data. The 32 threads within each thread bundle, based on their global thread index, collaboratively read contiguous blocks of communication data from the device-side memory pointed to by that pointer. For example, a thread bundle can read a contiguous 128-byte block of data at once, which corresponds precisely to multiple data units previously split and laid out in 128-bit widths on the host side. Because the data is stored contiguously and aligned in device memory, this access mode achieves efficient merged memory access, maximizing memory bandwidth utilization. When performing a specified set communication operation based on the read data blocks, the system precisely controls the number of concurrently executing thread bundles by explicitly limiting the dimensions of the grid or block in the kernel startup configuration, or by dynamically controlling the number of active thread bundles within the kernel. This control aims to ensure that the total demand of all active thread bundles on thread-local registers does not exceed the capacity of the hardware physical registers, thereby preventing register spills at the source. After the calculation is completed, the result data generated by the collection communication operation is written directly back to the same device-side storage space pointed to by the "target data pointer" (updated in place), or written to another specially allocated downstream communication buffer for use in subsequent operations.

[0072] This embodiment ensures high efficiency in memory access and improves L2 cache hit rate by forcing the kernel to access contiguous device memory blocks using pointers as the sole entry point. By precisely controlling the number of concurrent thread bundles, it effectively avoids register overflow issues caused by register resource contention, keeping critical data in high-speed registers, thereby significantly reducing the latency overhead of accessing global memory and ultimately greatly improving the computational efficiency and overall throughput of the collection communication kernel.

[0073] In another embodiment of the present invention, when the kernel performs a broadcast operation, each thread bundle, in addition to reading data using the target data pointer as an entry point, can also utilize the GPU's shared memory as an intermediate cache. The thread bundle first cooperatively loads the contiguous data blocks to be broadcast from the device's global memory into the shared memory, and then performs the subsequent broadcast operation. Simultaneously, by setting the size of the thread block to an integer multiple of the hardware thread bundle size and appropriately reducing it, the number of concurrently executing thread bundles on each streaming multiprocessor is precisely controlled. This method further reduces the access pressure on global memory when data needs to be repeatedly accessed by multiple thread bundles within a thread block, and, combined with thread bundle number control, achieves dual optimization.

[0074] In another embodiment of the present invention, a multi-stage kernel design can be employed when performing the All-Reduce operation. In the first stage, each thread bundle reads consecutive blocks of input data from the target data pointer and performs local reduction. During this stage, the total number of globally concurrent thread bundles is actively limited by starting fewer thread bundles to ensure that each thread bundle has sufficient register resources to process intermediate computation results and avoid overflow. In the subsequent global synchronization and result writing stages, different kernel configurations or more thread bundles can be started to write the final results to the downstream communication buffer in parallel. This phased, dynamically adjusted concurrent thread bundle size strategy enables fine-grained control of register resources within complex communication primitives.

[0075] S140. Release the device-side storage space after the collection communication operation is completed.

[0076] According to a performance optimization method for a graphics processor set communication operator provided by the present invention, the step of releasing the device-side storage space after the set communication operation is completed specifically includes: inserting an end event synchronized with the kernel function execution flow on the host side to confirm that all thread bundles have completed read and write operations on the device-side storage space; calling the storage release interface to reclaim the contiguous device-side storage space pointed to by the target data pointer; and setting the target data pointer maintained on the host side to null.

[0077] Specifically, after completing the collective communication operation on the graphics processor side, the host inserts a specific synchronization event (e.g., cudaEventRecord in CUDA followed by cudaEventSynchronize) after the kernel function execution flow. This completion event is tightly bound to the kernel execution flow. By querying the completion status of this event, the host accurately and reliably confirms that all dispatched thread bundles have been executed, especially ensuring that all read and write operations on the device-side memory space (including possible in-situ updates or writing results to that space) have been completely completed. Only after receiving the completion signal of this event does the host consider the memory region pointed to by the target data pointer to be in a state that can be safely released. Subsequently, the host immediately calls a memory release interface such as cudaFree to reclaim the previously allocated contiguous device-side memory space pointed to by the target data pointer. This operation returns this video memory to the device memory manager for subsequent allocation. As an important safety measure, the host explicitly sets the target data pointer variable it maintains to NULL (null pointer). This operation effectively prevents subsequent code from misusing the address that has become a "dangling pointer," thereby avoiding hard-to-track program errors and ensuring the rigor of resource management and the stability of program operation.

[0078] This embodiment ensures the safety of memory release operations through an event synchronization mechanism, strictly avoiding data races or program crashes caused by erroneous memory reclamation while the GPU kernel is still accessing data. Timely reclamation of temporary device memory effectively prevents video memory leaks, ensuring the stability of long-running applications. Finally, the good programming practice of setting pointers to null eliminates the potential risks of dangling pointers, improving the robustness and reliability of the entire optimization method.

[0079] In another embodiment of the present invention, in a streaming processing environment, the host side does not directly wait synchronously for an end event. Instead, after the kernel function, a device-side memory release operation is inserted as a task into the same CUDA stream. This release task is automatically scheduled in the stream by the GPU driver, ensuring that the release operation is only actually executed after all previous tasks in the stream (including kernels that need to release memory) have completed. The host program does not need to block and can continue processing other transactions. This method shifts the synchronous responsibility of the release operation from the host to the device stream, achieving a higher degree of asynchronous execution and low latency on the host side.

[0080] In another embodiment of the present invention, the host does not immediately release the device-side storage space, but instead reclaims it into a dedicated device memory pool maintained for small-scale temporary communication data. Before reclamation, it is also necessary to confirm the kernel's completion via a synchronization event. The memory pool manager receives this used memory and marks its status as free and available, rather than returning it to the system. When temporary space is needed again for a new small-scale collection of communication operations, the memory pool can quickly allocate a block of appropriate size from the free list, avoiding the overhead of frequently requesting and releasing memory from the system. This is particularly suitable for scenarios that require repeated small-scale communication, thereby improving the overall resource reuse rate and allocation efficiency.

[0081] The performance optimization device for the graphics processor aggregate communication operator provided by the present invention is described below. The performance optimization device for the graphics processor aggregate communication operator described below can be referred to in correspondence with the performance optimization method for the graphics processor aggregate communication operator described above.

[0082] like Figure 2 The image shows a performance optimization device for a graphics processor aggregation communication operator provided by the present invention, comprising:

[0083] The space allocation module 210 is used to request device-side storage space on the host side, copy the structured data to be communicated to the device-side storage space, and obtain the target data pointer;

[0084] The type adjustment module 220 is used to adjust the kernel function parameter of the set communication operator from a structure type to a pointer type and pass in the target data pointer;

[0085] The arithmetic module 230 is used to execute the kernel function on the graphics processor side and directly access the device-side storage space based on the target data pointer to complete the set communication operation;

[0086] The space release module 240 is used to release the device-side storage space after the collection communication operation is completed.

[0087] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340. The processor 310, communications interface 320, and memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute a performance optimization method for the graphics processor's aggregate communication operator. This method includes: requesting device-side storage space on the host side; copying the structured data to be communicated to the device-side storage space to obtain a target data pointer; adjusting the kernel function parameter of the aggregate communication operator from a structure type to a pointer type and passing in the target data pointer; executing the kernel function on the graphics processor side, directly accessing the device-side storage space based on the target data pointer to complete the aggregate communication operation; and releasing the device-side storage space after the aggregate communication operation is completed.

[0088] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0089] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the performance optimization method for the graphics processor set communication operator provided by the above methods. The method includes: requesting device-side storage space on the host side, copying the structured data to be communicated to the device-side storage space to obtain a target data pointer; adjusting the kernel function parameter of the set communication operator from a structure type to a pointer type and passing in the target data pointer; executing the kernel function on the graphics processor side, directly accessing the device-side storage space based on the target data pointer to complete the set communication operation; and releasing the device-side storage space after the set communication operation is completed.

[0090] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a performance optimization method for executing the graphics processor set communication operator provided by the above methods. The method includes: requesting device-side storage space on the host side; copying the structured data to be communicated to the device-side storage space to obtain a target data pointer; adjusting the kernel function parameter of the set communication operator from a structure type to a pointer type and passing in the target data pointer; executing the kernel function on the graphics processor side; directly accessing the device-side storage space based on the target data pointer to complete the set communication operation; and releasing the device-side storage space after the set communication operation is completed.

[0091] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0093] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A performance optimization method for a graphics processor ensemble communication operator, characterized in that, include: On the host side, request device-side storage space, copy the structured data to be communicated to the device-side storage space, and obtain the target data pointer; Change the kernel function parameter of the set communication operator from a structure type to a pointer type, and pass in the target data pointer; The kernel function is executed on the graphics processor side, and the collection communication operation is completed by directly accessing the device-side storage space based on the target data pointer. Specifically, this includes: on the graphics processor side, using the target data pointer in the kernel parameters as the only data entry point, reading consecutive communication data blocks directly from the device-side storage space at the thread bundle level of parallelism; executing the specified collection communication operation based on the read data blocks, controlling the number of concurrently executed thread bundles during the execution process, and writing the collection communication operation result back to the device-side storage space or downstream communication buffer; when the kernel is executing the broadcast operation, each thread bundle, in addition to reading data with the target data pointer as the entry point, also uses the graphics processor's shared memory as an intermediate cache; the thread bundle first cooperatively loads the consecutive data blocks to be broadcast from the device-side global memory into the shared memory, and then performs the subsequent broadcast operation; Release the device-side storage space after the collection communication operation is completed.

2. The performance optimization method for the graphics processor ensemble communication operator according to claim 1, characterized in that, The step of requesting device-side storage space on the host side, copying the structured data to be communicated to the device-side storage space, and obtaining the target data pointer specifically includes: While the preceding graphics processor kernel is executed asynchronously and in parallel on the host side, the storage allocation interface is called to request contiguous device-side storage space for the structured data; The structured data is continuously written into the device-side storage space in blocks of a preset bit width through the host-side storage copy interface, forming a data layout aligned with the kernel access mode. Use the returned starting address of the device's storage space as the target data pointer.

3. The performance optimization method for the graphics processor ensemble communication operator according to claim 1, characterized in that, The step of changing the kernel function parameter of the set communication operator from a structure type to a pointer type and passing in the target data pointer specifically includes: Parse the original kernel function interface of the set communication operator and replace the structure parameters in its parameter list with device memory pointer parameters; The target data pointer is directly assigned as the actual parameter to the modified memory pointer parameter.

4. The performance optimization method for the graphics processor ensemble communication operator according to claim 1, characterized in that, The step of releasing the device-side storage space after the collection communication operation is completed specifically includes: Insert an end event synchronized with the kernel function execution flow on the host side to confirm that all thread bundles have completed read and write operations on the device-side storage space; Call the storage release interface to reclaim the contiguous device-side storage space pointed to by the target data pointer; Set the target data pointer maintained on the host side to null.

5. The performance optimization method for a graphics processor ensemble communication operator according to any one of claims 1-4, characterized in that, The set communication operators include full reduction, reduction, broadcasting and full collection, and the amount of structured data does not exceed a preset number of bytes.

6. A performance optimization device for a graphics processor's integrated communication operator, characterized in that, include: The space allocation module is used to request device-side storage space on the host side, copy the structured data to be communicated to the device-side storage space, and obtain the target data pointer; The type adjustment module is used to adjust the kernel function parameters of the set communication operator from structure type to pointer type and pass in the target data pointer; The computation module is used to execute the kernel function on the graphics processor side, and to directly access the device-side storage space based on the target data pointer to complete the collection communication operation. Specifically, it includes: using the target data pointer in the kernel parameters as the only data entry point on the graphics processor side, reading continuous communication data blocks directly from the device-side storage space according to the thread bundle level of parallelism; executing the specified collection communication operation based on the read data blocks, and controlling the number of concurrently executed thread bundles during the execution process, and writing the collection communication operation result back to the device-side storage space or downstream communication buffer; when the kernel is executing the broadcast operation, each thread bundle, in addition to reading data with the target data pointer as the entry point, also uses the shared memory of the graphics processor as an intermediate cache; the thread bundle first cooperatively loads the continuous data blocks to be broadcast from the device-side global memory into the shared memory, and then performs the subsequent broadcast operation; The space release module is used to release the device-side storage space after the collection communication operation is completed.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the performance optimization method for the graphics processor set communication operator as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the performance optimization method for the graphics processor set communication operator as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the performance optimization method for the graphics processor set communication operator as described in any one of claims 1 to 5.

Images