High-bandwidth memory capacity expansion method and device, equipment and storage medium

Abstract

The invention provides a high-bandwidth memory capacity expansion method and device, equipment and a storage medium, and the method comprises the steps: connecting capacity expansion card equipment with a graphics processor through a PCIe interface on the graphics processor, and enabling the storage space of the capacity expansion card equipment to serve as the shared storage space of a high-bandwidth memory (HBM). Firstly, in a kernel function of a graphics processor, a shared storage space is declarated by using a shared keyword, and an extended declaration is generated. Secondly, copying HBM memory data to a shared storage space of the expansion card equipment through threads in the thread block, and performing thread synchronization on the thread block by using a synchronization function; and on the basis, parallel computing processing is performed, and a computing result is allocated and copied between the shared storage space of the HBM and the shared storage space of the capacity expansion card equipment, so that storage capacity expansion is realized. According to the method, the HBM capacity can be effectively expanded, the big data and big model processing requirements are met, the GPU computing efficiency is improved, and the expansion cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of graphics processors, and in particular to a high-bandwidth memory expansion method, device, equipment and storage medium. Background Art

[0002] With the increasing popularity of multi-type applications of big data and large models, data storage and computing performance have become key factors affecting the computing efficiency of graphics processing units (GPUs). High-bandwidth memory (HBM), as a cache storage medium for GPU core computing, greatly affects the amount of data that can be processed during artificial intelligence training and reasoning. However, the HBM capacity integrated in existing GPUs is limited and can usually only support smaller data models, which cannot meet the computing needs of current artificial intelligence models. Although expanding the HBM capacity by connecting multiple GPUs in parallel can solve some problems, this method is costly and has poor adaptability to various application scenarios. Summary of the invention

[0003] The main purpose of the present invention is to solve the technical problem that the existing high-bandwidth memory capacity is insufficient and it is difficult to meet the processing requirements of large-scale data models; A first aspect of the present invention provides a method for expanding the capacity of a high-bandwidth memory, wherein the graphics processor is integrated with a high-bandwidth memory, and the method for expanding the capacity of the high-bandwidth memory comprises: Connecting the expansion card device to the graphics processor via a PCIe interface on the graphics processor, so that the storage space in the expansion card device is used as the shared storage space of the high bandwidth memory; In the kernel function of the graphics processor, a declaration process is performed on the shared storage space according to a preset shared keyword to obtain an extended declaration of the expansion card device; In the kernel function, according to the extension declaration, the memory data of the high bandwidth memory is copied to the shared storage space of the expansion card device through the threads in the preset thread block, and the thread block is synchronized through the preset synchronization function; Parallel computing is performed on the shared data set on the high bandwidth memory and the expansion card device to obtain computing result data, and the computing result data is distributed and copied between the shared storage space of the high bandwidth memory and the expansion card device to achieve capacity expansion of the high bandwidth memory.

[0004] Optionally, in a first implementation of the first aspect of the present invention, connecting the expansion card device to the graphics processor through a PCIe interface on the graphics processor to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory includes: Connecting the expansion card device to the graphics processor through the PCIe interface on the graphics processor, and initializing the root complex of the graphics processor to obtain a system topology structure connecting the CPU, the memory subsystem and the PCIe device; According to the system topology, enumerate the PCIe bus of the PCIe interface to obtain a function list including different combinations of bus numbers and device numbers; Based on the function list, the configuration space of the expansion card device is read and processed to obtain the identity information and hardware characteristics of the expansion card device; According to the identity information and hardware characteristics, a base address register of the expansion card device is configured to obtain a mapping position of the expansion card device in the memory address space of the graphics processor; Based on the mapping position, the memory mapping I / O system of the graphics processor is updated to obtain an extended memory access mechanism including the storage space of the expansion card device, so as to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory.

[0005] Optionally, in a second implementation of the first aspect of the present invention, the declaring the shared storage space using a preset shared keyword in the kernel function of the graphics processor to obtain an extended declaration of the expansion card device includes: Preprocessing the kernel function code of the graphics processor to obtain a code segment of a shared keyword, and inserting an identifier of the expansion card device into the code segment to obtain a shared memory declaration statement of the extended memory; Performing syntax parsing on the shared memory declaration statement to obtain a parameter list including the storage space size of the expansion card device; According to the parameter list, the firmware version of the expansion card device is detected, and based on the firmware version, the extended shared memory declaration statement is conditionally compiled to obtain an extended declaration of the expansion card device adapted to the firmware version.

[0006] Optionally, in a third implementation manner of the first aspect of the present invention, in the kernel function, copying the memory data of the high bandwidth memory to the shared storage space of the expansion card device through threads in a preset thread block according to the extended declaration, and performing thread synchronization on the thread block through a preset synchronization function includes: In the kernel function, determining a thread block index and a thread index in a preset thread block according to the extended declaration; Determine the global memory access address corresponding to each thread according to the thread block index and the thread index; The memory data in the high bandwidth memory is copied to the shared storage space of the expansion card device through the threads in the thread block according to the global memory access address, and the thread block is synchronized through a preset synchronization function.

[0007] Optionally, in a fourth implementation manner of the first aspect of the present invention, the performing thread synchronization on the thread block by using a preset synchronization function includes: The execution status of each thread in the thread block is marked by a preset synchronization function to obtain a status identifier of the thread execution reaching the synchronization point; According to the state identifier, updating the thread counter in the thread block to obtain the number of threads that have reached the synchronization point; The number of threads that have reached the synchronization point is judged to obtain a judgment result of whether all threads in the thread block have reached the synchronization point, and the thread block is synchronized according to the judgment result.

[0008] Optionally, in a fifth implementation of the first aspect of the present invention, performing parallel computing processing on the high bandwidth memory and the shared data set on the expansion card device to obtain computing result data, and distributing and copying the computing result data between the high bandwidth memory and the shared storage space of the expansion card device to achieve capacity expansion of the high bandwidth memory includes: Performing task division processing on the shared data set on the high bandwidth memory and the expansion card device to obtain a data subset suitable for parallel computing and a corresponding computing instruction set; According to the data subset and the computing instruction set, the computing units of the graphics processor are parallelly scheduled to obtain parallel computing results that are simultaneously executed on the high-bandwidth memory and the expansion card device; Performing data distribution analysis on the parallel computing results to obtain an optimal allocation scheme for the computing result data in the shared storage space of the high-bandwidth memory and the expansion card device; According to the optimal allocation scheme, the calculation result data is selectively copied to achieve the expansion of the high bandwidth memory.

[0009] Optionally, in a sixth implementation of the first aspect of the present invention, performing task division processing on the high bandwidth memory and the shared data set on the expansion card device to obtain a data subset suitable for parallel computing and a corresponding computing instruction set includes: Analyze and process the storage characteristics of high-bandwidth memory and expansion card devices to obtain a storage performance model that includes access delay and bandwidth parameters; According to the storage performance model, data dependency analysis is performed on the shared data set to obtain data blocks that can be executed in parallel and corresponding computing task graphs; Based on the data blocks and the computing task graph, the computing tasks are optimized and allocated to obtain data subsets and corresponding computing instruction sets that are evenly distributed on the high-bandwidth memory and expansion card device.

[0010] A second aspect of the present invention provides a high-bandwidth memory expansion device, which is applied to a graphics processor, wherein the graphics processor is integrated with a high-bandwidth memory, and the high-bandwidth memory expansion device comprises: A connection module, used to connect the expansion card device to the graphics processor through a PCIe interface on the graphics processor, so as to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory; A declaration module, used to perform declaration processing on the shared storage space according to a preset shared keyword in a kernel function of the graphics processor to obtain an extended declaration of the expansion card device; A copy synchronization module, used in the kernel function to copy the memory data of the high bandwidth memory to the shared storage space of the expansion card device through the threads in the preset thread block according to the extension declaration, and to perform thread synchronization on the thread block through a preset synchronization function; The computing module is used to perform parallel computing processing on the shared data set on the high-bandwidth memory and the expansion card device to obtain computing result data, and distribute and copy the computing result data between the shared storage space of the high-bandwidth memory and the expansion card device to achieve the expansion of the high-bandwidth memory.

[0011] A third aspect of the present invention provides a high-bandwidth memory expansion device, comprising: a memory and at least one processor, wherein instructions are stored in the memory, and the memory and the at least one processor are interconnected via a line; and the at least one processor calls the instructions in the memory so that the high-bandwidth memory expansion device executes the steps of the above-mentioned high-bandwidth memory expansion method.

[0012] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the above-mentioned high-bandwidth memory expansion method.

[0013] The above-mentioned high-bandwidth memory expansion method, device, equipment and storage medium connect the expansion card device to the graphics processor through the PCIe interface on the graphics processor, so that the storage space of the expansion card device is used as the shared storage space of the high-bandwidth memory (HBM). First, in the kernel function of the graphics processor, the shared storage space is declared using the shared keyword to generate an extension declaration. Then, the HBM memory data is copied to the shared storage space of the expansion card device through the threads in the thread block, and the thread block is synchronized using the synchronization function. On this basis, parallel computing is performed, and the calculation results are distributed and copied between the shared storage space of the HBM and the expansion card device to achieve storage expansion. The present invention can effectively expand the capacity of HBM, meet the processing requirements of big data and large models, improve GPU computing efficiency, and reduce expansion costs.

[0014] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0015] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of a first embodiment of a high bandwidth memory expansion method according to an embodiment of the present invention; Figure 2 A schematic diagram of an embodiment of a high-bandwidth memory expansion device according to an embodiment of the present invention; Figure 3 It is a schematic diagram of an embodiment of a high bandwidth memory expansion device in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0019] To facilitate understanding of this embodiment, a high-bandwidth memory expansion method disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method is applied to a graphics processor, and the graphics processor is integrated with a high bandwidth memory, and comprises the following steps: 101. Connecting the expansion card device to the graphics processor through a PCIe interface on the graphics processor, so that the storage space in the expansion card device is used as a shared storage space of the high bandwidth memory; In one embodiment of the present invention, the step of connecting the expansion card device to the graphics processor through the PCIe interface on the graphics processor so as to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory comprises: connecting the expansion card device to the graphics processor through the PCIe interface on the graphics processor, and initializing the root complex of the graphics processor to obtain a system topology structure connecting the CPU, the memory subsystem and the PCIe device; enumerating the PCIe bus of the PCIe interface according to the system topology structure to obtain a function list including different bus number and device number combinations; reading the configuration space of the expansion card device based on the function list to obtain the identity information and hardware characteristics of the expansion card device; configuring the base address register of the expansion card device based on the identity information and the hardware characteristics to obtain the mapping position of the expansion card device in the memory address space of the graphics processor; and updating the memory mapping I / O system of the graphics processor based on the mapping position to obtain an extended memory access mechanism including the storage space of the expansion card device so as to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory.

[0020] The expansion card device is connected to the graphics processor through the PCIe interface on the graphics processor, and the root complex of the graphics processor is initialized to obtain the system topology connecting the CPU, memory subsystem and PCIe devices. First, it involves hardware connection. The expansion card device is physically inserted into the PCIe slot of the graphics processor. Subsequently, when the system starts, the BIOS or UEFI firmware will identify the newly inserted PCIe device and allocate basic resources to it. When the operating system is loaded, the driver of the graphics processor is activated and starts to initialize the root complex. The root complex is a core component in the PCIe architecture and is responsible for connecting the CPU and PCIe devices. During the initialization process, the driver scans all connected PCIe devices, including the newly inserted expansion card device. It reads the configuration space of each device to obtain information such as device type, function and resource requirements. Based on this information, the driver builds a detailed system topology that describes the physical and logical connection relationship between the CPU, memory subsystem and all PCIe devices. This topology is the basis for subsequent resource allocation and device management, ensuring that the system can correctly identify and utilize the expansion card device.

[0021] Specifically, according to the system topology, the PCIe bus of the PCIe interface is enumerated to obtain a function list containing different bus number and device number combinations. PCIe bus enumeration is a system-level operation, usually performed by an operating system or a dedicated device driver. The enumeration process starts from the root complex and probes down along the PCIe bus structure level by level. For each possible bus number and device number combination, the system attempts to read the device's configuration space, especially the vendor ID (VID) and device ID (DID). If the read is successful, it indicates that there is a valid PCIe device at that location. The system records the bus number, device number, and function number (BDF) of this device, as well as other key information such as device type, interrupt line, etc. During the detection process, the system also identifies PCIe bridges, which connect different PCIe bus segments. By recursively probing these bus segments, the system can discover all connected PCIe devices, including devices deeply nested in complex topologies. Finally, the system generates a comprehensive function list containing all discovered PCIe devices and their detailed information. This list is an important basis for subsequent device configuration and resource allocation.

[0022] Specifically, based on the function list, the configuration space of the expansion card device is read and processed to obtain the identity information and hardware characteristics of the expansion card device. The configuration space is a standardized memory area of ​​the PCIe device, containing 256 bytes (for traditional devices) or 4096 bytes (for extended capability devices). The reading process starts with the standard configuration header, which contains basic information about the device. The system first reads the manufacturer ID and device ID, which uniquely identify the manufacturer and specific model of the device. Next, the system reads the category code to determine the functional type of the device (such as graphics card, network card, etc.). Then, the system reads the status register of the device to understand the current status of the device and the supported functions. For expansion card devices, it is particularly important to read their memory requirement information, which is usually stored in the base address register (BAR). The system also reads interrupt line information, maximum load size, power management capabilities of the device, etc. If the device supports extended functions, the system will further read the extended capability pointer and traverse all extended capability structures to obtain more advanced feature information. This comprehensive reading process ensures that the system obtains all the key information required to configure and use the expansion card device.

[0023] Specifically, according to the identification information and hardware characteristics, the base address register of the expansion card device is configured and processed to obtain the mapping location of the expansion card device in the memory address space of the graphics processor. This process first involves analyzing the memory requirements of the expansion card device. The system checks each base address register (BAR) of the device to determine the required memory space type (memory mapping or I / O mapping) and size. For the memory mapping type, the system also needs to consider factors such as whether prefetch support is required and whether 64-bit addressing is required. Then, the system searches for free areas in the global memory address space of the graphics processor that meet these requirements. This search process needs to consider alignment requirements and possible address conflicts. After finding a suitable address range, the system writes the starting address of this range to the corresponding BAR of the device. If the device has multiple BARs, this process will be repeated multiple times. For 64-bit BARs, the system needs to configure two consecutive 32-bit registers at the same time. After the configuration is completed, the system will read the BAR again to verify whether the configuration is successful. If the verification passes, the system will update the internal data structure to record this newly allocated address mapping. This configuration process essentially establishes a mapping relationship between the physical memory of the expansion card device and the virtual address space accessible to the graphics processor.

[0024] Specifically, based on the mapping location, the memory mapping I / O system of the graphics processor is updated to obtain an extended memory access mechanism that includes the storage space of the expansion card device. This step mainly involves updating the memory management unit (MMU) of the graphics processor and related software data structures. First, the system needs to add the memory mapping information of the expansion card device to the page table of the graphics processor. This includes creating a new page table entry to map the virtual address range to the physical address of the expansion card device. At the same time, the system may need to adjust the configuration of the MMU, such as updating the TLB (translation lookaside buffer). Next, the system updates the memory management-related code in the device driver so that it can recognize and use these newly mapped memory areas. This may involve modifying the memory allocator, updating the DMA controller configuration, etc. The system also needs to establish an interrupt handling mechanism to ensure that the expansion card device can correctly trigger and handle interrupts. In addition, the system may need to implement a memory barrier or cache consistency mechanism to ensure data consistency between the graphics processor and the expansion card device. Finally, the system may need to update the power management and thermal management strategies to adapt to the addition of the expansion card device. Through these updates, the storage space of the expansion card device is successfully integrated into the memory system of the graphics processor, forming a seamless extended memory access mechanism.

[0025] 102. In a kernel function of a graphics processor, a shared storage space is declared using a preset shared keyword to obtain an extended declaration of an expansion card device; In one embodiment of the present invention, the use of a preset shared keyword in the kernel function of the graphics processor to declare the shared storage space to obtain the extended declaration of the expansion card device includes: preprocessing the kernel function code of the graphics processor to obtain a code segment of the shared keyword, and inserting an identifier of the expansion card device into the code segment to obtain a shared memory declaration statement of the extended memory; performing syntax parsing on the shared memory declaration statement to obtain a parameter list including the storage space size of the expansion card device; detecting the firmware version of the expansion card device according to the parameter list, and conditionally compiling the extended shared memory declaration statement based on the firmware version to obtain the extended declaration of the expansion card device adapted to the firmware version.

[0026] Specifically, the kernel function code of the graphics processor is preprocessed to obtain a code segment with a shared keyword, and the identifier of the expansion card device is inserted into the code segment to obtain a shared memory declaration statement for the extended memory. First, the lexical and grammatical analysis of the CUDA kernel function code is involved. The preprocessor scans the entire kernel function code and identifies all code segments using the __shared__ keyword. These code segments usually represent statements that declare variables in shared memory. Next, the preprocessor inserts a specific identifier of the expansion card device into these identified code segments. This identifier may be a predefined macro or a special comment used to mark that this part of the shared memory declaration is for the expansion card device. The insertion process needs to consider grammatical correctness to ensure that the inserted code still complies with the CUDA syntax specification. In addition, the preprocessor also needs to process possible conditional compilation instructions, such as #ifdef or #if, to ensure that the declaration of the expansion card device can be correctly processed under different compilation conditions. Finally, this step will produce a modified code segment that contains the original shared memory declaration and the newly added declaration related to the expansion card device.

[0027] Specifically, parsing the shared memory declaration statement to obtain a parameter list containing the storage space size of the expansion card device involves a deeper level of grammatical analysis. The parser will analyze the modified shared memory declaration statement in detail to extract key information. First, the parser needs to identify the declared variable type, variable name, and array size (if applicable). For the declaration of the expansion card device, the parser needs to pay special attention to the inserted identifier and extract relevant additional information based on this identifier. This may include the model of the expansion card device, the expected storage space size, etc. During the parsing process, it is also necessary to process CUDA-specific syntax elements such as possible template parameters and const qualifiers. The parser will build an abstract syntax tree (AST) to represent the structure and relationship of these declarations. From the AST, the parser extracts a parameter list containing detailed information about each shared memory declaration, especially the storage space size of the expansion card device. This parameter list contains not only the statically declared size, but also the dynamically calculated size information, which is crucial for subsequent memory allocation and management.

[0028] 103. In the kernel function, according to the extension declaration, the memory data of the high bandwidth memory is copied to the shared storage space of the expansion card device through the threads in the preset thread block, and the thread block is synchronized through the preset synchronization function; In one embodiment of the present invention, in the kernel function, according to the extended declaration, the memory data of the high bandwidth memory is copied to the shared storage space of the expansion card device through the threads in the preset thread block, and the thread block is thread-synchronized through the preset synchronization function, including: in the kernel function, according to the extended declaration, determining the thread block index and the thread index in the preset thread block; according to the thread block index and the thread index, determining the global memory access address corresponding to each thread; according to the global memory access address, by the threads in the thread block, copying the memory data in the high bandwidth memory to the shared storage space of the expansion card device, and thread-synchronizing the thread block through the preset synchronization function.

[0029] Specifically, in the kernel function, the thread block index and thread index in the preset thread block are determined according to the extension declaration. This process first involves parsing the thread block and thread structure defined in the extension declaration. In the CUDA programming model, each thread has a unique identifier, which consists of a thread block index and a thread index. The thread block index is obtained through the built-in variable blockIdx, which is a three-dimensional vector containing three components x, y, and z, indicating the position of the current thread block in the entire grid. The thread index is obtained through the built-in variable threadIdx, which is also a three-dimensional vector, indicating the position of the thread in the thread block to which it belongs. The kernel function calculates the global unique identifier of each thread through these built-in variables. In addition, the kernel function also needs to obtain the dimensions of the thread block (through the blockDim variable) and the dimensions of the grid (through the gridDim variable), which are used for subsequent address calculations. In actual implementation, the kernel function may store this index information in local variables for subsequent use. The key to this step is to ensure that each thread can correctly identify its own position in preparation for subsequent data copy operations.

[0030] Specifically, the global memory access address corresponding to each thread is determined based on the thread block index and thread index. This step involves mapping the logical index of the thread to the actual memory address. The calculation usually follows a certain pattern, the most common one-dimensional mapping: converting the three-dimensional thread and block index to a one-dimensional global thread ID. This can be achieved by the formula: globalThreadId = blockIdx.x * blockDim.x + threadIdx.x (for the one-dimensional case). For multi-dimensional cases, the calculation will be more complicated and the y and z dimensions need to be considered. After obtaining the global thread ID, it is mapped to the actual memory address. This mapping relationship depends on how the data is stored in global memory and the access pattern. For example, if the data is stored linearly, the memory address can be simply calculated by: address = baseAddress +globalThreadId * elementSize, where baseAddress is the starting address of the data in global memory and elementSize is the size of each data element. In some complex cases, more complex addressing algorithms may be required, especially when the data layout involves padding or alignment requirements. The goal of this step is to ensure that each thread can accurately calculate the global memory address it needs to access, preparing for the next step of data copying.

[0031] Finally, the threads in the thread block copy the memory data in the high bandwidth memory to the shared memory space of the expansion card device according to the global memory access address, and synchronize the threads of the thread block through the preset synchronization function. This process first involves the actual copying of the data. Each thread reads data from the high bandwidth memory using the previously calculated global memory address. The read operation usually uses the global memory load instruction, such as the ld.global instruction used in CUDA. The read data is then written to the shared memory space of the expansion card device. The write operation needs to consider the bank conflict problem of the shared memory, and a reasonable data layout can significantly improve the access efficiency. After the data is copied, the thread block needs to be synchronized using the preset synchronization function. In CUDA, this is usually implemented by calling the __syncthreads() function. This synchronization operation ensures that all threads in the thread block have completed the data copy before the next operation can be performed. Synchronization is necessary because different threads may execute at different speeds, and subsequent calculations may rely on all data being correctly loaded into the shared memory. In addition, synchronization also ensures memory consistency and prevents data competition problems.

[0032] Furthermore, the thread synchronization of the thread block through a preset synchronization function includes: marking the execution status of each thread in the thread block through a preset synchronization function to obtain a status identifier of the thread execution to the synchronization point; updating the thread counter in the thread block according to the status identifier to obtain the number of threads that have reached the synchronization point; judging the number of threads that have reached the synchronization point to obtain a judgment result of whether all threads in the thread block have reached the synchronization point and performing thread synchronization on the thread block according to the judgment result.

[0033] Specifically, the execution status of each thread in the thread block is marked by a preset synchronization function to obtain the status mark of the thread execution to the synchronization point. In the CUDA programming model, this is usually implemented by the __syncthreads() function. When the thread executes to the __syncthreads() call, the hardware-level synchronization mechanism is triggered. In the specific implementation, each thread sets its own status to "reached the synchronization point". This status mark is usually stored in a special hardware register shared by the thread block, with one bit for each thread. When the thread executes to __syncthreads(), the corresponding position is set to 1. This operation is atomic, ensuring correctness in a multi-threaded environment. At the same time, the hardware records the address of the last instruction executed by each thread, which is used as the synchronization point mark. If some threads in the thread block do not execute to __syncthreads() due to conditional branches, their corresponding status bits will not be set, which will be detected in subsequent steps.

[0034] Specifically, according to the status flag, the thread counter in the thread block is updated to obtain the number of threads that have reached the synchronization point. This step is performed automatically at the hardware level. CUDA's thread block scheduler maintains a counter to track the number of threads that have reached the synchronization point. Whenever a thread's status is marked as "reached the synchronization point", this counter will be automatically increased. The increase operation is atomic, ensuring accuracy in high concurrency situations. The initial value of the counter is 0, and the maximum value is equal to the total number of threads in the thread block. This counting process is efficient because it is implemented at the hardware level and does not require additional overhead at the software level. At the same time, the hardware also records the order in which each thread reaches the synchronization point, which is very important in some scenarios where the execution order needs to be guaranteed. If there are threads in the thread block that have not executed to the synchronization point, the value of the counter will not reach the total number of threads in the thread block, which will be detected in the next step.

[0035] Specifically, the number of threads that have reached the synchronization point is judged and processed to obtain the judgment result of whether all threads in the thread block have reached the synchronization point, and the thread block is synchronized according to the judgment result. This step is also automatically performed at the hardware level. CUDA's thread block scheduler will continuously monitor the thread counter that has reached the synchronization point. When the value of the counter is equal to the total number of threads in the thread block, it means that all threads have reached the synchronization point. At this time, the hardware will trigger an internal signal to indicate that the synchronization is complete. If the counter does not reach the total number of threads within the predetermined time, the hardware will think that a deadlock or other error has occurred, and an exception may be triggered. Once it is confirmed that all threads have reached the synchronization point, the hardware will reset the execution status of all threads and allow them to continue to execute subsequent instructions. This process is atomic, ensuring that all threads resume execution at exactly the same time. At the same time, the hardware will also refresh the cache to ensure that the shared memory state seen by all threads is consistent. This hardware-level synchronization mechanism greatly improves efficiency and minimizes the overhead of thread synchronization.

[0036] 104. Perform parallel computing processing on the shared data set on the high-bandwidth memory and the expansion card device to obtain computing result data, and distribute and copy the computing result data between the shared storage space of the high-bandwidth memory and the expansion card device to achieve capacity expansion of the high-bandwidth memory.

[0037] In one embodiment of the present invention, the method of performing parallel computing processing on the shared data set on the high bandwidth memory and the expansion card device to obtain computing result data, and distributing and copying the computing result data between the shared storage space of the high bandwidth memory and the expansion card device to achieve the expansion of the high bandwidth memory includes: performing task division processing on the shared data set on the high bandwidth memory and the expansion card device to obtain a data subset suitable for parallel computing and a corresponding computing instruction set; performing parallel scheduling processing on the computing unit of the graphics processor according to the data subset and the computing instruction set to obtain parallel computing results executed simultaneously on the high bandwidth memory and the expansion card device; performing data distribution analysis processing on the parallel computing results to obtain an optimal allocation scheme of the computing result data in the shared storage space of the high bandwidth memory and the expansion card device; and performing selective copying processing on the computing result data according to the optimal allocation scheme to achieve the expansion of the high bandwidth memory.

[0038] Specifically, the shared data sets on the high-bandwidth memory and the expansion card device are divided into tasks to obtain data subsets suitable for parallel computing and corresponding computing instruction sets. This process first involves analyzing the structure and size of the shared data set. The system evaluates the distribution characteristics of the data, such as the continuity, access pattern and dependency of the data. Based on this information, the system adopts a specific task division algorithm, such as block division, loop division or dynamic division. For the data in the high-bandwidth memory, the system considers its high-speed access characteristics and tends to assign computing-intensive tasks to this part of the data. For the data on the expansion card device, the system considers its lower access speed and is more suitable for processing IO-intensive tasks. The division process also needs to consider load balancing to ensure that each computing unit can get the right amount of work. At the same time, the system generates corresponding computing instruction sets, which contain detailed steps on how to process each data subset. The generation of the instruction set needs to consider the location of the data to minimize data movement. Finally, a series of optimized data subsets and corresponding computing instruction sets are generated.

[0039] Specifically, according to the data subsets and computing instruction sets, the computing units of the graphics processor are scheduled in parallel to obtain parallel computing results that are executed simultaneously on the high-bandwidth memory and the expansion card device. This step first involves the work of the task scheduler. The scheduler analyzes each data subset and the corresponding computing instruction set to evaluate its computing complexity and resource requirements. Based on this information, the scheduler will formulate an execution plan to determine which tasks are assigned to which computing units. For the data subsets in the high-bandwidth memory, the scheduler will give priority to assigning them to the stream processor on the GPU to utilize its high-speed data access capability. For the data subsets on the expansion card device, the scheduler may choose to assign tasks to dedicated asynchronous computing units to cover up data access delays. The scheduling process also needs to consider the dependencies between tasks to ensure data consistency. During execution, each computing unit will process the tasks assigned to them in parallel, and the system will monitor the execution progress and perform dynamic load balancing when necessary. During the calculation process, the system will use CUDA's stream and event mechanisms to manage concurrent execution and synchronization. Finally, this step produces parallel computing results that are executed simultaneously on the high-bandwidth memory and the expansion card device, making full use of the computing resources of the entire system.

[0040] Specifically, the data distribution analysis and processing of the parallel computing results is performed to obtain the optimal allocation scheme of the computing result data in the shared storage space of the high-bandwidth memory and the expansion card device. This process first involves collecting and analyzing the characteristics of the computing results. The system will evaluate factors such as the size, access frequency, and data relevance of the result data. For frequently accessed hot data, the system tends to allocate it to the high-bandwidth memory to improve the efficiency of subsequent operations. For cold data with low access frequency, it may be allocated to the storage space of the expansion card device. The analysis process will also consider the life cycle of the data, and data that needs to be frequently used in the short term will be preferentially retained in the high-bandwidth memory. At the same time, the system will evaluate the current storage capacity and usage of the high-bandwidth memory and the expansion card device to ensure that no storage device is overloaded. In addition, the analysis will also consider the correlation between the data, and try to allocate related data to the same storage device to reduce cross-device data access. Based on these analysis results, the system will use specific optimization algorithms, such as greedy algorithms or dynamic programming, to generate an optimal data allocation scheme. This scheme will specify in detail which device and location each piece of computing result data should be stored in to achieve the best overall performance.

[0041] Specifically, according to the optimal allocation scheme, the calculation result data is selectively copied to achieve the expansion of the high-bandwidth memory. This step first involves the planning of data movement operations. The system will determine which data needs to be moved from the original location to the new location based on the optimal allocation scheme. For data that needs to be moved from the high-bandwidth memory to the expansion card device, the system will use DMA (direct memory access) technology for efficient transmission and minimize CPU intervention. Conversely, for data that needs to be moved from the expansion card device to the high-bandwidth memory, the system will use the high-bandwidth characteristics of the PCIe bus to optimize the transmission path. During the data movement process, the system will use pipeline technology to perform data transmission and calculation at the same time to cover up the delay of data movement. For data that does not need to be moved, the system will update the memory mapping table to ensure that subsequent access can correctly locate the data. In addition, the system will maintain a global data location table to record the current location of each piece of data for subsequent fast access. Through this selective data replication and redistribution, the system effectively expands the capacity of the high-bandwidth memory, enabling large-scale data sets to be efficiently managed and accessed between limited high-bandwidth memory and expansion card devices, thereby achieving logical expansion of the high-bandwidth memory.

[0042] Furthermore, the task division processing of the shared data set on the high-bandwidth memory and the expansion card device to obtain a data subset suitable for parallel computing and a corresponding computing instruction set includes: analyzing and processing the storage characteristics of the high-bandwidth memory and the expansion card device to obtain a storage performance model including access delay and bandwidth parameters; performing data dependency analysis and processing on the shared data set according to the storage performance model to obtain data blocks that can be executed in parallel and a corresponding computing task graph; and optimizing and allocating computing tasks based on the data blocks and the computing task graph to obtain a data subset and a corresponding computing instruction set that are evenly distributed on the high-bandwidth memory and the expansion card device.

[0043] Specifically, the storage characteristics of high-bandwidth memory and expansion card devices are analyzed and processed to obtain a storage performance model that includes access latency and bandwidth parameters. This process first involves measuring the performance characteristics of high-bandwidth memory (HBM). The system evaluates the read and write latency, peak bandwidth, and sustained bandwidth of HBM through a series of micro-benchmarks. These tests include sequential read and write, random read and write, and access patterns of different data sizes. At the same time, the system also evaluates the capacity and cache hierarchy of HBM. For expansion card devices, the system also performs a series of performance tests, focusing on their data transfer characteristics through the PCIe interface. This includes measuring the effective bandwidth, latency, and transmission efficiency of the PCIe interface under different data sizes. In addition, the system also evaluates the internal storage characteristics of the expansion card device, such as the access speed and capacity of its internal memory. Based on these test results, the system builds a comprehensive storage performance model that includes not only static latency and bandwidth parameters, but also dynamic performance characteristics under different loads. This model will provide key decision-making basis for subsequent task division and data allocation.

[0044] Specifically, according to the storage performance model, the shared data set is analyzed for data dependency, and the data blocks that can be executed in parallel and the corresponding computing task graph are obtained. This step first involves an in-depth analysis of the structure and access pattern of the shared data set. The system builds a data dependency graph to represent the read and write relationship between data elements. This process uses static code analysis technology to identify data access patterns and potential parallel opportunities. At the same time, the system considers the latency and bandwidth parameters in the storage performance model to evaluate the performance impact of different data access strategies. Based on this information, the system uses a specific data partitioning algorithm, such as block partitioning or loop partitioning, to divide the shared data set into multiple data blocks that can be processed in parallel. The size and boundary of each data block are optimized according to the characteristics of the storage device to maximize data access efficiency. In addition, the system generates a computing task graph that describes the processing order and dependency between these data blocks. This task graph not only considers data dependencies, but also considers computational complexity and storage access patterns to ensure that computing resources can be fully utilized in subsequent parallel execution while minimizing data movement overhead.

[0045] Specifically, based on the data blocks and computing task graphs, the computing tasks are optimized and allocated to obtain data subsets and corresponding computing instruction sets that are evenly distributed on the high-bandwidth memory and expansion card devices. This process first involves evaluating the computing requirements and storage characteristics of each data block. The system considers the size, access frequency, computing complexity, and dependencies of the data blocks with other data blocks. At the same time, the system refers to the previously constructed storage performance model to weigh the high-speed access characteristics of the high-bandwidth memory and the large capacity advantages of the expansion card device. Based on these factors, the system uses heuristic algorithms or dynamic programming methods to optimize task allocation. For data blocks that are computationally intensive and need to be accessed frequently, the system tends to allocate them to the high-bandwidth memory; while for data blocks that are IO-intensive or have a low access frequency, they may be allocated to the expansion card device. The system also considers load balancing to ensure that both storage devices are fully utilized. At the same time, the system generates corresponding computing instruction sets, which contain detailed steps on how to process each data subset, including data loading, computing operations, and result storage. The generation of instruction sets takes into account data location, optimizes data movement paths, and utilizes advanced features of CUDA such as shared memory and warp-level operations to improve efficiency.

[0046] In this embodiment, the expansion card device is connected to the graphics processor through the PCIe interface on the graphics processor, so that the storage space of the expansion card device is used as the shared storage space of the high bandwidth memory (HBM). First, in the kernel function of the graphics processor, the shared storage space is declared using the shared keyword to generate an extension declaration. Then, the HBM memory data is copied to the shared storage space of the expansion card device through the threads in the thread block, and the thread block is synchronized using the synchronization function. On this basis, parallel computing is performed, and the calculation results are distributed and copied between the shared storage space of the HBM and the expansion card device to achieve storage expansion. The present invention can effectively expand the capacity of HBM, meet the processing requirements of big data and large models, improve GPU computing efficiency, and reduce expansion costs.

[0047] The above describes the high bandwidth memory expansion method in the embodiment of the present invention. The following describes the high bandwidth memory expansion device in the embodiment of the present invention. The high bandwidth memory expansion device is applied to a graphics processor, and the graphics processor is integrated with a high bandwidth memory. Figure 2 , an embodiment of a high bandwidth memory expansion device in an embodiment of the present invention includes: A connection module 201 is used to connect the expansion card device to the graphics processor through the PCIe interface on the graphics processor, so as to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory; A declaration module 202, configured to perform declaration processing on the shared storage space using a preset shared keyword in a kernel function of the graphics processor to obtain an extended declaration of the expansion card device; A copy synchronization module 203 is used to copy the memory data of the high bandwidth memory to the shared storage space of the expansion card device through the threads in the preset thread block according to the extension declaration in the kernel function, and perform thread synchronization on the thread block through a preset synchronization function; The computing module 204 is used to perform parallel computing processing on the shared data set on the high-bandwidth memory and the expansion card device to obtain computing result data, and distribute and copy the computing result data between the shared storage space of the high-bandwidth memory and the expansion card device to achieve capacity expansion of the high-bandwidth memory.

[0048] In an embodiment of the present invention, the high-bandwidth memory expansion device runs the above-mentioned high-bandwidth memory expansion method, and the high-bandwidth memory expansion device connects the expansion card device to the graphics processor through the PCIe interface on the graphics processor, so that the storage space of the expansion card device is used as the shared storage space of the high-bandwidth memory (HBM). First, in the kernel function of the graphics processor, the shared storage space is declared using the shared keyword to generate an extension declaration. Then, the HBM memory data is copied to the shared storage space of the expansion card device through the threads in the thread block, and the thread block is synchronized using the synchronization function. On this basis, parallel computing is performed, and the calculation results are distributed and copied between the shared storage space of the HBM and the expansion card device to achieve storage expansion. The present invention can effectively expand the capacity of HBM, meet the processing requirements of big data and large models, improve GPU computing efficiency, and reduce expansion costs.

[0049] above Figure 2 The medium and high bandwidth memory expansion device in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The high bandwidth memory expansion device in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0050] Figure 3 3 is a schematic diagram of the structure of a high-bandwidth memory expansion device provided by an embodiment of the present invention. The high-bandwidth memory expansion device 300 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage device terminals) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 may be short-term storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the high-bandwidth memory expansion device 300. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, and execute a series of instruction operations in the storage medium 330 on the high-bandwidth memory expansion device 300 to implement the steps of the above-mentioned high-bandwidth memory expansion method.

[0051] The high bandwidth memory expansion device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3The illustrated high bandwidth memory expansion device structure does not constitute a limitation on the high bandwidth memory expansion device provided by the present invention, and may include more or fewer components than illustrated, or a combination of certain components, or a different arrangement of components.

[0052] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the high-bandwidth memory expansion method.

[0053] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0054] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0055] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high bandwidth memory expansion method, characterized in that: Applied to a graphics processor, the graphics processor is integrated with a high-bandwidth memory, and the high-bandwidth memory expansion method includes: Connecting the expansion card device to the graphics processor via a PCIe interface on the graphics processor, so that the storage space in the expansion card device is used as the shared storage space of the high bandwidth memory; In the kernel function of the graphics processor, a declaration process is performed on the shared storage space according to a preset shared keyword to obtain an extended declaration of the expansion card device; In the kernel function, according to the extension declaration, the memory data of the high bandwidth memory is copied to the shared storage space of the expansion card device through the threads in the preset thread block, and the thread block is synchronized through the preset synchronization function; Parallel computing is performed on the shared data set on the high bandwidth memory and the expansion card device to obtain computing result data, and the computing result data is distributed and copied between the shared storage space of the high bandwidth memory and the expansion card device to achieve capacity expansion of the high bandwidth memory.

2. The high bandwidth memory expansion method according to claim 1, characterized in that: The method of connecting the expansion card device to the graphics processor through the PCIe interface on the graphics processor to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory includes: Connecting the expansion card device to the graphics processor through the PCIe interface on the graphics processor, and initializing the root complex of the graphics processor to obtain a system topology structure connecting the CPU, the memory subsystem and the PCIe device; According to the system topology, enumerate the PCIe bus of the PCIe interface to obtain a function list including different combinations of bus numbers and device numbers; Based on the function list, the configuration space of the expansion card device is read and processed to obtain the identity information and hardware characteristics of the expansion card device; According to the identity information and hardware characteristics, a base address register of the expansion card device is configured to obtain a mapping position of the expansion card device in the memory address space of the graphics processor; Based on the mapping position, the memory mapping I / O system of the graphics processor is updated to obtain an extended memory access mechanism including the storage space of the expansion card device, so as to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory.

3. The high bandwidth memory expansion method according to claim 1, characterized in that: The process of declaring the shared storage space using the preset shared keyword in the kernel function of the graphics processor to obtain the extended declaration of the expansion card device includes: Preprocessing the kernel function code of the graphics processor to obtain a code segment of a shared keyword, and inserting an identifier of the expansion card device into the code segment to obtain a shared memory declaration statement of the extended memory; Performing syntax parsing on the shared memory declaration statement to obtain a parameter list including the storage space size of the expansion card device; According to the parameter list, the firmware version of the expansion card device is detected, and based on the firmware version, the extended shared memory declaration statement is conditionally compiled to obtain an extended declaration of the expansion card device adapted to the firmware version.

4. The high bandwidth memory expansion method according to claim 1, characterized in that: In the kernel function, copying the memory data of the high bandwidth memory to the shared storage space of the expansion card device through the threads in the preset thread block according to the extension declaration, and performing thread synchronization on the thread block through the preset synchronization function includes: In the kernel function, determining a thread block index and a thread index in a preset thread block according to the extended declaration; Determine the global memory access address corresponding to each thread according to the thread block index and the thread index; The memory data in the high bandwidth memory is copied to the shared storage space of the expansion card device through the threads in the thread block according to the global memory access address, and the thread block is synchronized through a preset synchronization function.

5. The high bandwidth memory expansion method according to claim 4, characterized in that: The thread block is synchronized by a preset synchronization function, comprising: The execution status of each thread in the thread block is marked by a preset synchronization function to obtain a status identifier of the thread execution reaching the synchronization point; According to the state identifier, updating the thread counter in the thread block to obtain the number of threads that have reached the synchronization point; The number of threads that have reached the synchronization point is judged to obtain a judgment result of whether all threads in the thread block have reached the synchronization point, and the thread block is synchronized according to the judgment result.

6. The high bandwidth memory expansion method according to claim 1, characterized in that: The step of performing parallel computing on the shared data set on the high bandwidth memory and the expansion card device to obtain computing result data, and distributing and copying the computing result data between the high bandwidth memory and the shared storage space of the expansion card device to achieve the expansion of the high bandwidth memory includes: Performing task division processing on the shared data set on the high bandwidth memory and the expansion card device to obtain a data subset suitable for parallel computing and a corresponding computing instruction set; According to the data subset and the computing instruction set, the computing units of the graphics processor are parallelly scheduled to obtain parallel computing results that are simultaneously executed on the high-bandwidth memory and the expansion card device; Performing data distribution analysis on the parallel computing results to obtain an optimal allocation scheme for the computing result data in the shared storage space of the high-bandwidth memory and the expansion card device; According to the optimal allocation scheme, the calculation result data is selectively copied to achieve the expansion of the high bandwidth memory.

7. The high bandwidth memory expansion method according to claim 6, characterized in that: The task division processing of the shared data set on the high bandwidth memory and the expansion card device to obtain a data subset suitable for parallel computing and a corresponding computing instruction set includes: Analyze and process the storage characteristics of high-bandwidth memory and expansion card devices to obtain a storage performance model that includes access delay and bandwidth parameters; According to the storage performance model, data dependency analysis is performed on the shared data set to obtain data blocks that can be executed in parallel and corresponding computing task graphs; Based on the data blocks and the computing task graph, the computing tasks are optimized and allocated to obtain data subsets and corresponding computing instruction sets that are evenly distributed on the high-bandwidth memory and expansion card device.

8. A high bandwidth memory expansion device, characterized in that: Applied to a graphics processor, the graphics processor is integrated with a high-bandwidth memory, and the high-bandwidth memory expansion device comprises: A connection module, used to connect the expansion card device to the graphics processor through a PCIe interface on the graphics processor, so as to use the storage space in the expansion card device as the shared storage space of the high bandwidth memory; A declaration module, used to perform declaration processing on the shared storage space according to a preset shared keyword in a kernel function of the graphics processor to obtain an extended declaration of the expansion card device; A copy synchronization module, used in the kernel function to copy the memory data of the high bandwidth memory to the shared storage space of the expansion card device through the threads in the preset thread block according to the extension declaration, and to perform thread synchronization on the thread block through a preset synchronization function; The computing module is used to perform parallel computing processing on the shared data set on the high-bandwidth memory and the expansion card device to obtain computing result data, and distribute and copy the computing result data between the shared storage space of the high-bandwidth memory and the expansion card device to achieve the expansion of the high-bandwidth memory.

9. A high bandwidth memory expansion device, characterized in that: The high bandwidth memory expansion device comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory to enable the high bandwidth memory expansion device to perform the steps of the high bandwidth memory expansion method according to any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the high bandwidth memory expansion method as described in any one of claims 1 to 7 are implemented.

Images