Method for optimizing GPU operation based on CUDA (Compute Unified Device Architecture) shared storage mechanism and related equipment

Abstract

The invention provides a method and related equipment for optimizing GPU (Graphics Processing Unit) operation based on a CUDA (Compute Unified Device Architecture) shared storage mechanism, and the method comprises the following steps: connecting expansion card equipment with a GPU through a PCIe (Peripheral Component Interconnect Express) interface on the GPU, and carrying out equipment identification and address space mapping processing on the GPU and the expansion card equipment to obtain storage space configuration information required by the GPU operation; performing space division and data channel establishment processing on the expansion card equipment according to the storage space configuration information to obtain a data transmission channel configuration result of GPU operation; and carrying out storage area scheduling and thread synchronization on the to-be-operated data of the GPU between the GPU and the expansion card equipment through a CUDA shared storage mechanism according to a data transmission channel configuration result until the operation process of the GPU is completed. According to the method, storage scheduling and thread synchronization of the data between the GPU and the expansion equipment are realized through recognition, space division and efficient establishment of the data channel on the GPU expansion equipment by a CUDA shared storage mechanism, so that the calculation efficiency is improved, and large-scale data processing requirements are better supported.

Description

Technical Field

[0001] The present invention relates to the field of GPU, and in particular to a method and related equipment for optimizing GPU computing based on a CUDA shared storage mechanism. Background Art

[0002] With the rapid development of big data and artificial intelligence technology, the application of GPU in the field of high-performance computing has been continuously expanded, especially in deep learning, scientific computing and large-scale data processing, which puts forward higher requirements on the computing performance and storage capacity of GPU. At present, the computing efficiency of GPU is limited by its own storage capacity and bandwidth. When the amount of processed data exceeds its memory capacity, the bottleneck of frequent data scheduling and storage transmission is obvious, resulting in a decrease in computing efficiency and failure to meet the needs of high-performance computing.

[0003] In practical applications, efficient computing of GPUs usually requires dynamic scheduling and transmission of data between different storage areas to ensure the parallelism and consistency of data processing. However, traditional storage expansion methods have bottlenecks in data transmission and thread synchronization, and cannot achieve efficient coordination during large-scale data processing, resulting in low utilization of storage resources and insufficient data throughput, which in turn limits the computing potential of GPUs. Summary of the invention

[0004] The main purpose of the present invention is to solve the technical problem that the existing GPU computing cannot achieve efficient coordination during large-scale data processing, resulting in low storage resource utilization and insufficient data throughput; A first aspect of the present invention provides a method for optimizing GPU computing based on a CUDA shared memory mechanism, the method for optimizing GPU computing based on a CUDA shared memory mechanism comprising: Connect the expansion card device to the GPU through the PCIe interface on the GPU, perform device identification and address space mapping on the GPU and the expansion card device, and obtain the storage space configuration information required for GPU computing; Performing space division and data channel establishment processing on the expansion card device according to the storage space configuration information to obtain a data transmission channel configuration result for GPU computing; The CUDA shared storage mechanism performs storage area scheduling and thread synchronization on the GPU's to-be-calculated data between the GPU and the expansion card device according to the data transmission channel configuration result until the GPU's calculation process is completed.

[0005] Optionally, in a first implementation of the first aspect of the present invention, the expansion card device is connected to the GPU through a PCIe interface on the GPU, and device identification and address space mapping processing are performed on the GPU and the expansion card device to obtain storage space configuration information required for GPU computing, including: Connect the expansion card device to the GPU through the PCIe interface on the GPU, read the VID / DID and scan the BDF structure of the GPU device and the expansion card device to obtain the device identification code; Perform an HBM firmware version check on the expansion card device according to the device identification code to obtain firmware information, and perform a memory space size check on the expansion card device according to the firmware information through a BAR register in a configuration space in a PCIe interface to obtain available storage space information; The CPU allocates an address range to the expansion card device according to the memory space information to obtain the storage space configuration information required for the GPU operation.

[0006] Optionally, in a second implementation of the first aspect of the present invention, performing space division and data channel establishment processing on the expansion card device according to the storage space configuration information to obtain a data transmission channel configuration result for GPU computing includes: Controlling the root complex through the CPU to divide the storage space of the expansion card device into a running capacity area and a cache area according to the storage space configuration information, thereby obtaining a storage area division result for GPU computing; A data transmission channel between the kernel function of the GPU and each storage area is established according to the storage area division result through the command register of the root complex, a data transmission channel configuration result is obtained, and the data transmission sequence of the kernel function is planned according to the data transmission channel configuration result.

[0007] Optionally, in a third implementation of the first aspect of the present invention, the step of performing storage area scheduling and thread synchronization between the GPU and the expansion card device for the data to be calculated of the GPU according to the data transmission channel configuration result through the CUDA shared storage mechanism until the calculation process of the GPU is completed includes: Declare the expansion card device through the shared keyword in the CUDA shared storage mechanism to obtain the expansion card access configuration information, and allocate data access to the storage area of ​​the GPU according to the data transmission channel configuration result through the expansion card access configuration information to obtain a data access plan; The data to be calculated of the GPU is stored and calculated by adopting a high-speed SLC data block calculation method through the data access scheme, and thread synchronization is performed to obtain the thread execution status; The data access frequency in the GPU operation process is detected by the root complex according to the thread execution state, and the distribution of the data to be operated between the GPU and the expansion card device is dynamically adjusted according to the detection result until the GPU operation process is completed.

[0008] Optionally, in a fourth implementation of the first aspect of the present invention, the expansion card device is declared by using a shared keyword in a CUDA shared storage mechanism to obtain expansion card access configuration information, and data access is allocated to a storage area of ​​a GPU according to the data transmission channel configuration result by using the expansion card access configuration information to obtain a data access plan; comprising: Preprocessing the kernel function code of the GPU through the CUDA shared storage mechanism to obtain a code segment with a shared keyword, and inserting an identifier of the expansion card device into the code segment to obtain a shared memory declaration statement of the extended memory; Performing syntax parsing on the shared memory declaration statement to obtain a parameter list including the storage space size of the expansion card device; According to the parameter list, the firmware version of the expansion card device is detected, and based on the firmware version, the shared memory declaration statement is conditionally compiled to obtain the expansion card access configuration information; The expansion card access configuration information and the data transmission channel configuration result are used to allocate data access to the storage area of ​​the GPU to obtain the data access plan.

[0009] Optionally, in a fifth implementation of the first aspect of the present invention, the data access scheme is used to perform data storage and calculation on the GPU's to-be-calculated data in a high-speed SLC data block calculation manner, and thread synchronization is performed to obtain the thread execution state, which includes: The data to be calculated by the GPU is grouped into data blocks by using the data access scheme to obtain an SLC data block group, and data storage allocation is performed between the GPU and the expansion card device according to the SLC data block group to obtain a data storage state; Performing parallel computing processing on the SLC data block through the data storage state to obtain a data block computing result, and executing a synchronization function on the threads in the GPU thread block according to the data block computing result to obtain thread synchronization information; The operation progress of each thread block of the GPU is detected through the thread synchronization information to obtain the thread running state, and the execution state of the thread block of the GPU is marked according to the thread running state to obtain the thread execution state.

[0010] Optionally, in a sixth implementation of the first aspect of the present invention, detecting the data access frequency in the GPU computing process according to the thread execution state through the root complex, and dynamically adjusting the distribution of the data to be computed between the GPU and the expansion card device according to the detection result until the computing process of the GPU is completed includes: Counting the accesses to the data to be calculated in the GPU calculation according to the thread execution state through the root complex to obtain a data access frequency table, and analyzing the access pattern of the data to be calculated according to the data access frequency table to obtain data access characteristics; Performing data distribution evaluation on the video memory space of the GPU and the storage space of the expansion card device through the data access characteristics to obtain a space load state, and calculating the distribution of the data to be operated between the running capacity area and the cache area according to the space load state to obtain a data migration plan; The data to be calculated is redistributed between the GPU and the expansion card device through the data migration scheme to obtain a new data distribution result, and the data access efficiency of the GPU calculation process is statistically analyzed according to the new data distribution result until the calculation process of the GPU is completed.

[0011] A second aspect of the present invention provides a device for optimizing GPU computing based on a CUDA shared memory mechanism, which is applied to a GPU. The device for optimizing GPU computing based on a CUDA shared memory mechanism includes: The connection mapping module is used to connect the expansion card device to the GPU through the PCIe interface on the GPU, and perform device identification and address space mapping processing on the GPU and the expansion card device to obtain the storage space configuration information required for GPU operation; A channel configuration module, used to perform space division and data channel establishment processing on the expansion card device according to the storage space configuration information, and obtain a data transmission channel configuration result for GPU computing; The operation synchronization module is used to perform storage area scheduling and thread synchronization between the GPU and the expansion card device for the GPU's to-be-operated data according to the data transmission channel configuration result through the CUDA shared storage mechanism until the operation process of the GPU is completed.

[0012] A third aspect of the present invention provides a device for optimizing GPU computing based on a CUDA shared memory mechanism, comprising: a memory and at least one processor, the memory storing instructions, the memory and the at least one processor being interconnected via lines; the at least one processor calling the instructions in the memory so that the device for optimizing GPU computing based on a CUDA shared memory mechanism executes the steps of the above-mentioned method for optimizing GPU computing based on a CUDA shared memory mechanism.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium, which stores instructions, and when the computer-readable storage medium is executed on a computer, the computer executes the steps of the above-mentioned method for optimizing GPU computing based on the CUDA shared storage mechanism.

[0014] The above-mentioned method and related equipment for optimizing GPU computing based on CUDA shared storage mechanism connect the expansion card device to the GPU through the PCIe interface on the GPU, and perform device identification and address space mapping processing on the GPU and the expansion card device to obtain the storage space configuration information required for GPU computing; perform space division and data channel establishment processing on the expansion card device according to the storage space configuration information to obtain the data transmission channel configuration result of GPU computing; perform storage area scheduling and thread synchronization between the GPU and the expansion card device for the GPU's to-be-calculated data according to the data transmission channel configuration result through the CUDA shared storage mechanism until the GPU computing process is completed. The present invention recognizes the GPU expansion device, divides the space, and efficiently establishes the data channel through the CUDA shared storage mechanism, realizes the storage scheduling and thread synchronization of data between the GPU and the expansion device, thereby improving computing efficiency and better supporting large-scale data processing requirements.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0016] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a first embodiment of a method for optimizing GPU computing based on a CUDA shared storage mechanism in an embodiment of the present invention; Figure 2 A schematic diagram of an embodiment of a device for optimizing GPU computing based on a CUDA shared memory mechanism in an embodiment of the present invention; Figure 3 Schematic diagram of an embodiment of a device for optimizing GPU computing based on a CUDA shared memory mechanism in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] The terms "including" and "having" and any variations thereof mentioned in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device end including a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or device ends.

[0020] To facilitate understanding of this embodiment, a method for optimizing GPU computing based on a CUDA shared storage mechanism disclosed in an embodiment of the present invention is first described in detail. Figure 1 As shown, the method is applied to a GPU and comprises the following steps: 101. Connect the expansion card device to the GPU through the PCIe interface on the GPU, perform device identification and address space mapping processing on the GPU and the expansion card device, and obtain storage space configuration information required for GPU computing; In one embodiment of the present invention, the method of connecting the expansion card device to the GPU through the PCIe interface on the GPU, and performing device identification and address space mapping processing on the GPU and the expansion card device to obtain the storage space configuration information required for GPU operation includes: connecting the expansion card device to the GPU through the PCIe interface on the GPU, and performing VID / DID reading and BDF structure scanning on the GPU device and the expansion card device to obtain a device identification code; performing an HBM firmware version check on the expansion card device according to the device identification code to obtain firmware information, and performing a memory space size detection on the expansion card device according to the firmware information through the BAR register of the configuration space in the PCIe interface to obtain available storage space information; and performing address range allocation on the expansion card device according to the memory space information by the CPU to obtain the storage space configuration information required for the GPU operation.

[0021] Specifically, when the expansion card device is connected to the GPU through the PCIe interface on the GPU, the system needs to ensure that the PCIe bus is in normal working condition. When the expansion card is inserted into the PCIe slot, the PCIe bus will automatically detect the access of the new device. At this time, the system will trigger the device enumeration process. In this process, the system reads the VID / DID of the GPU device and the expansion card device through the PCIe bus. This reading process is achieved by accessing the configuration space of the device, specifically by setting the Configuration Address 0xCF8 address of the I / O space. Register to determine the BDF combination to be accessed, and then perform the actual read operation through the 0xCFC address to obtain the vendor ID and device ID of the device. At the same time, the system will also scan the BDF structure of these devices, that is, determine the unique identifier of each PCIe device node through the combination of bus, device and function. The root complex, as the root node of the entire I / O architecture, dominates the device identification process. It completes the identification and management of PCIe devices under the control of the CPU, and the BDF structure constitutes the identifier of each PCIe device node. In this way, the system can accurately identify each device connected to the PCIe bus, and ensure the correct matching of the device type through the combination of VID / DID. These device identification information and BDF structure information together constitute the device identification code.

[0022] Specifically, after obtaining the device identification code, the system performs a version check on the HBM firmware of the expansion card device according to the identification code. The checking process is completed by reading a specific register in the configuration space of the expansion card device. The check result includes the specific version number and functional characteristics of the HBM firmware. This information is recorded as firmware information. Then the system operates through the BAR register in the PCIe configuration space. This process is achieved by writing all 1s to the BAR register and then reading the return value. Since some bits of the BAR register are read-only, by observing which bits remain unchanged, the size of the storage space required by the device can be known. For example, if the lower 12 bits do not change, it indicates that 4KB of space is required. This method of determining the size of the storage space through read and write operations is a standard mechanism specified by the PCIe protocol. In addition, the lower bits of the BAR register also include some attribute information, such as whether the storage space is memory space or I / O space, whether it supports prefetching and other features. These feature information have an important impact on the subsequent data access method. The system will decide how to access the storage space of the device based on these features, so that the available storage space information including space size and access characteristics is obtained.

[0023] Specifically, the CPU will allocate an address range to the expansion card device according to the acquired memory space information. This allocation process is achieved by setting the BAR register in the PCIe configuration space. The CPU will allocate a continuous address range in the system address space to the expansion card device according to the storage space required by the expansion card device. This address range will be written into the BAR register, thereby establishing a mapping relationship from the physical address to the device storage space. In the PCIe system, due to the existence of the root complex, when the CPU needs to read the data of the PCIe device, it will first let the root complex read the data from the PCIe device to the CPU memory, and then the CPU will read the data from the memory. Conversely, when the CPU needs to write data to the device, it will also first write the data to the memory and then write it to the PCIe device through the root complex. This memory-mapped I / O method enables the CPU to access the device's storage space like accessing the memory. In this way, the storage space configuration information required for GPU computing is fully established. These configuration information include important parameters such as the device's identity information, storage capacity information, address mapping relationship, and access characteristics, laying the foundation for the subsequent GPU computing process.

[0024] 102. Perform space division and data channel establishment processing on the expansion card device according to the storage space configuration information to obtain a data transmission channel configuration result for GPU computing; In one embodiment of the present invention, the space division and data channel establishment processing of the expansion card device according to the storage space configuration information to obtain the data transmission channel configuration result of the GPU operation includes: controlling the root complex through the CPU to divide the storage space of the expansion card device into a running capacity area and a cache area according to the storage space configuration information to obtain the storage area division result of the GPU operation; establishing the data transmission channel between the kernel function of the GPU and each storage area according to the storage area division result through the command register of the root complex to obtain the data transmission channel configuration result, and planning the data transmission order of the kernel function according to the data transmission channel configuration result.

[0025] Specifically, when the CPU controls the root complex to divide the space of the expansion card device according to the storage space configuration information, it is first necessary to determine the total capacity of the expansion card according to the obtained storage space configuration information. This capacity information has been obtained through the read and write operations of the BAR register. The CPU divides this storage space into two main areas through the root complex: the running capacity area and the cache area. The running capacity area is used to store actual computing data. When dividing, 2TB of the total capacity of 4TB is divided into the running capacity area, and the remaining part is used as the cache area. This division is achieved by setting the corresponding address range register in the root complex. The root complex will write these divided address ranges into the PCIe configuration space, thereby completing the space division at the hardware level. This division process also includes the setting of access rights and attributes of each area. These settings are recorded in the storage area division result, which contains detailed information such as the starting address, size, and access attributes of each area. This information provides necessary parameters for the establishment of subsequent data transmission channels.

[0026] Specifically, the system establishes a data transmission channel between the GPU kernel function and each storage area according to the storage area division result through the command register of the root complex. This process first needs to set the BME bit and the MSE bit in the command register. The setting of the BME bit allows the controller to send a DMA Read / Write Memory request to the Host, while the setting of the MSE bit controls whether the controller register can be accessed. The setting of these two bits provides the basic conditions for the establishment of the data transmission channel. After completing these basic settings, the root complex will create an independent data transmission channel for each storage area. These channels use the differential signal pair of PCIe for data transmission. Each channel has its own independent transmission instruction management mechanism. This independent channel design ensures that the data transmission process will not interfere with each other, and also improves the parallelism of data transmission. The configuration information of these created data transmission channels, including the physical parameters of the channel, bandwidth allocation, priority setting, etc., are all recorded in the data transmission channel configuration result. According to this configuration result, the system will also plan the data transmission order of the GPU kernel function and determine the transmission timing of different data blocks on different channels. This planning process takes into account the data dependency and access mode to ensure the efficiency of data transmission.

[0027] 103. Perform storage area scheduling and thread synchronization for the GPU's to-be-calculated data between the GPU and the expansion card device according to the data transmission channel configuration result through the CUDA shared storage mechanism until the calculation process of the GPU is completed; In one embodiment of the present invention, the method of performing storage area scheduling and thread synchronization between the GPU and the expansion card device for the data to be calculated of the GPU according to the data transmission channel configuration result through the CUDA shared storage mechanism until the operation process of the GPU is completed includes: declaring the expansion card device through the shared keyword in the CUDA shared storage mechanism to obtain the expansion card access configuration information, and performing data access allocation to the storage area of ​​the GPU according to the data transmission channel configuration result through the expansion card access configuration information to obtain a data access plan; using the data access plan to adopt a high-speed SLC data block calculation method to store and calculate the data to be calculated of the GPU, and performing thread synchronization to obtain a thread execution state; detecting the data access frequency in the GPU operation process according to the thread execution state through the root complex, and dynamically adjusting the distribution of the data to be calculated between the GPU and the expansion card device according to the detection result until the operation process of the GPU is completed.

[0028] Specifically, the system first declares the expansion card device using the __shared__ keyword through the CUDA shared storage mechanism. This declaration process is performed in the GPU kernel function. The declaration statement is marked with the __shared__ keyword, so that the storage space in the expansion card is identified as a shared memory space. The declaration process includes specifying the identifier of the expansion card, determining the size of the shared storage space and other parameters. These parameters are all included in the declaration statement. When executing the kernel function, the GPU will parse these declaration statements and extract the access configuration information of the expansion card, including the access rights to the expansion card storage space, the storage space size and other attributes. This declaration process actually completes the mapping and initialization of the expansion card storage space, allowing the GPU to treat the storage space of the expansion card as a directly accessible shared memory. Subsequently, the system combines these expansion card access configuration information with the previously obtained data transmission channel configuration results, and configures the data transmission channel based on the PCIe bus. This configuration process includes setting parameters such as data transmission priority, bandwidth allocation and access rights. After completing the basic configuration, the system allocates data access to the GPU's storage area based on the physical characteristics and access features of the storage area. This allocation process takes into account the data access pattern, data block size, and bandwidth characteristics of the storage area, and allocates the data to be calculated to different storage areas according to the optimization strategy. All these allocation strategies and specific allocation plans are recorded by the system in the data access plan, which contains detailed data distribution strategies and access control parameters.

[0029] Specifically, according to the data access scheme that has been formulated, the system starts to process the data to be calculated by the GPU, and uses the high-speed SLC data block calculation method for data storage and calculation. This method first divides the data to be calculated into multiple data blocks suitable for parallel processing, and each data block is stored in the SLC method. This storage method can provide faster reading and writing speeds. After the data block storage is completed, the various threads of the GPU begin to perform parallel calculations on these data blocks. In order to ensure data consistency, the system uses the __syncthreads() function at key points to synchronize all threads in the thread block. This synchronization operation ensures that all threads complete the calculation of the current stage before entering the next stage. During the calculation process, the system will continuously monitor and record the execution status of each thread, including detailed information such as calculation progress, data access status, and calculation resource utilization. At the same time, the system will also record the execution status of the thread synchronization point, including parameters such as synchronization waiting time and thread execution balance. All these runtime status information will be integrated into the thread execution status, which not only includes the running status of the thread, but also includes data access efficiency statistics and resource utilization analysis. These comprehensive information provides important decision-making basis for subsequent dynamic adjustments.

[0030] Specifically, the root complex begins to monitor the data access during the GPU computing process based on the obtained thread execution status. This monitoring process includes statistics on the number of times each data block is accessed, the time interval between accesses, the characteristics of the access pattern, and other detailed information. Through these statistical data, the system obtains a complete data access frequency distribution. These frequency data reflect the importance and usage characteristics of different data blocks in the computing process. After in-depth analysis of these frequency data, the root complex will formulate a data distribution adjustment strategy based on the analysis results. This adjustment process will comprehensively consider multiple factors such as data access frequency, performance characteristics of storage areas, and data migration overhead. Specifically, the system will dynamically migrate data blocks with high access frequency to storage areas with better performance, and migrate data blocks with low access frequency to slower storage areas. This dynamic adjustment is a continuous process. The system will continuously collect new access data, update optimization strategies, and perform corresponding data migration operations. At the same time, after each data migration, the system will evaluate the effect of the adjustment, including indicators such as changes in access latency and improvements in bandwidth utilization. These evaluation results will serve as an important reference for the next round of adjustments. This optimization process will continue until the entire GPU computing task is completed. In this way, the system continuously optimizes the storage location of data during operation, ensuring the efficient execution of the entire computing process.

[0031] Furthermore, the expansion card device is declared through the shared keyword in the CUDA shared storage mechanism to obtain the expansion card access configuration information, and data access allocation is performed on the storage area of ​​the GPU according to the data transmission channel configuration result through the expansion card access configuration information to obtain a data access plan; including: preprocessing the kernel function code of the GPU through the CUDA shared storage mechanism to obtain a code segment with the shared keyword, and inserting the identifier of the expansion card device into the code segment to obtain a shared memory declaration statement of the extended memory; performing syntax parsing processing on the shared memory declaration statement to obtain a parameter list including the storage space size of the expansion card device; detecting the firmware version of the expansion card device according to the parameter list, and conditionally compiling the shared memory declaration statement based on the firmware version to obtain the expansion card access configuration information; performing data access allocation on the storage area of ​​the GPU using the expansion card access configuration information and the data transmission channel configuration result to obtain the data access plan.

[0032] Specifically, when starting the CUDA program compilation process, the system first needs to preprocess the GPU kernel function code. This preprocessing process mainly focuses on the memory management related code in the kernel function, and finds all code segments that use the __shared__ keyword by scanning the source code. The preprocessor will analyze the grammatical structure of these code segments, extract the declaration parts related to shared memory, and form independent code segments. After obtaining these code segments, the system needs to insert the identifier of the expansion card device into these code segments. This insertion process needs to consider the grammatical correctness and semantic integrity of the code. Specifically, the system will add the unique identifier of the expansion card device to the code segment of the shared memory declaration, and add necessary type descriptions and size restrictions. This process will generate a complete extended memory declaration statement, which contains both the original shared memory declaration information and the specific properties of the expansion card device, so that the compiler can correctly identify and process this extended memory space.

[0033] Specifically, when parsing the generated shared memory declaration statement, the system will start a special parser to perform lexical analysis and grammatical analysis on the declaration statement. The lexical analysis stage will decompose the declaration statement into a series of tokens, including keywords, identifiers, operators, etc. The grammatical analysis stage will build an abstract syntax tree according to the grammatical rules of CUDA. In this process, the parser pays special attention to parameter information related to storage space, such as the size of storage space, alignment requirements, access rights, etc. These parameter information will be extracted to form a structured parameter list. This parameter list not only contains the storage space size information of the expansion card device, but also other important configuration parameters, such as the base address of the storage space, the granularity of data access, etc. These parameters together constitute the basic information set for accessing and controlling the expansion card device.

[0034] Specifically, after obtaining the parameter list, the system will detect the firmware version of the expansion card device based on these parameter information. This detection process is completed by reading specific registers in the configuration space of the expansion card device. The detection result contains the specific version number and functional characteristics of the HBM firmware. Based on the detected firmware version information, the system will conditionally compile the shared memory declaration statement. This process will select different compilation strategies according to different firmware versions to ensure that the generated code can correctly adapt to the current firmware version. The result of the conditional compilation process is the expansion card access configuration information, which contains all the necessary access parameters and control information, providing a complete configuration basis for subsequent data access operations.

[0035] Specifically, the system will allocate data access to the GPU's storage area based on the expansion card access configuration information and the previously existing data transmission channel configuration results. This allocation process will first analyze the bandwidth and latency characteristics of the data transmission channel, and then design the data access strategy based on these characteristics. Specifically, the system will reasonably allocate data access requests to different storage areas based on the performance characteristics of the storage area. High-frequency access data will be preferentially allocated to high-speed storage areas, while data with lower access frequency will be allocated to ordinary storage areas. This allocation process will comprehensively consider multiple factors such as data access patterns, bandwidth limitations of storage areas, and data consistency requirements, and ultimately form a complete data access plan that specifies in detail the storage location, access method, and synchronization strategy of each data block. These plans will guide subsequent actual data access operations.

[0036] Furthermore, the data access scheme is used to perform data storage and calculation on the GPU's data to be calculated in a high-speed SLC data block calculation manner, and thread synchronization is performed to obtain a thread execution status, including: grouping the GPU's data to be calculated in data blocks through the data access scheme to obtain an SLC data block group, and performing data storage allocation between the GPU and the expansion card device for the data to be calculated according to the SLC data block group to obtain a data storage status; performing parallel calculation processing on the SLC data blocks through the data storage status to obtain data block calculation results, and executing synchronization functions on threads in the GPU thread blocks according to the data block calculation results to obtain thread synchronization information; performing status detection on the calculation progress of each GPU thread block through the thread synchronization information to obtain a thread running status, and performing execution status marking on the GPU thread blocks according to the thread running status to obtain the thread execution status.

[0037] Specifically, when processing the GPU data to be calculated according to the data access scheme, the system first needs to group the data into data blocks. This grouping process is based on the characteristics of the SLC data block calculation method. Specifically, the data to be calculated is divided into multiple data blocks according to a fixed size. The size of each data block will take into account the characteristics of the SLC storage unit and the processing power of the GPU. During the grouping process, the system will number and classify the data blocks according to the access mode and dependency of the data, and organize the data blocks with similar access characteristics or calculation dependencies together to form an SLC data block group. In the process of organizing these data blocks, the system will establish an association mapping table between data blocks, record the dependency and access order between data blocks, and this information has important guiding significance for subsequent parallel processing. After obtaining these data block groups, the system will allocate data storage according to the characteristics of each data block group, combined with the storage characteristics of the GPU and expansion card devices. This allocation process will consider the access frequency of data blocks, the correlation between data blocks, and the bandwidth characteristics of the storage device, and reasonably allocate data blocks to different storage areas. During the allocation process, the system also needs to consider the physical characteristics of the storage device, such as read and write latency, bandwidth limitations, etc., to ensure that the data allocation plan can maximize the performance advantages of the storage device. After the allocation is completed, the system will record the storage location, access rights, and synchronization requirements of each data block, and establish a complete data block storage mapping table. This mapping table contains detailed information such as the physical location, logical location, and access attributes of the data block. At the same time, the system will also record the topological relationship between data blocks. This information is crucial to ensure the correctness and efficiency of data access. All of this information together constitutes the data storage state, which not only contains static storage allocation information, but also dynamic data access control information, providing a complete storage location and access control basis for subsequent parallel computing.

[0038] Specifically, based on the obtained data storage status, the system starts to perform parallel computing on the SLC data blocks. This processing process first finds the corresponding data blocks according to the location information recorded in the data storage status, and then starts the parallel processing unit of the GPU to operate on these data blocks. During the operation process, the system tracks the processing progress of each data block and records the intermediate results and data dependencies of the operation. The operation of each data block will go through multiple processing stages, including data loading, calculation execution, result storage, etc. The system will record detailed execution information at each stage, including calculation time, resource occupation, data access mode, etc. When the operation of a group of data blocks is completed, the system will store the operation results to the specified storage location and update the corresponding status mark. These operation results will be organized into a structured data set, which contains key data such as the value, precision information, and timestamp of the calculation results. Based on these operation results, the system will call the synchronization function (__syncthreads()) in the GPU thread block to ensure that all related threads have completed the operation of the current stage. When performing synchronization operations, the system will record the specific situation of each synchronization point, including the number of threads participating in synchronization, the execution status of each thread, and the synchronization waiting time. These synchronization operations not only ensure data consistency, but also provide important performance monitoring data. At the same time, the system also records the data exchange within the thread block, including shared memory access mode, data handling volume and other information. All these execution records are integrated to form complete thread synchronization information, which not only reflects the current execution status, but also contains detailed performance indicators and resource utilization data.

[0039] Specifically, based on the obtained thread synchronization information, the system starts to perform a comprehensive status detection on the computing progress of each GPU thread block. This detection process first collects the basic operation data of each thread block, including the number of completed computing tasks, the current execution stage, resource usage, etc. The system will conduct an in-depth analysis of this data and calculate various performance indicators, such as computing throughput, memory bandwidth utilization, thread block scheduling efficiency, etc. By analyzing these indicators, the system can accurately evaluate the running status and performance of each thread block. After obtaining these basic data, the system will build a complete thread running status model, which not only contains static configuration information but also dynamic execution data. Based on this running status, the system will set corresponding execution status marks for each GPU thread block. These marks are a multi-dimensional data structure that contains information on the running stage, completion degree, performance indicators, resource occupancy, etc. of the thread block. When setting these marks, the system will consider the dependencies and resource competition between thread blocks to ensure that the status marks can accurately reflect the actual execution status. At the same time, the system will also establish an association graph between thread blocks to record the data dependencies and execution order relationships between them. All these status tags and related information are finally integrated into a complete thread execution state, which not only contains detailed operation information at the current moment, but also contains execution history and performance statistics, providing a comprehensive basis for subsequent operation monitoring and optimization. This execution state is a dynamically updated data structure that will be continuously updated and improved as the operation proceeds, ensuring that the system can grasp the execution status of the entire operation process in real time.

[0040] Further, the detecting of the data access frequency in the GPU operation process according to the thread execution state through the root complex, and dynamically adjusting the distribution of the data to be operated between the GPU and the expansion card device according to the detection result until the operation process of the GPU is completed includes: counting the access to the data to be operated in the GPU operation according to the thread execution state through the root complex to obtain a data access frequency table, analyzing the access pattern of the data to be operated according to the data access frequency table to obtain data access characteristics; performing data distribution evaluation on the video memory space of the GPU and the storage space of the expansion card device according to the data access characteristics to obtain a space load state, calculating the distribution of the data to be operated between the running capacity area and the cache area according to the space load state to obtain a data migration plan; redistributing the data to be operated between the GPU and the expansion card device according to the data migration plan to obtain a new data distribution result, and performing statistics on the data access efficiency of the GPU operation process according to the new data distribution result until the operation process of the GPU is completed.

[0041] Specifically, the root complex starts to count the accesses to the data to be calculated in the GPU calculation according to the thread execution status. This counting process is achieved by setting an access counter in the root complex. Whenever a data access request occurs, the corresponding counter will increase. During the statistical process, the system will record the timestamp of each data block being accessed, the thread ID of the access, the type of access (read / write), and other detailed information. These original access records will be organized into a structured data access frequency table, which contains information such as the number of accesses, access time distribution, and access source of each data block. Based on this frequency table, the system will conduct an in-depth analysis of the data access pattern. The analysis process will consider the time locality, spatial locality, and regularity of the access. By analyzing these access patterns, the system can identify features such as hot data, access regularity, and data dependency. These features are organized into data access features, which include detailed access pattern descriptions, access frequency distribution, and correlation between data blocks. These access features not only reflect the current data access status, but also contain important basis for predicting future access patterns.

[0042] Next, based on the data access characteristics obtained, the system begins to conduct a comprehensive data distribution evaluation of the GPU's video memory space and the storage space of the expansion card device. This evaluation process first analyzes the current usage of each storage area, including space occupancy, data block distribution, access bandwidth utilization, etc. The physical characteristics of different storage areas, such as access latency, bandwidth limitations, and concurrency capabilities, are considered during the evaluation. At the same time, the system also evaluates the impact of the current data distribution on performance, including data access latency, bandwidth utilization efficiency, storage space utilization, and other aspects. These evaluation results are integrated into a complete space load state, which describes the operation of the storage system in detail. Based on this load state, the system further calculates the optimal data allocation plan between the operating capacity area and the cache area. This calculation process considers multiple factors such as data access frequency, storage area performance characteristics, and data migration overhead, and a balanced data migration plan is obtained through comprehensive analysis. This plan not only includes specific data migration strategies, but also includes the migration timing arrangement and resource allocation plan.

[0043] Finally, the system starts to perform specific data redistribution operations according to the formulated data migration plan. This redistribution process will first select the appropriate migration time according to the migration plan, and then start data migration in a predetermined order. During the migration process, the system will monitor the execution of the migration operation in real time, including migration speed, resource usage, and impact on normal operations. When the data migration is completed, the system will update the data location mapping table to record the new data distribution. This new data distribution result includes updated data location information, access path information, etc. Based on this new distribution result, the system will continue to monitor the data access efficiency during the GPU computing process, including multiple performance indicators such as access latency, bandwidth utilization, and cache hit rate. These performance data will be continuously collected and analyzed to evaluate the effect of data redistribution and serve as an important basis for the next round of data distribution optimization. This monitoring and optimization process will continue until the entire GPU computing process is completed to ensure that high data access efficiency is maintained throughout the computing process.

[0044] In this embodiment, the expansion card device is connected to the GPU through the PCIe interface on the GPU, and the GPU and the expansion card device are subjected to device identification and address space mapping processing to obtain the storage space configuration information required for GPU operation; the expansion card device is subjected to space division and data channel establishment processing according to the storage space configuration information to obtain the data transmission channel configuration result of GPU operation; the storage area scheduling and thread synchronization of the GPU's to-be-operated data between the GPU and the expansion card device are performed according to the data transmission channel configuration result through the CUDA shared storage mechanism until the GPU operation process is completed. The present invention recognizes the GPU expansion device, divides the space, and efficiently establishes the data channel through the CUDA shared storage mechanism, realizes the storage scheduling and thread synchronization of data between the GPU and the expansion device, thereby improving the computing efficiency and better supporting the large-scale data processing requirements.

[0045] The above describes the method for optimizing GPU computing based on the CUDA shared memory mechanism in the embodiment of the present invention. The following describes the device for optimizing GPU computing based on the CUDA shared memory mechanism in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a device for optimizing GPU computing based on a CUDA shared storage mechanism includes: The connection mapping module 201 is used to connect the expansion card device to the GPU through the PCIe interface on the GPU, and perform device identification and address space mapping processing on the GPU and the expansion card device to obtain the storage space configuration information required for GPU operation; A channel configuration module 202 is used to perform space division and data channel establishment processing on the expansion card device according to the storage space configuration information to obtain a data transmission channel configuration result for GPU computing; The operation synchronization module 203 is used to perform storage area scheduling and thread synchronization for the GPU's to-be-operated data between the GPU and the expansion card device according to the data transmission channel configuration result through the CUDA shared storage mechanism until the operation process of the GPU is completed.

[0046] In an embodiment of the present invention, the device for optimizing GPU computing based on the CUDA shared storage mechanism runs the above-mentioned method for optimizing GPU computing based on the CUDA shared storage mechanism, and the device for optimizing GPU computing based on the CUDA shared storage mechanism connects the expansion card device to the GPU through the PCIe interface on the GPU, and performs device identification and address space mapping processing on the GPU and the expansion card device to obtain the storage space configuration information required for GPU computing; performs space division and data channel establishment processing on the expansion card device according to the storage space configuration information to obtain the data transmission channel configuration result of GPU computing; performs storage area scheduling and thread synchronization between the GPU and the expansion card device for the GPU's to-be-calculated data according to the data transmission channel configuration result through the CUDA shared storage mechanism, until the GPU computing process is completed. The present invention recognizes the GPU expansion device, divides the space, and efficiently establishes the data channel through the CUDA shared storage mechanism, realizes the storage scheduling and thread synchronization of data between the GPU and the expansion device, thereby improving the computing efficiency and better supporting the large-scale data processing requirements.

[0047] above Figure 2 From the perspective of modular functional entities, the device for optimizing GPU computing based on the CUDA shared memory mechanism in the embodiment of the present invention is described in detail. The following describes in detail the device for optimizing GPU computing based on the CUDA shared memory mechanism in the embodiment of the present invention from the perspective of hardware processing.

[0048] Figure 3It is a structural schematic diagram of a device for optimizing GPU computing based on a CUDA shared storage mechanism provided by an embodiment of the present invention. The device 300 for optimizing GPU computing based on a CUDA shared storage mechanism may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage device terminals) storing application programs 333 or data 332. Among them, the memory 320 and the storage medium 330 may be short-term storage or persistent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the device 300 for optimizing GPU computing based on a CUDA shared storage mechanism. Furthermore, the processor 310 may be configured to communicate with the storage medium 330, and execute a series of instruction operations in the storage medium 330 on the device 300 for optimizing GPU computing based on a CUDA shared storage mechanism to implement the steps of the above-mentioned method for optimizing GPU computing based on a CUDA shared storage mechanism.

[0049] The device 300 for optimizing GPU computing based on the CUDA shared memory mechanism may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It will be appreciated by those skilled in the art that Figure 3 The device structure for optimizing GPU computing based on the CUDA shared memory mechanism shown does not constitute a limitation on the device for optimizing GPU computing based on the CUDA shared memory mechanism provided by the present invention, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0050] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the steps of the method for optimizing GPU operations based on the CUDA shared storage mechanism.

[0051] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device, or unit can refer to the corresponding process in the aforementioned method embodiment and will not be repeated here.

[0052] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0053] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for optimizing GPU computing based on CUDA shared memory mechanism, characterized in that: The method for optimizing GPU computing based on CUDA shared storage mechanism includes: Connect the expansion card device to the GPU through the PCIe interface on the GPU, perform device identification and address space mapping on the GPU and the expansion card device, and obtain the storage space configuration information required for GPU computing; Performing space division and data channel establishment processing on the expansion card device according to the storage space configuration information to obtain a data transmission channel configuration result for GPU computing; The CUDA shared storage mechanism performs storage area scheduling and thread synchronization on the GPU's to-be-calculated data between the GPU and the expansion card device according to the data transmission channel configuration result until the GPU's calculation process is completed.

2. The method for optimizing GPU computing based on CUDA shared memory mechanism according to claim 1, characterized in that: The expansion card device is connected to the GPU through the PCIe interface on the GPU, and the GPU and the expansion card device are subjected to device identification and address space mapping processing to obtain the storage space configuration information required for GPU operation, including: Connect the expansion card device to the GPU through the PCIe interface on the GPU, read the VID / DID and scan the BDF structure of the GPU device and the expansion card device to obtain the device identification code; Perform an HBM firmware version check on the expansion card device according to the device identification code to obtain firmware information, and perform a memory space size check on the expansion card device according to the firmware information through a BAR register in a configuration space in a PCIe interface to obtain available storage space information; The CPU allocates an address range to the expansion card device according to the memory space information to obtain the storage space configuration information required for the GPU operation.

3. The method for optimizing GPU computing based on CUDA shared memory mechanism according to claim 1, characterized in that: The performing space division and data channel establishment processing on the expansion card device according to the storage space configuration information to obtain the data transmission channel configuration result of the GPU operation includes: Controlling the root complex through the CPU to divide the storage space of the expansion card device into a running capacity area and a cache area according to the storage space configuration information, thereby obtaining a storage area division result for GPU computing; A data transmission channel between the kernel function of the GPU and each storage area is established according to the storage area division result through the command register of the root complex, a data transmission channel configuration result is obtained, and the data transmission sequence of the kernel function is planned according to the data transmission channel configuration result.

4. The method for optimizing GPU computing based on CUDA shared memory mechanism according to claim 1, characterized in that: The process of performing storage area scheduling and thread synchronization between the GPU and the expansion card device for the data to be calculated by the GPU according to the data transmission channel configuration result through the CUDA shared storage mechanism until the calculation process of the GPU is completed includes: Declare the expansion card device through the shared keyword in the CUDA shared storage mechanism to obtain the expansion card access configuration information, and allocate data access to the storage area of ​​the GPU according to the data transmission channel configuration result through the expansion card access configuration information to obtain a data access plan; The data to be calculated of the GPU is stored and calculated by adopting a high-speed SLC data block calculation method through the data access scheme, and thread synchronization is performed to obtain the thread execution status; The data access frequency in the GPU operation process is detected by the root complex according to the thread execution state, and the distribution of the data to be operated between the GPU and the expansion card device is dynamically adjusted according to the detection result until the GPU operation process is completed.

5. The method for optimizing GPU computing based on CUDA shared memory mechanism according to claim 4, characterized in that: The expansion card device is declared by using the shared keyword in the CUDA shared storage mechanism to obtain the expansion card access configuration information, and data access is allocated to the storage area of ​​the GPU according to the data transmission channel configuration result by using the expansion card access configuration information to obtain a data access plan; including: Preprocessing the kernel function code of the GPU through the CUDA shared storage mechanism to obtain a code segment with a shared keyword, and inserting an identifier of the expansion card device into the code segment to obtain a shared memory declaration statement of the extended memory; Performing syntax parsing on the shared memory declaration statement to obtain a parameter list including the storage space size of the expansion card device; According to the parameter list, the firmware version of the expansion card device is detected, and based on the firmware version, the shared memory declaration statement is conditionally compiled to obtain the expansion card access configuration information; The expansion card access configuration information and the data transmission channel configuration result are used to allocate data access to the storage area of ​​the GPU to obtain the data access plan.

6. The method for optimizing GPU computing based on CUDA shared memory mechanism according to claim 4, characterized in that: The data access scheme adopts a high-speed SLC data block calculation method to store and calculate the GPU data to be calculated, and performs thread synchronization to obtain the thread execution state, including: The data to be calculated by the GPU is grouped into data blocks by using the data access scheme to obtain an SLC data block group, and data storage allocation is performed between the GPU and the expansion card device according to the SLC data block group to obtain a data storage state; Performing parallel computing processing on the SLC data block through the data storage state to obtain a data block computing result, and executing a synchronization function on the threads in the GPU thread block according to the data block computing result to obtain thread synchronization information; The operation progress of each thread block of the GPU is detected through the thread synchronization information to obtain the thread running state, and the execution state of the thread block of the GPU is marked according to the thread running state to obtain the thread execution state.

7. The method for optimizing GPU computing based on CUDA shared memory mechanism according to claim 4, characterized in that: The step of detecting the data access frequency in the GPU computing process according to the thread execution state through the root complex, and dynamically adjusting the distribution of the data to be computed between the GPU and the expansion card device according to the detection result until the computing process of the GPU is completed includes: Counting the accesses to the data to be calculated in the GPU calculation according to the thread execution state through the root complex to obtain a data access frequency table, and analyzing the access pattern of the data to be calculated according to the data access frequency table to obtain data access characteristics; Performing data distribution evaluation on the video memory space of the GPU and the storage space of the expansion card device through the data access characteristics to obtain a space load state, and calculating the distribution of the data to be operated between the running capacity area and the cache area according to the space load state to obtain a data migration plan; The data to be calculated is redistributed between the GPU and the expansion card device through the data migration scheme to obtain a new data distribution result, and the data access efficiency of the GPU calculation process is statistically analyzed according to the new data distribution result until the calculation process of the GPU is completed.

8. A device for optimizing GPU computing based on CUDA shared storage mechanism, characterized in that: Applied to GPU, the device for optimizing GPU computing based on CUDA shared storage mechanism includes: The connection mapping module is used to connect the expansion card device to the GPU through the PCIe interface on the GPU, and perform device identification and address space mapping processing on the GPU and the expansion card device to obtain the storage space configuration information required for GPU operation; A channel configuration module, used to perform space division and data channel establishment processing on the expansion card device according to the storage space configuration information, and obtain a data transmission channel configuration result for GPU computing; The operation synchronization module is used to perform storage area scheduling and thread synchronization between the GPU and the expansion card device for the GPU's to-be-operated data according to the data transmission channel configuration result through the CUDA shared storage mechanism until the operation process of the GPU is completed.

9. A device for optimizing GPU computing based on CUDA shared storage mechanism, characterized in that: The device for optimizing GPU computing based on the CUDA shared storage mechanism includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instruction in the memory so that the device for optimizing GPU computing based on the CUDA shared memory mechanism executes the steps of the method for optimizing GPU computing based on the CUDA shared memory mechanism as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the method for optimizing GPU computing based on the CUDA shared memory mechanism as described in any one of claims 1 to 7 are implemented.

Images