A GPGPU memory optimization method, device, equipment and storage medium

Abstract

The application discloses a GPGPU storage optimization method and device, equipment and a storage medium, and relates to the technical field of data access. The method comprises the following steps: obtaining task data, determining target ratios corresponding to each task data based on preset address index entries; judging whether the memory access operation corresponding to each task data is a write operation; if yes, storing each first target task data in the corresponding position of the target storage space according to the space occupation condition of the target storage space and the target ratios corresponding to each target task data, and periodically adjusting the position of the task data in the target storage space; wherein, the target storage space comprises an MRAM main storage area and a hybrid cache area, and the hybrid cache area comprises an SRAM storage area and an MRAM storage area. Through the data storage of the SRAM storage area, the storage efficiency is improved, and through the data storage of the MRAM, the system power consumption is reduced.

Description

Technical Field

[0001] This invention relates to the field of data access technology, and in particular to a GPGPU storage optimization method, apparatus, device, and storage medium. Background Technology

[0002] In modern high-performance computing scenarios, GPGPUs (General-purpose computing on graphics processing units) are being used more and more widely, and their requirements for data storage and processing capabilities are becoming increasingly stringent. Traditional storage architectures are gradually revealing many shortcomings when dealing with the large-scale, high-concurrency data read and write demands of GPGPUs.

[0003] Currently, while DRAM (Dynamic Random Access Memory) is commonly used for data storage in GPGPUs and has a certain storage capacity, it suffers from relatively slow read / write speeds, high power consumption, and susceptibility to data loss. For example, during deep learning training and inference, frequent data exchanges can make DRAM bandwidth a performance bottleneck, and its periodic refresh operations consume a significant amount of power. On the other hand, while SRAM (Static Random-Access Memory) offers fast read / write speeds, its high cost and low integration make it difficult to deploy on a large scale in GPGPUs as the primary storage medium. Therefore, how to achieve low-power, high-security, and fast data read / write speeds for GPGPU data storage remains a pressing technical challenge. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a GPGPU storage optimization method, apparatus, device, and storage medium that improves storage efficiency by utilizing SRAM storage area for data storage and reduces system power consumption by utilizing MRAM for data storage. The specific solution is as follows:

[0005] Firstly, this application provides a GPGPU storage optimization method, including:

[0006] The first target task data issued by the GPGPU computing core is obtained, and the target ratio corresponding to each first target task data is determined based on the preset address index entries. The preset address index entries include memory access address information, memory access count information, and clock count information. The memory access address information includes the memory access address corresponding to each first target task data. The memory access count information is the number of read operations performed after the memory access address is written to the preset address index entry. The clock count information is the number of clock cycles performed after the memory access address is written to the preset address index entry. The target ratio is the ratio of the memory access count information to the clock count information.

[0007] Determine whether the memory access operation corresponding to each first target task data is a write operation. If the memory access operation corresponding to each first target task data is a write operation, then store each first target task data in the corresponding location of the target storage space according to the space occupancy of the target storage space and the target ratio corresponding to each first target task data; wherein, the target storage space includes an MRAM main storage area and a hybrid cache area, and the hybrid cache area includes an SRAM storage area and an MRAM storage area;

[0008] Upon receiving a new memory access operation, the storage location of the first target task data in the target storage space is periodically and dynamically adjusted according to the target ratio of the corresponding second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space.

[0009] Optionally, the step of periodically and dynamically adjusting the storage location of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space, includes:

[0010] If the data storage volume of the SRAM storage area has reached its upper limit, and the data storage volume of the MRAM storage area has not reached its upper limit, then the preset address index entries are sorted in ascending order of the target ratio, and it is determined whether the first target task data with the smallest target ratio meets the preset hot data judgment criteria within the current preset time period.

[0011] If the first target task data with the smallest target ratio meets the preset hot data judgment criteria within the current preset time period, then the new memory access address corresponding to the new memory access operation is stored in the preset address index entry, and the second target task data corresponding to the new memory access address is migrated from the MRAM main storage area to the MRAM storage area; wherein, the number of configurable entries of the preset address index entry is the same as the maximum number of tasks stored in the hybrid cache area;

[0012] If the first target task data with the smallest target ratio does not meet the preset hot data determination criteria within the current preset time period, then the first target task data with the smallest target ratio will be migrated from the SRAM storage area to the MRAM storage area, and the second target task data corresponding to the new memory access address will be migrated from the MRAM main storage area to the SRAM storage area.

[0013] Optionally, the step of periodically and dynamically adjusting the storage location of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space, includes:

[0014] If the data storage capacity of both the SRAM storage area and the MRAM storage area has reached its limit, then determine whether the memory access address corresponding to the new memory access operation is in the preset address index entry;

[0015] If the memory access address corresponding to the new memory access operation is in the preset address index entry, then it is determined whether the new memory access operation is a read operation. If the new memory access operation is a read operation, then the storage location of the first target task data is adjusted according to the target ratio of the second target task data corresponding to the new memory access operation.

[0016] If the new memory access operation is a write operation, the second target task data corresponding to the new memory access operation is stored in the MRAM main memory area, the second target task data is deleted from the hybrid cache area, and the storage location corresponding to the second target task data in the preset address index entry is recorded as an invalid cache.

[0017] Optionally, adjusting the storage location of the first target task data based on the target ratio of the second target task data corresponding to the new memory access operation includes:

[0018] If the second target task data corresponding to the new memory access operation is stored in the SRAM storage area, and the target ratio corresponding to the second target task data is less than the maximum target ratio corresponding to each of the first target task data in the MRAM storage area, then the storage area of ​​the second target task data is swapped with the storage area of ​​the first target task data corresponding to the maximum target ratio.

[0019] If the second target task data corresponding to the new memory access operation is stored in the MRAM storage area, and the target ratio corresponding to the second target task data is greater than the minimum target ratio corresponding to each of the first target task data in the SRAM storage area, then the storage area of ​​the second target task data is swapped with the storage area of ​​the first target task data corresponding to the minimum target ratio.

[0020] Optionally, the step of periodically and dynamically adjusting the storage location of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space, includes:

[0021] If the memory access address corresponding to the new memory access operation is not in the preset address index entry, then it is determined whether the new memory access operation is a read operation. If the new memory access operation is a read operation, then it is determined whether there is an entry in the preset address index entry whose storage location is an invalid cache.

[0022] If there is no entry in the preset address index that is stored in an invalid cache, then the second target task data corresponding to the new memory access operation is read from the MRAM main memory area;

[0023] If there is an entry in the preset address index that is an invalid cache, then it is determined whether the space occupied by the first target task data in the SRAM storage area is not greater than the total storage space of the SRAM storage area. If the space occupied by the first target task data in the SRAM storage area is not greater than the total storage space of the SRAM storage area, then the second target task data corresponding to the new memory access operation is migrated from the MRAM main storage area to the SRAM storage area.

[0024] If the new memory access operation is a write operation, the second target task data corresponding to the new memory access operation is stored in the MRAM main memory area.

[0025] Optionally, after storing each target task data in its corresponding location within the target storage space, the method further includes:

[0026] Based on the data type and data access frequency of each target task data, the data to be used is predicted from each target task data, and the data to be used is loaded from the MRAM main storage area to the hybrid cache area so that the GPGPU computing core can process the data to be used.

[0027] Optionally, the GPGPU storage optimization method further includes: expanding the initial instruction set corresponding to the GPGPU computing core to obtain a corresponding target instruction set, so as to use the target instruction set to control the data storage operations of the MRAM main storage area and the corresponding MRAM storage area.

[0028] Secondly, this application provides a GPGPU storage optimization device, comprising:

[0029] The data acquisition module is used to acquire the first target task data issued by the GPGPU computing core, and determine the target ratio corresponding to each of the first target task data based on the preset address index entries; wherein, the preset address index entries include memory access address information, memory access count information, and clock count information, the memory access address information includes the memory access address corresponding to each of the first target task data, the memory access count information is the number of read operations experienced after the memory access address is written to the preset address index entry, the clock count information is the number of clock cycles experienced after the memory access address is written to the preset address index entry, and the target ratio is the ratio of the memory access count information to the clock count information;

[0030] The data storage module is used to determine whether the memory access operation corresponding to each first target task data is a write operation. If the memory access operation corresponding to each first target task data is a write operation, then the first target task data is stored in the corresponding location of the target storage space according to the space occupancy of the target storage space and the target ratio corresponding to each first target task data. The target storage space includes an MRAM main storage area and a hybrid cache area, and the hybrid cache area includes an SRAM storage area and an MRAM storage area.

[0031] The storage location adjustment module is used to periodically and dynamically adjust the storage location of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage status of the target storage space after a new memory access operation is obtained, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space.

[0032] Thirdly, this application provides an electronic device, comprising:

[0033] Memory, used to store computer programs;

[0034] A processor for executing the computer program to implement the aforementioned GPGPU storage optimization method.

[0035] Fourthly, this application provides a computer-readable storage medium for storing a computer program, which, when executed by a processor, implements the aforementioned GPGPU storage optimization method.

[0036] In this application, firstly, the first target task data issued by the GPGPU computing core is obtained, and the target ratio corresponding to each first target task data is determined based on a preset address index entry. The preset address index entry includes memory access address information, memory access count information, and clock count information. The memory access address information includes the memory access address corresponding to each first target task data. The memory access count information is the number of read operations performed after the memory access address is written to the preset address index entry. The clock count information is the number of clock cycles performed after the memory access address is written to the preset address index entry. The target ratio is the ratio of the memory access count information to the clock count information. Then, it is determined whether the memory access operation corresponding to each first target task data is a write operation. If each... If the memory access operation corresponding to the first target task data is a write operation, then according to the space occupancy of the target storage space and the target ratio corresponding to each of the first target task data, the first target task data is stored in the corresponding location of the target storage space; wherein, the target storage space includes an MRAM main storage area and a hybrid cache area, and the hybrid cache area includes an SRAM storage area and an MRAM storage area. After a new memory access operation is obtained, the storage location of the first target task data in the target storage space is periodically and dynamically adjusted according to the target ratio of the corresponding second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space. Therefore, this application optimizes the storage structure of the GPGPU by constructing a storage area that includes MRAM and SRAM. This allows the MRAM and SRAM storage areas to store data with different access frequencies, solving the problem that SRAM has low integration density and cannot be applied to GPGPUs. It also fully utilizes the advantage of SRAM's fast read and write speed, improving data storage efficiency. By using MRAM and SRAM for data storage, the data loss problem caused by using DRAM for data storage is avoided. By storing data with lower access frequencies in MRAM, frequent data exchange between the main storage area and the cache area is avoided, reducing power consumption. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0038] Figure 1 This is a flowchart of a GPGPU storage optimization method disclosed in this application;

[0039] Figure 2 This is a schematic diagram of a GPGPU storage structure disclosed in this application;

[0040] Figure 3 This is a schematic diagram of an address index entry structure disclosed in this application;

[0041] Figure 4 This is a schematic diagram of a specific GPGPU storage optimization method disclosed in this application;

[0042] Figure 5 This is a flowchart of a data prefetching method disclosed in this application;

[0043] Figure 6 This is a schematic diagram of the structure of a GPGPU storage optimization device disclosed in this application;

[0044] Figure 7 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Currently, GPGPU data storage suffers from relatively slow read / write speeds, high power consumption, and susceptibility to data loss. To address these issues, this application provides a GPGPU storage optimization method. By constructing a storage region comprising MRAM and SRAM, and utilizing the SRAM region for high-frequency data storage, storage efficiency is improved. Conversely, by utilizing MRAM for low-frequency data storage, system power consumption is reduced.

[0047] See Figure 1 As shown, this embodiment of the invention discloses a GPGPU storage optimization method, including:

[0048] Step S11: Obtain the first target task data issued by the GPGPU computing core, and determine the target ratio corresponding to each first target task data based on the preset address index entries; wherein, the preset address index entries include memory access address information, memory access count information, and clock count information, the memory access address information includes the memory access address corresponding to each first target task data, the memory access count information is the number of read operations experienced after the memory access address is written to the preset address index entry, the clock count information is the number of clock cycles experienced after the memory access address is written to the preset address index entry, and the target ratio is the ratio of the memory access count information to the clock count information.

[0049] This embodiment provides a novel GPGPU storage architecture, the specific architecture of which is as follows: Figure 2 As shown, it mainly consists of an MRAM (Magnetoresistive Random Access Memory) main storage area, a hybrid cache layer, an intelligent scheduler, and a GPGPU computing core. The GPGPU computing core is the upper computing layer of the storage architecture of this invention, responsible for completing specific computing and storage instructions. To ensure the backward compatibility of this invention with the original GPGPU computing core, this invention does not change the external interface of the original GPGPU computing core, but achieves performance optimization for the MRAM storage architecture by expanding the instruction set of the GPGPU computing core and adding GPGPU functional modules.

[0050] The intelligent scheduler is the core control unit of the storage architecture of this invention. By setting up instruction detection units and data detection units in the GPGPU computing core, it maintains the historical data access trajectory internally. It also sets up storage status detection units in the hybrid cache area and MRAM main storage area to complete the data flow detection between storage levels. Through internal intelligent algorithms, it dynamically adjusts the data allocation and transmission strategy between the MRAM main storage area, the hybrid cache layer, and the computing core based on factors such as the computing task type of the GPGPU, data access frequency, and temporal locality.

[0051] The hybrid cache layer is a transitional cache module in the storage architecture of this invention, located between the MRAM main storage area and the GPGPU computing core, integrating the advantages of SRAM and MRAM. The SRAM storage area is used to cache frequently accessed hot data during GPGPU computation, ensuring the computing core can quickly acquire data thanks to its high-speed read / write characteristics. The MRAM storage area is used to store warm data that may be accessed soon, as well as some data with relatively low read / write speed requirements but needing long-term storage. Its non-volatility and large storage capacity reduce frequent data exchange between the main storage area and the cache area, reducing energy consumption. In this embodiment, hot data, warm data, and cold data are determined by the target ratio corresponding to the task data in the current cycle; the preset address index entries in this embodiment are as follows: Figure 3 As shown, it includes memory access address information, memory access count information, and clock count information. The memory access address information includes the memory access address corresponding to each first target task data. The memory access count information is the number of read operations experienced after the memory access address is written into the preset address index entry. The clock count information is the number of clock cycles experienced after the memory access address is written into the preset address index entry. The above target ratio is the ratio of the memory access count information to the clock count information.

[0052] The MRAM main storage area is the core storage module in the storage architecture of this invention. It is constructed using multi-module, multi-layer, three-dimensional stacked MRAM chips, fully utilizing the performance advantages of MRAM storage to achieve high storage density. Simultaneously, in the MRAM main storage area of ​​this invention, RAID5 (a data storage technology) is used to achieve data redundancy backup protection, ensuring data security. By constructing address index entries, the data access frequency of each task data within the current cycle can be determined based on the total number of times the task data is called and the number of cycles elapsed after the task data enters the address index entry. This allows the task data to be stored in the corresponding location of the target storage space according to the data access frequency of the task data.

[0053] Step S12: Determine whether the memory access operation corresponding to each of the first target task data is a write operation. If the memory access operation corresponding to each of the first target task data is a write operation, then store each of the first target task data in the corresponding location of the target storage space according to the space occupancy of the target storage space and the target ratio corresponding to each of the first target task data; wherein, the target storage space includes an MRAM main storage area and a hybrid cache area, and the hybrid cache area includes an SRAM storage area and an MRAM storage area.

[0054] The cache management and replacement strategy in this embodiment is based on an intelligent scheduler and mainly consists of a cache data classification mechanism, a read / write judgment mechanism, and an eviction judgment mechanism. The implementation process is as follows: Figure 4As shown: The GPGPU computing core issues instruction and data flow information for computing tasks. Subsequently, the cache data classification mechanism classifies and identifies the data into hot data, warm data, and cold data. Then, based on the determination of hot data, warm data, and cold data, the read / write judgment mechanism is entered. According to the storage status of the MRAM main memory area / hybrid cache area, it is determined which of the three storage areas—SRAM storage area in the hybrid cache area, MRAM storage area in the hybrid cache area, and MRAM main memory area—to write / read the data. At the same time, if the corresponding cache area is full, the eviction judgment mechanism is also required to determine the storage location of the evicted data.

[0055] The above process is accomplished by the address index entries with a configurable depth of n maintained in the intelligent scheduler. The contents of these entries mainly include memory access address information, memory access count information, clock count information, and storage location information. The configurable depth n is equal to the storable address depth of the SRAM and MRAM storage areas in the hybrid cache. The memory access address information includes the memory access address sent by the GPGPU core to the intelligent scheduler. The information maintained by this entry is constantly refreshed. The memory access count information is the number of read operations that have occurred since the memory access address was written to the address index entry. This location information is cleared with write operations to the address and address changes. The clock count information is the number of clock cycles that have occurred since the memory access address was written to the address index entry. The upper limit is the number of clock cycles set by the system. The number of clock cycles set by the system is a configurable option that can be dynamically configured according to the type of software being executed. This location information is also cleared with write operations to the address and address changes. The storage location information is the physical storage location where the address is stored. It is identified by 00 / 01 / 10 / 11, which respectively represent the SRAM storage area in the hybrid cache, the MRAM storage area in the hybrid cache, the MRAM main storage area, and the invalidated cache.

[0056] Specifically, when the GPGPU starts running, the address index entry information is empty. At this time, the memory access address is stored sequentially into the memory access address information in the address index entry, and the clock count information of the entry is counted starting from this time as zero. At the same time, since there is no data written to the GPGPU cache area at this time, the data, namely the first target task data mentioned above, is marked as hot data. Then, the storage location is marked as 00, and the address data is extracted from the MRAM main storage area and stored in the SRAM storage area of ​​the hybrid cache area.

[0057] When the next memory access operation arrives, the memory access address is first checked. If the memory access address does not match the address information maintained in the existing address index entry, the memory access address is stored sequentially in the direction of increasing address index entry depth, and the subsequent clock cycles are counted and written into the clock count entry. Since the address index entry has not yet increased to the upper limit of the SRAM capacity in the hybrid cache, this data is also marked as hot data, the storage location is marked as 00, and the address data is retrieved from the MRAM main storage area and stored in the SRAM storage area of ​​the hybrid cache. If the memory access address matches the address information maintained in the existing address index entry, a read / write check is performed. If it is a read operation, the memory access count is incremented by 1, and the data is read from the SRAM storage area of ​​the hybrid cache according to the address. If it is a write operation, the memory access count and clock count are cleared to zero, and the data is written to the corresponding address in the SRAM storage area of ​​the hybrid cache and the corresponding address in the MRAM main storage area. By first saving the task data to the SRAM storage area, the data access efficiency in the GPGPU is guaranteed.

[0058] Step S13: After a new memory access operation is obtained, the storage location of the first target task data in the target storage space is periodically and dynamically adjusted according to the target ratio of the corresponding second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space.

[0059] As new memory access operations continue to occur, the address index entry depth will increase to the upper limit of the SRAM capacity in the hybrid cache, indicating that the SRAM storage area in the hybrid cache of the GPGPU cache system is full. Therefore, it is necessary to update the address index entries and replace the GPGPU cache system. Correspondingly, the process of periodically and dynamically adjusting the storage location of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space, may specifically include: if S If the RAM storage area has reached its data storage limit, but the MRAM storage area has not, then the preset address index entries are sorted in ascending order of target ratio. It is then determined whether the first target task data with the smallest target ratio meets the preset hot data judgment criteria within the current preset time period. If the first target task data with the smallest target ratio meets the preset hot data judgment criteria within the current preset time period, then the new memory access address corresponding to the new memory access operation is stored in the preset address index entry, and the second target task data corresponding to the new memory access address is migrated from the MRAM main storage area to the MRAM storage area. If the target ratio... If the smallest first target task data does not meet the preset hot data determination criterion within the current preset time period, then the first target task data with the smallest target ratio is migrated from the SRAM storage area to the MRAM storage area, and the second target task data corresponding to the new memory access address is migrated from the MRAM main storage area to the SRAM storage area. That is, the memory access count / clock count ratios in the existing address index entries are sorted from low to high, and it is determined whether the lowest memory access count / clock count ratio meets the hot data criterion, i.e., the aforementioned preset hot data determination criterion. If it does, the new memory access address is stored in the subsequent address index entry, and subsequent accesses are counted. The clock cycle is written into the clock count entry, the storage location is marked as 01, and the data at that address is extracted from the MRAM main storage area and stored in the MRAM storage area of ​​the hybrid cache. If the hot data standard is not met, the storage address in the address index entry corresponding to the lowest memory access count / clock count ratio is marked as 01, and the data at that address is migrated from the SRAM storage area of ​​the hybrid cache to the MRAM storage area of ​​the hybrid cache. At the same time, the storage location of the new memory access address is marked as 00, and the data corresponding to the new memory access address is extracted from the MRAM main storage area and stored in the space just left by the migration of the SRAM storage area in the hybrid cache.

[0060] When new memory access operations continue to occur, and the address index entry depth increases to the maximum total capacity of SRAM and MRAM in the hybrid cache, it indicates that the SRAM and MRAM storage areas in the GPGPU cache system's hybrid cache are full. After this, the system enters the address index entry update and normal operation phase of the GPGPU cache system's management strategy. Correspondingly, the aforementioned periodic dynamic adjustment of the first target task data's storage location in the target storage space based on the target ratio of the second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space, includes: if the data storage capacity of both the SRAM and MRAM storage areas has reached its limit, determining whether the memory access address corresponding to the new memory access operation is in the preset address index entry; if the memory access address corresponding to the new memory access operation is in the preset address index entry, determining whether the new memory access operation is a read operation; if the new memory access operation is a read operation, then... The target ratio of the second target task data is used to adjust the storage location of the first target task data. If the new memory access operation is a write operation, the second target task data corresponding to the new memory access operation is stored in the MRAM main memory area, the second target task data is deleted from the hybrid cache area, and the storage location corresponding to the second target task data in the preset address index entry is recorded as an invalid cache. Specifically, when a new memory access operation enters the intelligent scheduler, it first determines whether its address is in the address index entry. If it exists, it performs a read / write judgment for the memory access operation. If it is a read operation, it first reads the data from the corresponding hybrid cache area according to the storage location in the address index entry, then updates the memory access count, and calculates its memory access count / clock count ratio. If it is a write operation, it first writes the data to be written to the MRAM main memory area, then enters the corresponding hybrid cache module to clear its data according to the storage location 00 / 01 in its entry, then clears the memory access address, memory access count, and clock count value in the address index entry, and writes the storage location as 11.

[0061] Specifically, the process of adjusting the storage location of the first target task data based on the target ratio of the second target task data corresponding to the new memory access operation may include: if the second target task data corresponding to the new memory access operation is stored in the SRAM storage area, and the target ratio of the second target task data is less than the maximum target ratio of each task data in the MRAM storage area, then the storage area of ​​the second target task data is swapped with the storage area of ​​the first target task data corresponding to the maximum target ratio; if the second target task data corresponding to the new memory access operation is stored in the MRAM storage area, and the target ratio of the second target task data is greater than the number of first target tasks in the SRAM storage area. Based on the corresponding minimum target ratio, the storage area of ​​the second target task data is swapped with the storage area of ​​the first target task data corresponding to the minimum target ratio; that is, if the storage location in the address index entry corresponding to the new memory access operation is 00 and the memory access count / clock count ratio is lower than the maximum memory access count / clock count ratio at the storage location in the address index entry is 01, or the storage location in the address index entry is 01 and the memory access count / clock count ratio is higher than the minimum memory access count / clock count ratio at the storage location in the address index entry is 00, then the storage location values ​​of the two are changed and the two addresses are swapped in the SRAM and MRAM storage areas in the hybrid cache.

[0062] The process of periodically and dynamically adjusting the storage location of the first target task data in the target storage space based on the target ratio of the corresponding second target task data and the storage status of the target storage space, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space, can specifically include: if the memory access address corresponding to the new memory access operation is not in the preset address index entry, then determine whether the new memory access operation is a read operation; if the new memory access operation is a read operation, then determine whether there is an entry in the preset address index entry whose storage location is invalid cache; if there is no entry in the preset address index entry whose storage location is invalid cache, then read the second target task data corresponding to the new memory access operation from the MRAM main storage area; if there is an entry in the preset address index entry whose storage location is invalid cache, then determine whether the occupied space of the first target task data in the SRAM storage area is not greater than the total storage space of the SRAM storage area; if the occupied space of the first target task data in the SRAM storage area is not greater than the total storage space of the SRAM storage area, then... If the new memory access operation is a write operation, the second target task data corresponding to the new memory access operation will be migrated from the MRAM main memory area to the SRAM memory area. If the new memory access operation is a write operation, the second target task data corresponding to the new memory access operation will be stored in the MRAM main memory area. That is, if the address corresponding to the new memory access operation is not in the address index entry, the read / write judgment of the memory access operation will be performed first. If it is a write operation, the data will be directly written to the MRAM main memory area. If it is a read operation, it will be checked whether there is an entry position equal to 11 in the address index entry. If not, the data will be read directly from the MRAM main memory area. If it is, it will be checked whether all entries with storage position equal to 00 in the address index entry meet the requirements of the SRAM storage area. If they meet the requirements, the data will be migrated from the MRAM main memory area to the MRAM area in the hybrid cache and the address index entry storage position will be written to 01. If they do not meet the requirements, the data will be migrated from the MRAM main memory area to the SRAM area in the hybrid cache and the address index entry storage position will be written to 00.

[0063] It should be noted that, in order to ensure the temporal locality of memory access address determination and the absoluteness of hot data, when the clock count of a certain entry in the address index reaches the number of clock cycles set by the system, the present invention will clear the memory access address, memory access reception, and clock count value in that memory access address entry, write invalidation flag 11 to the storage location of the entry, and clear the value maintained in the SRAM or MRAM storage area of ​​the hybrid cache.

[0064] Furthermore, to better adapt to the hybrid cache storage architecture in this invention, the instruction set of the GPGPU computing core is further extended by adding specific instructions for MRAM storage operations. That is, the initial instruction set corresponding to the GPGPU computing core is extended to obtain a corresponding target instruction set, so as to control the corresponding data storage operations of the MRAM main storage area and the MRAM storage area using the target instruction set. The specific instruction table is shown below:

[0065] Table 1 GPGPU Computing Core Extended Instruction Set

[0066]

[0067] Therefore, this application optimizes the storage structure of the GPGPU by constructing a storage area that includes MRAM and SRAM. This allows the MRAM and SRAM storage areas to store data with different access frequencies, solving the problem that SRAM has low integration density and cannot be applied to GPGPUs. It also fully utilizes the advantage of SRAM's fast read and write speed, improving data storage efficiency. By using MRAM and SRAM for data storage, the data loss problem caused by using DRAM for data storage is avoided. By storing data with lower access frequencies in MRAM, frequent data exchange between the main storage area and the cache area is avoided, reducing power consumption.

[0068] Based on the foregoing embodiments, this application describes the overall process of data storage in a GPGPU. Next, this application will elaborate on the data prefetching process in a GPGPU, see [link to relevant documentation]. Figure 5 As shown, an embodiment of the present invention discloses a data prefetching process, including:

[0069] Step S21: Predict the data to be used from the target task data according to the data type and data access frequency of each target task data.

[0070] In this embodiment, to further improve storage performance, a data prefetching and transmission optimization system is constructed. First, the computation task type is determined through the upper-layer cache data classification module. Then, based on the computation task category, dynamic identification is performed to determine the overall data and currently executed data of the current computation task. Data that the GPGPU may need in the next few computation cycles is predicted in advance, and this data is prefetched from the MRAM main storage area to the hybrid cache area. Specifically, taking the deep learning convolutional neural network training process as an example, during the execution of the GPGPU computation core, while the storage architecture of this invention performs front-end hot, warm, and cold data read and write operations, the back-end also predicts the data required for the next layer of convolutional computation based on the parameters of the current network layer and the characteristics of the input data, and loads it into the cache in advance to reduce the time the computation core waits for data. By predicting the data required by the GPGPU and prefetching the data in advance, the data processing efficiency of the GPGPU is improved.

[0071] Step S22: Load the data to be used from the MRAM main storage area to the hybrid cache area so that the GPGPU computing core can process the data to be used.

[0072] In this embodiment, to further adapt to the high concurrency characteristics of GPGPU computing tasks, the physical transmission links between the GPGPU computing core and the intelligent scheduler, hybrid cache, and MRAM main storage area are all organized in units of thread bundles. Thread bundle identification, multi-threaded addresses, and multi-threaded concurrent data transmission are employed. Simultaneously, hot spot, warm spot, and cold spot data identification, data reading, data writing, and data migration are performed in units of thread bundles. Furthermore, within the MRAM main storage area, a multi-module, multi-layer, three-dimensional stacked structure is adopted, where the depth of each module matches the number of threads accommodated by the thread bundle in the transmission link. Multiple independent threads within each thread bundle simultaneously read and write to the multi-layer MRAM storage chips of each MRAM module, achieving concurrent data read and write. By processing data in parallel, the data processing speed is improved.

[0073] Therefore, this application optimizes the storage structure of the GPGPU by constructing a storage area that includes MRAM and SRAM. This allows the MRAM and SRAM storage areas to store data with different access frequencies, solving the problem that SRAM has low integration density and cannot be applied to GPGPUs. It also fully utilizes the advantage of SRAM's fast read and write speed, improving data storage efficiency. By using MRAM and SRAM for data storage, the data loss problem caused by using DRAM for data storage is avoided. By storing data with lower access frequencies in MRAM, frequent data exchange between the main storage area and the cache area is avoided, reducing power consumption.

[0074] See Figure 6 As shown, an embodiment of the present invention discloses a GPGPU storage optimization device, comprising:

[0075] The data acquisition module 11 is used to acquire the first target task data issued by the GPGPU computing core and determine the target ratio corresponding to each of the first target task data based on the preset address index entries; wherein, the preset address index entries include memory access address information, memory access count information and clock count information, the memory access address information includes the memory access address corresponding to each of the first target task data, the memory access count information is the number of read operations experienced after the memory access address is written to the preset address index entry, the clock count information is the number of clock cycles experienced after the memory access address is written to the preset address index entry, and the target ratio is the ratio of the memory access count information to the clock count information;

[0076] The data storage module 12 is used to determine whether the memory access operation corresponding to each of the first target task data is a write operation. If the memory access operation corresponding to each of the first target task data is a write operation, then the data of each of the first target task data is stored in the corresponding location of the target storage space according to the space occupancy of the target storage space and the target ratio corresponding to each of the first target task data. The target storage space includes an MRAM main storage area and a hybrid cache area, and the hybrid cache area includes an SRAM storage area and an MRAM storage area.

[0077] The storage location adjustment module 13 is used to periodically and dynamically adjust the storage location of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage status of the target storage space after a new memory access operation is obtained, so as to store the second target task data corresponding to the new memory access operation in the corresponding location of the target storage space.

[0078] Therefore, this application optimizes the storage structure of the GPGPU by constructing a storage area that includes MRAM and SRAM. This allows the MRAM and SRAM storage areas to store data with different access frequencies, solving the problem that SRAM has low integration density and cannot be applied to GPGPUs. It also fully utilizes the advantage of SRAM's fast read and write speed, improving data storage efficiency. By using MRAM and SRAM for data storage, the data loss problem caused by using DRAM for data storage is avoided. By storing data with lower access frequencies in MRAM, frequent data exchange between the main storage area and the cache area is avoided, reducing power consumption.

[0079] In some specific embodiments, the data storage module 12 may specifically include:

[0080] The index entry sorting unit is used to sort the preset address index entries according to the target ratio in ascending order if the data storage volume of the SRAM storage area has reached the upper limit and the data storage volume of the MRAM storage area has not reached the upper limit, and to determine whether the first target task data with the smallest target ratio meets the preset hot data judgment standard in the current preset time period.

[0081] The first data migration unit is used to store the new memory access address corresponding to the new memory access operation into the preset address index entry if the first target task data with the smallest target ratio meets the preset hot data judgment criteria within the current preset time period, and to migrate the second target task data corresponding to the new memory access address from the MRAM main storage area to the MRAM storage area; wherein, the configurable number of the preset address index entries is the same as the maximum number of tasks stored in the hybrid cache area;

[0082] The second data migration unit is used to migrate the first target task data with the smallest target ratio from the SRAM storage area to the MRAM storage area if the first target task data with the smallest target ratio does not meet the preset hot data judgment criteria within the current preset time period, and to migrate the second target task data corresponding to the new memory access address from the MRAM main storage area to the SRAM storage area.

[0083] In some specific embodiments, the storage location adjustment module 13 may specifically include:

[0084] The memory access address determination unit is used to determine whether the memory access address corresponding to the new memory access operation is in the preset address index entry if the data storage volume of both the SRAM storage area and the MRAM storage area has reached the upper limit.

[0085] The storage location adjustment submodule is used to determine whether the new memory access operation is a read operation if the memory access address corresponding to the new memory access operation is in the preset address index entry. If the new memory access operation is a read operation, the storage location of the first target task data is adjusted according to the target ratio of the second target task data corresponding to the new memory access operation.

[0086] The data deletion unit is used to, if the new memory access operation is a write operation, store the second target task data corresponding to the new memory access operation in the MRAM main memory area, delete the second target task data from the hybrid cache area, and record the storage location corresponding to the second target task data in the preset address index entry as an invalid cache.

[0087] In some specific embodiments, the storage location adjustment submodule may specifically include:

[0088] The first storage area swapping unit is configured to swap the storage area of ​​the second target task data with the storage area of ​​the first target task data corresponding to the maximum target ratio if the second target task data corresponding to the new memory access operation is stored in the SRAM storage area and the target ratio corresponding to the second target task data is less than the maximum target ratio corresponding to each first target task data in the MRAM storage area.

[0089] The second storage area swapping unit is configured to swap the storage area of ​​the second target task data with the storage area of ​​the first target task data corresponding to the minimum target ratio if the second target task data corresponding to the new memory access operation is stored in the MRAM storage area and the target ratio corresponding to the second target task data is greater than the minimum target ratio corresponding to each of the first target task data in the SRAM storage area.

[0090] In some specific embodiments, the storage location adjustment module 13 may specifically include:

[0091] The index entry judgment unit is used to determine whether the new memory access operation is a read operation if the memory access address corresponding to the new memory access operation is not in the preset address index entry, and to determine whether there is an entry in the preset address index entry whose storage location is an invalid cache if the new memory access operation is a read operation.

[0092] The third data migration unit is used to determine whether the space occupied by the first target task data in the SRAM storage area is not greater than the total storage space of the SRAM storage area if there is an entry with an invalid cache location in the preset address index entry. If the space occupied by the first target task data in the SRAM storage area is not greater than the total storage space of the SRAM storage area, the second target task data corresponding to the new memory access operation is migrated from the MRAM main storage area to the SRAM storage area.

[0093] The data storage unit is used to store the second target task data corresponding to the new memory access operation into the MRAM main storage area if the new memory access operation is a write operation.

[0094] In some specific embodiments, the storage location adjustment module 13 further includes:

[0095] The data preloading unit is used to predict the data to be used from the target task data according to the data type and data access frequency of each target task data, and load the data to be used from the MRAM main storage area to the hybrid cache area so that the GPGPU computing core can process the data to be used.

[0096] In some specific embodiments, the GPGPU storage optimization device further includes:

[0097] The instruction set extension module is used to extend the initial instruction set corresponding to the GPGPU computing core to obtain a corresponding target instruction set, so as to use the target instruction set to control the data storage operations of the MRAM main memory area and the corresponding MRAM memory area.

[0098] Furthermore, embodiments of this application also disclose an electronic device, Figure 7 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0099] Figure 7 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the GPGPU storage optimization method disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0100] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0101] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0102] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the GPGPU storage optimization method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0103] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned GPGPU storage optimization method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0104] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0105] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0107] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0108] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for GPGPU memory optimization, comprising: The method comprises: acquiring first target task data issued by a GPGPU computing core, and determining a target ratio corresponding to each of the first target task data based on a preset address index entry; the preset address index entry comprises memory address information, memory count information, and clock count information; the memory address information comprises a memory address corresponding to each of the first target task data; the memory count information is a number of read operations experienced after the memory address is written into the preset address index entry; the clock count information is a number of clock cycles experienced after the memory address is written into the preset address index entry; and the target ratio is a ratio of the memory count information to the clock count information; determining whether a memory operation corresponding to each of the first target task data is a write operation; if the memory operation corresponding to each of the first target task data is a write operation, storing each of the first target task data in a corresponding position of a target storage space according to a space occupation condition of the target storage space and the target ratio corresponding to each of the first target task data; the target storage space comprises an MRAM main storage area and a hybrid cache area; the hybrid cache area comprises an SRAM storage area and an MRAM storage area; after a new memory operation is acquired, periodically and dynamically adjusting a storage position of the first target task data in the target storage space according to a target ratio of corresponding second target task data and a storage state of the target storage space, so as to store the second target task data corresponding to the new memory operation in a corresponding position of the target storage space.

2. The GPGPU storage optimization method of claim 1, wherein, The periodically and dynamically adjusting a storage position of the first target task data in the target storage space according to a target ratio of corresponding second target task data and a storage state of the target storage space, so as to store the second target task data corresponding to the new memory operation in a corresponding position of the target storage space, comprises: if a data storage amount of the SRAM storage area has reached an upper limit and a data storage amount of the MRAM storage area has not reached an upper limit, sorting the preset address index entry according to an order from small to large of the target ratio, and determining whether first target task data with a minimum target ratio meets a preset hot data determination criterion in a preset time period; if the first target task data with the minimum target ratio meets the preset hot data determination criterion in the preset time period, storing a new memory address corresponding to the new memory operation in the preset address index entry, and migrating the second target task data corresponding to the new memory address from the MRAM main storage area to the MRAM storage area; a number of configurable entries of the preset address index entry is the same as a maximum task storage number of the hybrid cache area. If the first target task data with the minimum target ratio does not satisfy the preset hot data determination criterion in the current preset time period, the first target task data with the minimum target ratio is migrated from the SRAM storage area to the MRAM storage area, and the second target task data corresponding to the new memory access address is migrated from the MRAM main storage area to the SRAM storage area.

3. The GPGPU storage optimization method of claim 2, wherein, The periodic dynamic adjustment of the storage position of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage state of the target storage space so as to store the second target task data corresponding to the new memory operation to the corresponding position of the target storage space comprises: If the data storage amounts of the SRAM storage area and the MRAM storage area have reached the upper limit, it is determined whether the memory access address corresponding to the new memory operation is in the preset address index entry; If the memory access address corresponding to the new memory operation is in the preset address index entry, it is determined whether the new memory operation is a read operation, if the new memory operation is a read operation, the storage position of the first target task data is adjusted according to the target ratio of the second target task data corresponding to the new memory operation; If the new memory operation is a write operation, the second target task data corresponding to the new memory operation is stored to the MRAM main storage area, the second target task data is deleted from the hybrid cache area, and the storage position corresponding to the second target task data in the preset address index entry is marked as invalid cache.

4. The GPGPU storage optimization method of claim 3, wherein, The adjustment of the storage position of the first target task data according to the target ratio of the second target task data corresponding to the new memory operation comprises: If the second target task data corresponding to the new memory operation is stored in the SRAM storage area, and the target ratio corresponding to the second target task data is smaller than the maximum target ratio corresponding to each first target task data in the MRAM storage area, the storage area of the second target task data and the storage area of the first target task data corresponding to the maximum target ratio are interchanged; If the second target task data corresponding to the new memory operation is stored in the MRAM storage area, and the target ratio corresponding to the second target task data is greater than the minimum target ratio corresponding to each first target task data in the SRAM storage area, the storage area of the second target task data and the storage area of the first target task data corresponding to the minimum target ratio are interchanged.

5. The GPGPU storage optimization method of claim 3, wherein, The periodic dynamic adjustment of the storage position of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage state of the target storage space so as to store the second target task data corresponding to the new memory operation to the corresponding position of the target storage space comprises: If the memory access address corresponding to the new memory access operation is not in the preset address index entry, it is determined whether the new memory access operation is a read operation. If the new memory access operation is a read operation, it is determined whether there is an entry with an invalid cache storage location in the preset address index entry. If there is no entry with an invalid cache storage location in the preset address index entry, the second target task data corresponding to the new memory access operation is read from the MRAM main storage area. If there is an entry with an invalid cache storage location in the preset address index entry, it is determined whether the occupied space of the first target task data in the SRAM storage area is not greater than the total storage space of the SRAM storage area. If the occupied space of the first target task data in the SRAM storage area is not greater than the total storage space of the SRAM storage area, the second target task data corresponding to the new memory access operation is migrated from the MRAM main storage area to the SRAM storage area. If the new memory access operation is a write operation, the second target task data corresponding to the new memory access operation is stored to the MRAM main storage area.

6. The GPGPU storage optimization method of claim 1, wherein, After storing each target task data to the corresponding position of the target storage space, the method further includes: According to the data type and data access frequency of each target task data, the data to be used is predicted from each target task data, and the data to be used is loaded from the MRAM main storage area to the hybrid cache area, so that the GPGPU computing core processes the data to be used.

7. The GPGPU storage optimization method of any one of claims 1-6, wherein, Further comprising: Extending the initial instruction set corresponding to the GPGPU computing core to obtain a corresponding target instruction set, so as to control the corresponding data storage operation of the MRAM main storage area and the MRAM storage area by using the target instruction set.

8. A GPGPU storage optimization apparatus, comprising: Comprising: A data acquisition module is configured to acquire first target task data issued by a GPGPU computing core, and determine a target ratio corresponding to each first target task data based on a preset address index entry. The preset address index entry includes memory access address information, memory access count information, and clock count information. The memory access address information includes the memory access address corresponding to each first target task data. The memory access count information is the number of read operations experienced after the memory access address is written into the preset address index entry. The clock count information is the number of clock cycles experienced after the memory access address is written into the preset address index entry. The target ratio is the ratio of the memory access count information to the clock count information. A data storage module is configured to determine whether the memory access operation corresponding to each first target task data is a write operation. If the memory access operation corresponding to each first target task data is a write operation, each first target task data is stored to a corresponding position of a target storage space according to the space occupation of the target storage space and the target ratio corresponding to each first target task data. The target storage space includes an MRAM main storage area and a hybrid cache area. The hybrid cache area includes an SRAM storage area and an MRAM storage area. The storage position adjustment module is configured to periodically and dynamically adjust the storage position of the first target task data in the target storage space according to the target ratio of the corresponding second target task data and the storage state of the target storage space after obtaining a new memory access operation, so as to store the second target task data corresponding to the new memory access operation to the corresponding position of the target storage space.

9. An electronic device, comprising: Comprising: a memory for saving a computer program; a processor for executing the computer program to implement the GPGPU storage optimization method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, a memory for saving a computer program, which, when executed by a processor, implements the GPGPU storage optimization method according to any one of claims 1 to 7.

Images