Memory prefetching cooperation method and system, electronic equipment and storage medium

Abstract

The invention discloses a memory prefetching cooperation method and system, electronic equipment and a storage medium, and relates to the field of computer memory access prefetching strategies. The method comprises the steps that a GPU obtains an access address and generates a metadata packet; after the metadata packet is segmented into metadata slices, the ECC check bit of the standard PCIe data packet is replaced, and a complete PCIe data packet is generated and transmitted to a CPU; the CPU analyzes the complete PCIe data packet and then constructs a thermodynamic diagram, updates a thermodynamic value according to the metadata packet and executes periodic attenuation; querying a thermodynamic value based on the analyzed physical address, generating a prefetch address and a memory request packet if a condition is met, and adding the prefetch address and the memory request packet into a GPU request queue; and determining an arbitration sign bit according to the queue depth, the confidence coefficient and the bandwidth utilization rate, and further determining a service queue. Compared with the problems of invisible requests and fixed bandwidth waste caused by address isolation of the GPU and the CPU in the prior art, the method can automatically adapt to access requirements of the GPU and the CPU, avoids CPU response delay, improves efficiency and bandwidth utilization rate, and reduces resource conflicts.

Description

Technical Field

[0001] This invention relates to the field of computer memory access prefetching strategy technology, specifically to a memory prefetching coordination method, system, electronic device, and storage medium. Background Technology

[0002] In computer systems, GPUs and CPUs are core components of heterogeneous computing, and their memory access efficiency directly affects the overall system performance. Specifically, discrete graphics cards (GPUs) and CPUs use independent physical address spaces. The GPU memory controller is integrated within the GPU chip, while the CPU memory controller can only monitor access behavior to local DDR memory and is completely invisible to memory access requests initiated by the GPU via the PCIe bus.

[0003] This address space isolation means that when the CPU needs to process data output by the GPU, it must wait for the GPU to complete the full data transfer via the PCIe bus before it can begin its operation, increasing the response latency of the CPU execution unit. Simultaneously, existing technologies use a fixed-priority quota mechanism for bandwidth allocation between the CPU memory controller and the GPU. When the GPU performs tasks such as large-scale matrix calculations, its prefetcher can continuously occupy up to 90% of the bus bandwidth, causing CPU memory access requests to be blocked in the arbitration queue and unable to receive timely responses. This fixed bandwidth allocation strategy not only exacerbates the conflict between the GPU and CPU in resource competition but also wastes bandwidth resources. When GPU access demand is low, the reserved high quota bandwidth cannot be effectively utilized by the CPU; while when the CPU has urgent requests, the high priority of the GPU further reduces the CPU's efficiency.

[0004] Therefore, breaking down the information barriers between GPUs and CPUs in memory access and achieving dynamic allocation of bandwidth resources to alleviate their competition has become a key issue in improving the performance of heterogeneous computing systems. Summary of the Invention

[0005] The purpose of this invention is to provide a memory prefetching coordination method, system, electronic device, and storage medium to solve the problems mentioned in the background art.

[0006] In a first aspect, one embodiment of this application provides a memory prefetching coordination method, which includes: the GPU acquiring an access address, generating a metadata data packet, the metadata data packet including: a direction flag bit, a step size level, a confidence level, and an address range field; the metadata data packet is a 64-bit binary data packet; the metadata data packet is segmented into metadata slices, and the metadata slices are used to replace the error correction code of the standard Peripheral Component Interconnect Express (PCIe) data packet. The CPU retrieves the code (ECC) checksum and regenerates a complete PCIe data packet, transmitting it to the CPU. The CPU parses the complete PCIe data packet, constructs a heatmap, updates the heatmap value based on the metadata, and performs periodic decay operations. Based on the physical address obtained from parsing the complete PCIe data packet, the CPU queries the heatmap index to obtain the heatmap value. If the heatmap value and confidence level meet preset conditions, a prefetch address and memory request packet are generated and added to the GPU request queue. Based on the queue depths of the GPU and CPU request queues, the confidence level of the memory request packet, and bandwidth utilization, the CPU calculates the confidence gain coefficient, dynamic weight, protection signal, and queue depth ratio. Based on the dynamic weight, protection signal, and queue depth ratio, the CPU determines the arbitration symbol bit. Based on the arbitration symbol bit, the CPU determines the service queue.

[0007] In conjunction with the first aspect, in certain implementations of the first aspect, the metadata data packet is segmented into metadata slices, the ECC check bits of the standard PCIe data packet are replaced with metadata slices, and a complete PCIe data packet is regenerated and transmitted to the CPU. This includes: receiving the metadata data packet; uniformly segmenting the 64-bit data into 8 metadata slices, with the metadata slices starting from slice 0 and ending at slice 7; each metadata slice is an 8-bit binary bit; obtaining the standard PCIe data packet, including the payload and ECC check bits; modifying the highest bit of the ECC check bits to an identifier bit based on the ECC check bits and the metadata slices to obtain the highest bit of the new ECC field; each ECC check bit is an 8-bit binary bit; storing the data of the metadata slices into the lower 7 bits of the new ECC field; grouping every 8 consecutive standard PCIe data packets into a group and embedding each group into slices 0 to 7 of the metadata data packet; inserting a type identifier into the header of the modified standard PCIe data packet; calculating a Cyclic Redundancy Check (CRC) code based on the payload and the new ECC field; and assembling the type identifier, payload, new ECC field, and CRC code into a complete PCIe data packet.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the CPU parses the complete PCIe data packet, constructs a heatmap, and updates the heatmap value based on the metadata. Simultaneously, it performs a periodic decay operation, including: parsing the complete PCIe data packet to obtain the direction flag, step size level, confidence level, and address range field; the address range field is a 58-bit binary field; calculating the step size exponent based on the step size level, where the step size exponent is the binary value of the step size level minus 1; calculating the heatmap increment based on the confidence level and the step size exponent, where the heatmap increment is determined by the product of the confidence level and 2 raised to the power of the step size exponent; extracting the base address from the high 52 bits of the address range field, and using hash mapping to calculate... The block index is determined by taking the remainder of the lower 6 bits of the address range field and the upper limit of the heatmap's index entries; the block index is the index of the heatmap. The block index is output to the heatmap. Based on the block index, the current heat value is read. A new heat value is calculated based on the block index, heat increment, and current heat value. The new heat value is determined by the sum of the current heat value and the heat increment, with an upper limit of 255. The new heat value is written back to the position specified by the block index in the heatmap. After a preset period, all block indices are traversed, the current heat value at the position specified by the block index is read, and the decayed heat value is calculated. The decayed heat value is three-quarters of the current heat value. The decayed heat value is written back to the position specified by the block index in the heatmap.

[0009] In conjunction with the first aspect, in certain implementations of the first aspect, based on the queue depths of the GPU request queue and the CPU request queue, the confidence level of memory request packets, and bandwidth utilization, a confidence gain coefficient, dynamic weight, protection signal, and queue depth ratio are calculated; an arbitration sign bit is determined based on the dynamic weight, protection signal, and queue depth ratio; and a service queue is determined based on the arbitration sign bit, including: obtaining the GPU request queue depth, CPU request queue depth, confidence level of memory request packets, and bandwidth utilization; calculating the confidence gain coefficient based on the confidence level; and calculating the dynamic weight of GPU request queue accessing memory based on the confidence gain coefficient and the base weight. The base weight is a base coefficient set within a preset value range. If the bandwidth utilization is greater than the preset bandwidth threshold, the protection signal is set to 1. If the bandwidth utilization is less than or equal to the preset bandwidth threshold, the protection signal is set to 0. If the protection signal is 1, the CPU request queue is selected and the arbitration sign bit is set to 10. If the protection signal is 0 and the ratio of the GPU request queue depth to the total queue depth is less than or equal to the dynamic weight, the CPU request queue is selected and the arbitration sign bit is set to 10. If the protection signal is 0 and the ratio of the GPU request queue depth to the total queue depth is greater than the dynamic weight, the GPU request queue is selected and the arbitration sign bit is set to 01.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the GPU obtains the access address and generates a meta-data packet. The meta-data packet includes: a direction flag, a step size level, a confidence level, and an address range field. The meta-data packet is a 64-bit binary file, including: obtaining the current access address and the access address of the previous cycle, and calculating the difference between consecutive addresses; calculating the step size based on the absolute value of the difference between consecutive addresses, and quantizing the step size into a step size level; counting the number of positive and negative differences based on the difference between consecutive addresses to determine the direction flag; obtaining prefetch records from historical prefetch records, calculating the prefetch hit rate, and calculating the confidence level based on the prefetch hit rate; extracting the lower 6 bits of the address based on the current access address as the base address, and generating an address range field by combining the step size level; and concatenating the direction flag, step size level, confidence level, and address range field into a meta-data packet.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, the GPU obtains the access address and generates a metadata packet. The metadata packet includes: a direction flag bit, a step size level, a confidence level, and an address range field. The metadata packet is a 64-bit binary data packet, including: quantizing the step size to a preset step size level, with the step size level stored using 2 bits; if the number of positive difference values ​​is greater than or equal to the number of negative difference values, the direction flag bit is 0; if the number of positive difference values ​​is less than the number of negative difference values, the direction flag bit is 1; the confidence level is a linear mapping of the prefetch hit rate to levels 0-7; the address range field is obtained by left-shifting the base address and step size level by 58 bits and performing a bitwise logical OR operation.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, based on the physical address obtained by parsing the complete PCIe data packet, a heatmap index is queried to obtain a heat value. If the heat value and confidence level meet preset conditions, a prefetch address and a memory request packet are generated and added to the GPU request queue. This includes: parsing the complete PCIe data packet to obtain the physical address, calculating the block index and mapping it to the heatmap index, and querying to obtain the heat value; if the heat value is greater than a preset activation threshold and the confidence level is greater than or equal to 5, a prefetch address is calculated based on the physical address, direction flag, and step size level; a priority flag and a prefetch type identifier are added before the prefetch address to generate a memory request packet; if there are no incomplete requests with the same prefetch address in the GPU request queue, the memory request packet is added to the tail of the GPU request queue; if there are incomplete requests with the same prefetch address, the memory request packet in the GPU request queue is moved forward one line.

[0013] Secondly, one embodiment of this application provides a memory prefetching coordination system for implementing the memory prefetching coordination method mentioned in the first aspect. The system includes: a lightweight prefetch heuristic unit (PHU) module for acquiring access address information, calculating direction flag bits, step size level, confidence level, and address range fields, and generating a 64-bit metadata data packet; a metadata data transmission module for segmenting the 64-bit metadata data packet into metadata slices, embedding ECC check bits of the PCIe data packet, and assembling them into a complete PCIe data packet; a PCIe parsing module for parsing the complete PCIe data packet on the CPU side, constructing a heatmap, and performing heatmap value updates and periodic decay operations; a prefetch address generation module for calculating the prefetch address and generating a memory request packet based on the physical address and heatmap information, and adding it to the GPU request queue; and an arbitration decision module for calculating the arbitration symbol bit based on queue depth, confidence level, and bandwidth utilization, and determining the service queue.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, including a processor and a memory, wherein the memory stores a computer program that can be executed by the processor, and the processor executes the computer program to implement a memory prefetching coordination method mentioned in the first aspect above.

[0015] Fourthly, embodiments of the present invention also provide a storage medium storing a computer program, which, when executed by a processor, implements a memory prefetching coordination method mentioned in the first aspect above.

[0016] Beneficial Effects: The memory prefetching coordination method, system, electronic device, and storage medium provided in this application generate metadata containing address features through a PHU module, embed PCIe data packets through a metadata data transmission module to achieve cross-address space communication, dynamically update access features using a heatmap constructed by a PCIe parsing module, and then implement prefetching control and dynamic bandwidth allocation based on heatmap values ​​and confidence levels through a prefetching address generation module and an arbitration decision module. Compared with the problems of request invisibility caused by GPU and CPU address space isolation and resource waste caused by fixed bandwidth allocation in traditional technologies, this application can automatically adapt to the access needs of GPU and CPU, avoid CPU response delay caused by waiting for data or bandwidth blockage, improve CPU working efficiency, increase bandwidth utilization, and reduce resource idleness and conflicts. Attached Figure Description

[0017] Figure 1 The diagram shown is a flowchart of a memory prefetching coordination method provided in an embodiment of this application.

[0018] Figure 2The diagram shown is a flowchart of a memory prefetching coordination method provided in another exemplary embodiment of this application.

[0019] Figure 3 The diagram shown is a flowchart of a memory prefetching coordination method provided in another exemplary embodiment of this application.

[0020] Figure 4 The diagram shown is a flowchart of a memory prefetching coordination method provided in another exemplary embodiment of this application.

[0021] Figure 5 The diagram shown is a flowchart of a memory prefetching coordination method provided in another exemplary embodiment of this application.

[0022] Figure 6 The diagram shown is a flowchart of a memory prefetching coordination method provided in another exemplary embodiment of this application.

[0023] Figure 7 The diagram shown is a schematic representation of a memory prefetching coordination system provided by an exemplary embodiment of the present invention.

[0024] Figure 8 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0025] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0026] The following is combined Figures 1 to 6 This application provides a detailed description of a memory prefetching coordination method.

[0027] Figure 1 The diagram shown is a flowchart illustrating a memory prefetching coordination method according to an embodiment of this application. Figure 1 As shown in the embodiment of this application, a memory prefetching coordination method is provided, which includes the following steps.

[0028] Step 100: The GPU obtains the access address and generates a meta data packet.

[0029] Specifically, the GPU obtains the current access address and the previous access address, calculates the direction flag, step size level, confidence level, and address range field based on the current access address and the previous access address, and then concatenates them to obtain the meta data packet, which is a 64-bit binary data packet.

[0030] Step 101: Divide the metadata data packet into metadata slices, replace the ECC check bits of the standard PCIe data packet with the metadata slices, and regenerate the complete PCIe data packet and transmit it to the CPU.

[0031] For example, a standard PCIe packet includes a 128-bit payload and an 8-bit ECC checksum.

[0032] For example, a complete PCIe packet consists of a type identifier, payload, new ECC field, and CRC checksum.

[0033] Step 102: The CPU parses the complete PCIe data packet, constructs a heatmap, updates the heatmap value based on the metadata, and performs a periodic decay operation.

[0034] Specifically, a heatmap is constructed on the CPU side. Based on the received complete PCIe data packets, block indices are calculated and mapped to index entries in the heatmap. The heatmap value is updated based on the block index, heatmap increment, and current heatmap value. The decayed heatmap value is calculated based on the block index and current heatmap value. The heatmap value update operation is performed when each memory access request falls on the corresponding index entry in the heatmap. The decay operation is performed by traversing all index entries in the heatmap, and the decay operation is performed only once for each index entry per clock cycle.

[0035] Step 103: Based on the physical address obtained by parsing the complete PCIe data packet, query the heatmap index to obtain the heat value. If the heat value and confidence level meet the preset conditions, generate the prefetch address and memory request packet and add it to the GPU request queue.

[0036] For example, the prefetch address is calculated from the physical address, direction sign, and step size displacement.

[0037] For example, the memory request packet includes a prefetch type identifier, a priority flag, and a prefetch address.

[0038] Step 104: Calculate the confidence gain coefficient, dynamic weight, protection signal, and queue depth ratio based on the queue depth of the GPU request queue and the CPU request queue, the confidence of the memory request packet, and the bandwidth utilization; determine the arbitration symbol bit based on the dynamic weight, protection signal, and queue depth ratio; and determine the service queue based on the arbitration symbol bit.

[0039] This application provides a memory prefetching coordination method. It generates a 64-bit data packet containing address features using a GPU, embeds it into a PCIe data packet via metadata slicing, and transmits it to the CPU. The CPU parses the packet, constructs a heatmap, and dynamically updates the heatmap value. Prefetching requests are generated by combining the heatmap value with confidence levels. Finally, the service queue is determined based on queue status and bandwidth utilization. Compared to traditional technologies where GPU and CPU address space isolation leads to request invisibility and resource waste caused by fixed bandwidth allocation, this application can automatically adapt to access requirements in heterogeneous computing scenarios, avoid CPU response delays due to request blocking, improve bandwidth utilization, and reduce resource idleness and conflicts.

[0040] Figure 2 The diagram shown is a flowchart illustrating a memory prefetching coordination method provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 2 The illustrated embodiment will be described in detail below. Figure 2 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0041] like Figure 2 As shown in the embodiment of this application, a memory prefetching coordination method is provided, in which the metadata data packet is divided into metadata slices, the ECC check bits of the standard PCIe data packet are replaced with metadata slices, and the complete PCIe data packet is regenerated and transmitted to the CPU (step 101), including the following steps.

[0042] Step 200: Receive the metadata data packet and evenly divide the 64-bit data into 8 metadata slices.

[0043] Specifically, the 64-bit data is evenly divided into eight 8-bit metadata slices, numbered 0 to 7. Starting from slice 0, each slice corresponds to the highest 8 bits of the metadata data packet.

[0044] Step 201: Obtain a standard PCIe data packet, including the payload and ECC check bit. Based on the ECC check bit and the metadata slice, modify the highest bit of the ECC check bit to an identifier bit to obtain the highest bit of the new ECC field.

[0045] For example, a standard PCIe packet originates from a PCIe write request packet sent by the GPU memory controller to the CPU, with an ECC checksum bit used to detect and correct errors in system memory or stored data.

[0046] Step 202: Store the data of the metadata slice into the lower 7 bits of the new ECC field.

[0047] Step 203: Each group of 8 consecutive standard PCIe packets is embedded into slices 0 to 7 of the metadata packet.

[0048] For example, standard PCIe packet 0 is embedded in slice 0, standard PCIe packet 1 is embedded in slice 1, and so on.

[0049] Step 204: For the modified standard PCIe data packet, insert a type identifier into the packet header, calculate the CRC checksum based on the payload and the new ECC field, and assemble the type identifier, payload, new ECC field and CRC checksum into a complete PCIe data packet.

[0050] Specifically, a complete PCIe packet is 168 bits long, composed of a 4-bit type identifier, a 124-bit payload, an 8-bit new ECC field, and a 32-bit CRC checksum. This 168-bit complete PCIe packet is transmitted to the Physical Layer, converted into a serial bit stream, and then sent as an electrical signal through the differential lines of the PCIe channel, ultimately outputting to the CPU's PCIe parsing module.

[0051] In one specific embodiment, the highest bit of the new ECC field is calculated using formula (1), which is: New ECC[7]=1; The modification of the highest bit of the ECC check bit is to add metadata identifiers. A highest bit of 1 indicates that metadata is carried, while a highest bit of 0 indicates that no metadata is carried, which is a normal ECC check bit.

[0052] A 4-bit type identifier is inserted into the data packet header and calculated using formula (2), which is: PCIe[127:124]=0xB; Here, 0xB represents a predefined metadata type; the 4-bit type identifier comes from the 128-bit payload, so the payload becomes 124 bits.

[0053] The 32-bit CRC checksum is calculated using formula (3), which is: CRC = f(payload || new ECC); f(x) = x 32 +x 26 +x 23 +x 22 +x 16 +x 12 +x 11 +x 10 +x 8 +x 7 +x 5 +x 4 +x2 +x+1; Where || represents binary bit concatenation, and the function f(x) represents the polynomial for calculating the 32-bit CRC.

[0054] The memory prefetching coordination method provided in this application overcomes the semantic invisibility caused by GPU-CPU address space isolation and avoids adding new transmission pins by reusing PCIe data packet ECC check bits to embed metadata slices and extending the PCIe protocol type header, thereby reducing transmission latency.

[0055] Figure 3 The diagram shown is a flowchart illustrating a memory prefetching coordination method provided in another exemplary embodiment of this application. Figure 2 This application extends from the embodiments shown. Figure 3 The illustrated embodiment will be described in detail below. Figure 3 The illustrated embodiments and Figure 2 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0056] like Figure 3 As shown in the embodiment of this application, in a memory prefetching coordination method, after inserting a type identifier into the header of the modified standard PCIe data packet, calculating the CRC checksum based on the payload and the new ECC field, and assembling the type identifier, payload, new ECC field, and CRC checksum into a complete PCIe data packet (step 204), the method further includes the following steps: Step 300: For the complete PCIe data packet, parse and obtain the direction flag, step size level, confidence level, and address range fields.

[0057] Specifically, the PCIe parsing module parses the received 168-bit complete PCIe data packet to obtain a 1-bit direction flag (stride_dir), a 2-bit stride magnitude level (stride_mag), a 3-bit position confidence level (Confidence), and a 58-bit address range field (addr_range). The obtained data is input into the register group in binary form for subsequent bit operations.

[0058] Step 301: Calculate the step size index based on the step size level.

[0059] The step size index is calculated using formula (4), which is: Step size index = stride_mag binary bits - 1; Where stride_mag is the step size level, stride_mag is stored in 2 bits, so when stride_mag=2, the step size exponent is 1, when stride_mag=3, the step size exponent is 2, and when stride_mag=4, the step size exponent is 3.

[0060] Step 302: Calculate the thermal increment based on the confidence level and the step size exponent.

[0061] The thermal increment ΔH is calculated using formula (5), which is: ΔH = Confidence × (2^step exponent); For example, the maximum confidence level is 7 and the maximum step size exponent is 3, so the maximum ΔH is 56, and the thermal increment ΔH can be represented by 6 bits.

[0062] Step 303: Take the high 52 bits of the address range field, extract the base address, and use hash mapping to calculate the block index.

[0063] The block index block_id is calculated using formula (6), which is: block_id= addr_range[57:6] mod 1024; Here, addr_range represents a 58-bit address range field, with the lower 6 bits of addr_range aligned to 0; mod is the modulo operator; 1024 represents the upper limit of the heatmap index entries.

[0064] For example, the memory range covered by the heatmap is limited, therefore, memory addresses need to be mapped to a limited number of heatmap entries.

[0065] Step 304: Output the block index to the heat map, read the current heat value based on the block index, calculate the new heat value based on the block index, heat increment and the current heat value, and write the new heat value back to the position specified by the block index in the heat map.

[0066] Specifically, the Static Random-Access Memory (SRAM) of the heatmap is a dual-port memory that stores heatmap data. The updating and decaying of the heatmap need to handle the problem of concurrent access. The dual-port design uses one port for metadata updates and the other port for periodic decay. Specifically, when update and decay operations access the same entry simultaneously within a clock cycle, the dual ports can read and write to different addresses at the same time, thus enabling concurrent processing.

[0067] The new thermal value new_H is calculated using formula (7), which is: new_H = min(255, current_H+ΔH); Where current_H represents the current heat value. If the updated heat value new_H exceeds 255, then let new_H = 255.

[0068] Step 305: After a preset period, traverse all block indices, read the current thermal value at the specified position of the block index, calculate the attenuated thermal value, and write the attenuated thermal value back to the specified position of the block index in the heat map.

[0069] Specifically, when the lower 10 bits of the system clock cycle_count are 0, that is, every 1024 cycles, all blocks are traversed, 1024 SRAM addresses block_id are accessed sequentially, and the current heat value current_H is read.

[0070] The decayed thermal value old_H is calculated using formula (8), which is: old_H=(current_H>>1)+(current_H>>2); Here, >>1 means shifting the binary bit to the right by 1 bit; old_H is equivalent to 3 / 4 of current_H.

[0071] The memory prefetching coordination method provided in this application overcomes the prediction bias caused by the long-term residence of historical access patterns through a confidence-weighted index hot update and periodic decay mechanism, thereby reducing the misjudgment rate of memory access areas.

[0072] Figure 4 The diagram shown is a flowchart illustrating a memory prefetching coordination method provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 4 The illustrated embodiment will be described in detail below. Figure 4 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0073] like Figure 4 As shown in the embodiment of this application, a memory prefetching coordination method is provided, which calculates the confidence gain coefficient, dynamic weight, protection signal, and queue depth ratio based on the queue depths of the GPU request queue and the CPU request queue, the confidence of the memory request packet, and the bandwidth utilization; determines the arbitration symbol bit based on the dynamic weight, protection signal, and queue depth ratio; and determines the service queue based on the arbitration symbol bit (step 104), including the following steps: Step 400: Obtain the GPU request queue depth, CPU request queue depth, confidence level of memory request packets, and bandwidth utilization.

[0074] Specifically, bandwidth utilization is the ratio of used bandwidth to total bandwidth.

[0075] Step 401: Calculate the confidence gain coefficient based on the confidence level.

[0076] The confidence gain coefficient α is calculated using formula (9), which is: α = Confidence / 8; For example, α∈[0, 0.875], increasing the confidence level is important for GPU memory access requests. This is consistent with the fact that confidence level calculation is related to prefetch hit rate. The higher the prefetch hit rate, the more important the prefetched data is in the GPU and the more priority it needs to be executed.

[0077] Step 402: Combine the confidence gain coefficient and the basic weight to calculate the dynamic weight of GPU request queue access to memory.

[0078] The dynamic weight w_gpu is calculated using formula (10), which is: w_gpu = w_base × (1 + α); Among them, w_base is the base weight, which can be set within the range of [0,1]. The default value in this application is 0.7.

[0079] Step 403: Determine the service request queue based on the bandwidth utilization and the relationship between the queue depth ratio and the dynamic weight.

[0080] Specifically, if the bandwidth utilization > 0.85, the protection signal safe_guard is triggered and safe_guard = 1; if the bandwidth utilization ≤ 0.85, the protection signal safe_guard is not triggered and safe_guard = 0.

[0081] Furthermore, if safe_guard=1, then the CPU request queue is selected and the arbitration sign bit Arbitration_sign=10; if safe_guard=0 and ratio≤w_gpu, then the CPU request queue is selected and the arbitration sign bit Arbitration_sign=10; if safe_guard=0 and ratio>w_gpu, then the GPU request queue is selected and the arbitration sign bit Arbitration_sign=01; the arbitration sign bit Arbitration_sign is represented by 2 binary bits, where '10' represents the CPU queue and '01' represents the GPU request queue.

[0082] For example, the arbitration decision module outputs the arbitration sign bit Arbitration_sign to the physical layer (PHY) interface module of the CPU memory controller.

[0083] The present application provides a memory prefetching coordination method that overcomes the blocking of CPU resources by GPU streaming prefetching through confidence-driven dynamic weight setting and bandwidth over-limit protection, thereby improving CPU efficiency.

[0084] Figure 5 The diagram shown is a flowchart illustrating a memory prefetching coordination method provided in another exemplary embodiment of this application. Figure 1 This application extends from the embodiments shown. Figure 5 The illustrated embodiment will be described in detail below. Figure 5 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0085] like Figure 5 As shown in the embodiment of this application, in a memory prefetching coordination method, the GPU obtains the access address and generates a metadata packet. The metadata packet includes: a direction flag, a step size level, a confidence level, and an address range field (step 100), including the following steps: Step 500: Obtain the current access address and the access address of the previous cycle, and calculate the difference between consecutive addresses.

[0086] Specifically, the PHU module captures memory access requests in real time, obtains the current access address addr_current and the previous cycle access address addr_previous, and stores the current access address addr_current in a 32-bit circular buffer. The PHU is contained within the GPU memory controller. The access address is a 64-bit physical address. One cycle refers to one clock cycle, which includes a rising edge and a falling edge. The access address is obtained on the rising edge of each cycle. The previous cycle access address addr_previous is read from the history register. The circular buffer uses a first-in-first-out queue.

[0087] The difference between consecutive addresses, diff, is calculated using formula (11), which is: diff=addr_current- addr_previous; The address difference, diff, is also a 64-bit binary value.

[0088] Step 501: Calculate the step size based on the absolute value of the consecutive address differences, and quantize the step size into step size levels.

[0089] The stride is calculated using formula (12), which is: stride=(abs_diff[0]+abs_diff[1]+...+abs_diff

[31] ) / 32; Among them, abs_diff[0, 1, ..., 31] is an absolute value sequence. One address is processed in each clock cycle, and metadata generation is triggered once every 32 cycles. Therefore, 32 consecutive address difference diffs are read. The address difference diffs are read from the circular buffer.

[0090] Specifically, the stride is quantized to a preset stride level, stride_mag, which is stored using 2 bits: if the stride ∈ [0B, 64B], then the stride level stride_mag is set to 1; if the stride ∈ (64B, 128B], then the stride level stride_mag is set to 2; if the stride ∈ (128B, 256B], then the stride level stride_mag is set to 3; if the stride ∈ (256B, ∞), then the stride level stride_mag is set to 4.

[0091] For example, stride_mag is 1, stored as 00, indicating spanning 1 cache line; stride_mag is 2, stored as 01, indicating spanning 2 cache lines; stride_mag is 3, stored as 10, indicating spanning 4 cache lines; stride_mag is 4, stored as 11, indicating spanning 8 cache lines. Here, a cache line is the smallest block of data stored in the circular buffer (Cache). When the CPU reads data from memory, it does not read it one byte or one unit at a time, but loads the entire block of data into the circular buffer at once, in units of "cache lines".

[0092] Step 502: Based on the consecutive address differences, count the number of positive and negative differences to determine the direction flag bit.

[0093] Specifically, if the number of positive differences pos_count ≥ the number of negative differences neg_count, it indicates an increasing direction, and the direction flag stride_dir=0; if the number of positive differences pos_count < the number of negative differences neg_count, it indicates a decreasing direction, and the direction flag stride_dir=1.

[0094] Step 503: Obtain prefetch records from historical prefetch records, calculate the prefetch hit rate, and calculate the confidence level based on the prefetch hit rate.

[0095] The prefetch hit rate P_prefetch is calculated using formula (13), which is: P_prefetch=hit / 256; Here, hit represents the number of prefetch hits in the most recent 256 prefetch records.

[0096] For example, the historical prefetch records are stored in a 256-bit shift register, where each bit represents one prefetch record, with 1 for a hit and 0 for a miss.

[0097] The prefetch hit rate P_prefetch is linearly mapped to levels 0 to 7, and the confidence level is calculated using formula (14): Confidence=floor(8×P_prefetch); Here, the function floor(u) is the floor function, and the variable u is the object to be floored; when Confidence > 7, then let Confidence = 7.

[0098] Step 504: Extract the lower 6 bits of the address based on the current access address as the base address, and generate an address range field by combining the step size level; concatenate the direction flag, step size level, confidence level, and address range field into a meta-data packet.

[0099] Specifically, for the current access address addr_current (64 bits), the lower 6 bits are truncated as the base address base_addr. The base address base_addr is calculated using formula (15), which is: base_addr=addr_current AND 0xFFFFFFFFFFFFFFFC0; AND indicates a bitwise logical AND operation on binary bits; the binary form of 0xFFFFFF - FFFFFFFFC0 is that the lowest 6 bits are 0 and the rest are 1.

[0100] Specifically, based on the base address base_addr and the stride magnitude level stride_mag, the stride information is combined to obtain the address range field addr_range. The address range field addr_range is calculated using formula (16), which is: addr_range=base_addr OR(stride_mag<<58); Here, OR represents performing a bitwise logical OR operation on binary bits; <<58 represents shifting left by 58 bits.

[0101] Specifically, the metadata is calculated using formula (17), which is: Metadata packet = [stride_dir][stride_mag][Confidence][addr_range]; In this context, the 63rd position corresponds to the direction flag bit stride_dir, the 62nd-61st positions correspond to the stride magnitude level stride_mag, the 60th-58th positions correspond to the confidence level, and the 57th-0th positions correspond to the address range field addr_range.

[0102] This application provides a memory prefetching coordination method that, by analyzing the address difference sequence of the GPU memory access flow in real time and quantizing the step size characteristics and confidence, solves the problem that traditional prefetchers cannot capture non-linear GPU access, thereby improving prefetching accuracy.

[0103] Figure 6 The diagram shown is a flowchart illustrating a memory prefetching coordination method provided in another exemplary embodiment of this application. Figure 1This application extends from the embodiments shown. Figure 6 The illustrated embodiment will be described in detail below. Figure 6 The illustrated embodiments and Figure 1 The differences between the embodiments shown are not repeated here, and the similarities are not repeated here.

[0104] like Figure 6 As shown in the embodiment of this application, in a memory prefetching coordination method, based on the physical address obtained by parsing a complete PCIe packet, a heatmap index is queried to obtain a heat value. If the heat value and the confidence level meet preset conditions, a prefetch address and a memory request packet are generated and added to the GPU request queue (step 103), including the following steps: Step 600: Parse the complete PCIe data packet to obtain the physical address, calculate the block index and map it to the heat map index, and query to obtain the heat value.

[0105] Specifically, the physical address addr_cpu is obtained from parsing a complete 168-bit PCIe packet, and the block index block_id is calculated using formula (18), which is: block_id=floor(addr_cpu / 64); Here, the function floor(u) is the floor function, and the variable u is the object to be floored.

[0106] Specifically, the heatmap index (heatmap_index) is calculated using formula (19), which is: heatmap_index=block_id mod 1024; The maximum remainder of heatmap_index is 1023, so it requires 10 bits to represent.

[0107] Specifically, the heatmap index heatmap_index is accessed at a specified address in the heatmap SRAM to obtain the heat value H. The heat value H is calculated using formula (20), which is: H = SRAM[heatmap_index]; The thermal value H is an 8-bit unsigned binary number.

[0108] Step 601: If the heat value is greater than the preset activation threshold and the confidence level is greater than or equal to 5, calculate the prefetch address based on the physical address, direction flag bit and step size level; add priority flag and prefetch type identifier before the prefetch address to generate a memory request packet.

[0109] Specifically, the preset activation threshold T_active is set to 60 by default (maximum 255), and the prefetch address prefetch_addr is calculated using formula (21), which is: prefetch_addr=addr_cpu+sign×stride_offset; Where addr_cpu is the physical address, stride_offset is the step size displacement, and sign is the direction sign.

[0110] Specifically, the stride displacement amount stride_offset is calculated using the stride magnitude level stride_mag. The stride displacement amount stride_offset is calculated using formula (22), which is: stride_offset=stride_mag×cache_line×stride_prefetch; Here, cache_line represents the cache line size, and most caches on the market have a cache line size of 64B; stride_prefetch represents the number of lookahead steps, which can be set, and the default value in this application is 4.

[0111] Specifically, the direction symbol `sign` is calculated using the direction flag `stride_dir`. If `stride_dir = 0`, the direction symbol `sign` is set to `+1`; if `stride_dir = 1`, the direction symbol `sign` is set to `-1`.

[0112] Specifically, add a priority flag PRIORITY before the prefetch_addr and set PRIORITY=LOW, and add a prefetch type flag REQ_TYPE before the priority flag PRIORITY and set REQ_TYPE=PREFETCH.

[0113] For example, the priority is set to low because the request comes from the GPU. As explained in the background, GPU requests are mostly stream prefetching, i.e., large-scale continuous access. If a high priority is set, prioritizing processing will block other "short and fast" requests from the CPU, reducing CPU efficiency. Therefore, the priority is set to low.

[0114] Step 602: Add the memory request packet to the GPU request queue.

[0115] Specifically, memory request packets are added to the GPU request queue. Before entering the queue, a check is performed. If an incomplete request with the same prefetch address already exists, the memory request packet that has not been added is deleted, and the existing memory request packet is moved up one line in the queue. If no incomplete request with the same prefetch address exists, it is added to the tail of the GPU request queue.

[0116] For example, the GPU request queue uses a FIFO queue with a queue depth of Q_gpu of 31 (which can store up to 32 memory request packets).

[0117] The memory prefetching coordination method provided in this application overcomes the bandwidth abuse caused by aggressive prefetching by triggering a prefetching mechanism under two conditions: heat value and confidence level, thereby improving bandwidth utilization and prefetching accuracy.

[0118] Figure 7 The diagram shown is a schematic representation of a memory prefetching coordination system provided in an exemplary embodiment of this application. Figure 7 As shown in the figure, a memory prefetching coordination system provided in this application includes: a PHU module 700, a metadata data transmission module 701, a PCIe parsing module 702, a prefetch address generation module 703, and an arbitration decision module 704.

[0119] The PHU module 700 acquires access address information, calculates the direction flag, step size level, confidence level, and address range fields, and generates a 64-bit metadata data packet. The metadata data transmission module 701 segments the 64-bit metadata data packet into metadata slices, embeds the ECC checksum of the PCIe data packet, and assembles them into a complete PCIe data packet. The PCIe parsing module 702 parses the complete PCIe data packet on the CPU side, constructs a heatmap, and performs heatmap value updates and periodic decay operations. The prefetch address generation module 703 generates prefetch addresses and memory request packets based on the physical address and heatmap information, and adds them to the GPU request queue. The arbitration decision module 704 calculates the arbitration symbol bit based on the queue status, confidence level, and bandwidth utilization to determine the service queue.

[0120] Figure 8 The diagram shown is a structural schematic of an electronic device provided in an exemplary embodiment of this application. Figure 8 As shown, the electronic device 800 includes a processor 801 and a memory 802.

[0121] The processor 801 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 800 to perform desired functions.

[0122] The memory 802 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory and / or cache memory. Non-volatile memory may include, for example, read-only memory, hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 801 may execute the program instructions to implement a memory prefetching coordination method and / or other desired functions according to the various embodiments of this application described above.

[0123] In one example, the electronic device 800 may also include an input device 803 and an output device 804, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0124] The input device 803 may include, for example, a keyboard, a mouse, etc.

[0125] The output device 804 can output various information to the outside. The output device 804 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0126] Of course, for the sake of simplicity, Figure 8 Only some of the components of the electronic device 800 relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device 800 may include any other suitable components depending on the specific application.

[0127] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps of a memory prefetching coordination method according to various embodiments of this application as described in the "Exemplary Methods" section of this specification.

[0128] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0129] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform steps in a memory prefetching coordination method according to various embodiments of this application as described in the "Exemplary Methods" section above.

[0130] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0131] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0132] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0133] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0134] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features of the invention herein.

[0135] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of this application to the forms of the invention herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

[0136] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications or equivalent substitutions made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A memory prefetching coordination method, characterized in that, include: The GPU obtains the access address and generates a metadata packet, which includes: a direction flag, a step size level, a confidence level, and an address range field; the metadata packet is a 64-bit binary data packet. The metadata data packet is divided into metadata slices, the metadata slices are used to replace the ECC check bits of the standard PCIe data packet, and a complete PCIe data packet is regenerated and transmitted to the CPU. The CPU parses the complete PCIe data packet, constructs a heatmap, updates the heatmap value based on the meta-data packet, and performs a periodic decay operation. Based on the physical address obtained by parsing the complete PCIe data packet, the heat map index is queried to obtain the heat value. If the heat value and the confidence level meet the preset conditions, a prefetch address and memory request packet are generated and added to the GPU request queue. Based on the queue depths of the GPU request queue and the CPU request queue, the confidence level of the memory request packet, and the bandwidth utilization, calculate the confidence gain coefficient, dynamic weight, protection signal, and queue depth ratio; determine the arbitration symbol bit based on the dynamic weight, the protection signal, and the queue depth ratio; and determine the service queue based on the arbitration symbol bit.

2. The memory prefetching coordination method according to claim 1, characterized in that, The process involves segmenting the metadata data packet into metadata slices, replacing the ECC checksum bits of the standard PCIe data packet with the metadata slices, regenerating the complete PCIe data packet, and transmitting it to the CPU, including: The metadata data packet is received, and the 64-bit data is evenly divided into 8 metadata slices, with each metadata slice starting from slice 0 and ending at slice 7; each metadata slice is an 8-bit binary bit. The standard PCIe data packet, including the payload and ECC check bit, is obtained. Based on the ECC check bit and the metadata slice, the highest bit of the ECC check bit is modified to an identifier bit to obtain the highest bit of the new ECC field. The ECC check bit is an 8-bit binary bit. Store the data of the metadata slice into the lower 7 bits of the new ECC field; Each group consists of eight consecutive standard PCIe packets, which are then embedded into slices 0 to 7 of the metadata packet. For the modified standard PCIe data packet, a type identifier is inserted into the packet header, a CRC checksum is calculated based on the payload and the new ECC field, and the type identifier, the payload, the new ECC field and the CRC checksum are assembled into the complete PCIe data packet.

3. The memory prefetching coordination method according to claim 2, characterized in that, For the modified standard PCIe data packet, a type identifier is inserted into the packet header. A CRC checksum is calculated based on the payload and the new ECC field. The type identifier, payload, new ECC field, and CRC checksum are then assembled into the complete PCIe data packet. The method further includes: For the complete PCIe data packet, the direction flag, the step size level, the confidence level, and the address range field are parsed and obtained; the address range field is a 58-bit binary field. Calculate the step size index based on the step size level, where the step size index is the binary number of the step size level minus 1. The thermal increment is calculated based on the confidence level and the step size exponent, and the thermal increment is determined by the product of the confidence level and 2 raised to the power of the step size exponent. Take the high 52 bits of the address range field, extract the base address, use hash mapping, and calculate the block index. The block index is determined by the remainder of the low 6 bits of the address range field and the upper limit of the index entries of the heatmap; the block index is the index of the heatmap. Output the block index to the heatmap, read the current heat value based on the block index, calculate the new heat value based on the block index, the heat increment, and the current heat value, and write the new heat value back to the position specified by the block index in the heatmap; the new heat value is determined by the sum of the current heat value and the heat increment, with an upper limit of 255; After a preset period, all the block indices are traversed, the current heat value at the specified position of the block index is read, the attenuated heat value is calculated, and the attenuated heat value is written back to the specified position of the block index in the heat map; the attenuated heat value is three-quarters of the current heat value.

4. The memory prefetching coordination method according to claim 1, characterized in that, Based on the queue depths of the GPU request queue and the CPU request queue, the confidence level of the memory request packet, and the bandwidth utilization, calculate the confidence gain coefficient, dynamic weight, protection signal, and queue depth ratio. The arbitration symbol bit is determined based on the dynamic weight, the protection signal, and the queue depth ratio; Determining the service queue based on the arbitration symbol bit includes: Obtain the GPU request queue depth, the CPU request queue depth, the confidence level of the memory request packet, and the bandwidth utilization rate; Calculate the confidence gain coefficient based on the confidence level; The dynamic weight of the GPU request queue accessing memory is calculated by combining the confidence gain coefficient and the base weight; the base weight is a base coefficient set within a preset value range. If the bandwidth utilization rate is greater than the preset bandwidth threshold, set the protection signal to 1; If the bandwidth utilization rate is less than or equal to the preset bandwidth threshold, the protection signal is set to 0; If the protection signal is 1, then select to serve the CPU request queue and set the arbitration symbol bit to 10; If the protection signal is 0, and the ratio of the GPU request queue depth to the total queue depth is less than or equal to the dynamic weight, then the CPU request queue is selected to be served, and the arbitration sign bit is set to 10. If the protection signal is 0, and the ratio of the GPU request queue depth to the total queue depth is greater than the dynamic weight, then the GPU request queue is selected for service, and the arbitration sign bit is set to 01.

5. A memory prefetching coordination method according to claim 1, characterized in that, The GPU obtains the access address and generates a meta-data packet. The meta-data packet includes: a direction flag, a step size level, a confidence level, and an address range field, including: Obtain the current access address and the access address of the previous cycle, and calculate the difference between consecutive addresses; The step size is calculated based on the absolute value of the consecutive address differences, and the step size is quantized into the step size level; The direction flag is determined based on the number of positive and negative differences in the consecutive address differences. Obtain prefetched records from historical prefetched records, calculate the prefetch hit rate, and calculate the confidence level based on the prefetch hit rate; The lower 6 bits of the address are extracted from the current access address as the base address, and the address range field is generated by combining the step size level; the direction flag, the step size level, the confidence level, and the address range field are concatenated to form the metadata packet.

6. A memory prefetching coordination method according to any one of claims 1 or 5, characterized in that, The GPU obtains the access address and generates a meta-data packet, which includes: a direction flag, a step size level, a confidence level, and an address range field; the meta-data packet is a 64-bit binary file, including: The step size is quantized to a preset step size level, and the step size level is stored using 2 bits of binary; If the number of positive difference values ​​is greater than or equal to the number of negative difference values, the direction flag is 0; if the number of positive difference values ​​is less than the number of negative difference values, the direction flag is 1. The confidence level is a linear mapping of the prefetch hit rate to a 0-7 level. The address range field is obtained by left-shifting the base address and the step size level by 58 bits and performing a bitwise logical OR operation.

7. A memory prefetching coordination method according to any one of claims 1 or 3, characterized in that, Based on the physical address obtained from parsing the complete PCIe packet, a heatmap index is queried to obtain a heat value. If the heat value and the confidence level meet preset conditions, a prefetch address and memory request packet are generated and added to the GPU request queue, including: Parse the complete PCIe data packet to obtain the physical address, calculate the block index and map it to a heatmap index, and query to obtain the heatmap value; If the heat value is greater than the preset activation threshold and the confidence level is greater than or equal to 5, then the prefetch address is calculated based on the physical address, the direction flag, and the step size level; a priority flag and a prefetch type identifier are added before the prefetch address to generate the memory request packet; If there are no incomplete requests with the same prefetch address in the GPU request queue, the memory request packet is added to the tail of the GPU request queue; if there are incomplete requests with the same prefetch address, the memory request packet in the GPU request queue is moved forward one line.

8. A memory prefetching coordination system, characterized in that, include: The PHU module is used to obtain access address information, calculate the direction flag, step size level, confidence level and address range field, and generate a 64-bit metadata data packet. The metadata data transmission module is used to divide the 64-bit metadata data packet into metadata slices, embed the ECC check bits of the PCIe data packet, and assemble them into a complete PCIe data packet. The PCIe parsing module is used to parse complete PCIe packets on the CPU side, build heatmaps, and perform heatmap value updates and periodic decay operations. The prefetch address generation module is used to calculate the prefetch address and generate memory request packets based on physical address and heatmap information, and add them to the GPU request queue. The arbitration decision module is used to calculate the arbitration symbol bit based on queue depth, confidence level, and bandwidth utilization to determine the service queue.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a computer program executable by the processor, the processor being able to execute the computer program to implement a memory prefetching coordination method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements a memory prefetching coordination method as described in any one of claims 1 to 7.

Images