Memory access method and related device
By using a prefetch request mechanism between the processor and memory to determine and return matching prefetch data, the problem of low data transmission efficiency in on-chip networks is solved, the routing and cache management of the prefetch network are optimized, and data transmission efficiency is improved.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- HYGON INFORMATION TECH CO LTD
- Filing Date
- 2025-06-13
- Publication Date
- 2026-06-04
Smart Images
Figure CN2025100799_04062026_PF_FP_ABST
Abstract
Description
A memory access method and related devices
[0001] This application claims priority to Chinese Patent No. 202411736241.2, filed on November 28, 2024, entitled “A Memory Access Method and Related Device”, all contents of which are incorporated herein by reference. Technical Field
[0002] This application relates to the field of computer technology, specifically to a memory access method and related devices. Background Technology
[0003] In Network-on-Chip (NOC) design, the efficiency of data transfer between the processor (e.g., CPU) and memory, particularly bandwidth and latency, has always been a core challenge for designers. In recent years, as the pace of advancement in semiconductor manufacturing processes has slowed, chip sizes have continued to increase with the maturation of these processes, pushing many chip designs close to the physical limits of lithography technology. This large-scale chip design not only increases design complexity but also makes data latency issues increasingly significant, becoming a major bottleneck for processor access to memory.
[0004] Against this backdrop, how to reduce latency and improve data transmission efficiency has become a technical problem that urgently needs to be solved by those skilled in the art.
[0005] This application content
[0006] In view of this, embodiments of this application provide a memory access method and related devices to improve data transmission efficiency.
[0007] To achieve the above objectives, the embodiments of this application provide the following technical solutions.
[0008] In a first aspect, embodiments of this application provide a memory access method, including:
[0009] Obtain a prefetch request, wherein the prefetch request is a prediction request generated by the processor, and / or an extraction request obtained by extracting the access request sent to memory by the processor; wherein the prefetch request includes at least a memory address and control information;
[0010] Based on the prefetch request, read the prefetch data corresponding to the prefetch request from the memory;
[0011] Determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request;
[0012] If the memory address of the access request sent by the processor to memory matches the memory address of the prefetch request, the prefetched data is returned to the processor as feedback data of the access request.
[0013] Optionally, if the memory address of the access request sent by the processor to memory does not match the memory address of the prefetch request, the prefetched data is discarded.
[0014] Optionally, when the prefetch request is a prediction request generated by the processor and an extraction request obtained by extracting the access request sent by the processor to memory, after the step of obtaining the prefetch request and before the step of reading the prefetch data corresponding to the prefetch request from the memory based on the prefetch request, the method further includes:
[0015] Deduplicate prefetch requests with the same memory address;
[0016] Based on the prefetch request after deduplication of the prefetch network transmission.
[0017] Optionally, after the step of deduplicating prefetch requests with the same memory address and before the step of transmitting the deduplicated prefetch requests based on the prefetch network, the method further includes:
[0018] Based on preset rules, preset tags are configured for prefetch requests entering the prefetch network.
[0019] Optionally, the memory access method is adapted to multi-chip devices, and the prefetch request is transmitted across chip lines based on the prefetch network; the step of transmitting the deduplicated prefetch request based on the prefetch network includes:
[0020] The prefetch request is sent from the transmitter of the prefetch network across the chip line in one chip to the receiver of the prefetch network across the chip line in another chip.
[0021] The transmitting end of the prefetch network cross-chip line is located at the position closest to all processors in the chip where the transmitting end is located, and the receiving end of the prefetch network cross-chip line is located near the memory in the chip where the receiving end is located.
[0022] Optionally, the step of transmitting the deduplicated prefetch request based on the prefetch network transmission further includes:
[0023] The prefetch request is cached.
[0024] Optionally, the step of caching the prefetch request includes:
[0025] When the cached prefetch requests overflow, new prefetch requests are discarded or replaced.
[0026] If the prefetch request is not sent to memory within a preset time, the prefetch request is discarded.
[0027] Optionally, the prefetch request has a lower priority than the access request sent to memory by the processor.
[0028] Optionally, the prefetch network includes a weight selector for assigning different weight values to different links or nodes in the prefetch network.
[0029] Secondly, embodiments of this application provide a memory access device, including:
[0030] An acquisition module is used to acquire a prefetch request, wherein the prefetch request is a prediction request generated by the processor, and / or an extraction request obtained by extracting the access request sent to memory by the processor; wherein the prefetch request includes at least a memory address and control information;
[0031] A prefetch request control module is configured to read prefetch data corresponding to the prefetch request from the memory based on the prefetch request; determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request; wherein, if the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, the prefetch data is returned to the processor as feedback data of the access request.
[0032] Optionally, the prefetch request control module includes: a reading module and a judgment module;
[0033] The reading module is used to read prefetch data corresponding to the prefetch request from the memory based on the prefetch request;
[0034] The judgment module is used to determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request; wherein, if the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, the prefetch data is returned to the processor as feedback data of the access request.
[0035] Optionally, if the memory address of the access request sent by the processor to memory does not match the memory address of the prefetch request, the prefetched data is discarded.
[0036] Optional, also includes:
[0037] A deduplication module is used to deduplicat the prefetch requests;
[0038] The transmission module is used to transmit deduplicated prefetch requests based on the prefetch network.
[0039] Optional, also includes:
[0040] The tagging module is used to configure preset tags for prefetch requests entering the prefetching network based on preset rules.
[0041] Thirdly, embodiments of this application provide an electronic device including at least one memory and at least one processor, wherein the memory stores one or more computer-executable instructions, and the processor invokes the one or more computer-executable instructions to execute the memory access method as described in the first aspect above.
[0042] Fourthly, embodiments of this application provide a storage medium that stores one or more computer-executable instructions, which, when executed, implement the memory access method described in the first aspect above.
[0043] Fifthly, embodiments of this application provide a computer program product including one or more computer-executable instructions, which, when executed, implement the memory access method as described in the first aspect above.
[0044] This application provides a memory access method and related apparatus. The memory access method includes: acquiring a prefetch request, wherein the prefetch request is a prediction request generated by a processor, and / or an extraction request obtained by extracting an access request sent to memory by the processor; wherein the prefetch request includes at least a memory address and control information; reading prefetch data corresponding to the prefetch request from the memory based on the prefetch request; determining whether the memory address of the access request sent to memory by the processor matches the memory address of the prefetch request; if the memory address of the access request sent to memory by the processor matches the memory address of the prefetch request, then returning the prefetch data as feedback data of the access request to the processor.
[0045] As can be seen, the memory access method provided in this application reads prefetched data corresponding to the prefetch request from the memory based on the prefetch request. Then, when the processor issues an access request, it determines whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request. If the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, it means that the processor is requesting the prefetched data that has already been read from the memory. At this time, the prefetched data can be directly returned to the processor as feedback data of the access request without accessing the slower memory, thereby reducing latency and improving data transmission efficiency. Attached Figure Description
[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0047] Figure 1 is a schematic diagram of an optional process of the memory access method provided in an embodiment of this application;
[0048] Figure 2 is a schematic diagram of an optional CPU memory access structure provided in an embodiment of this application;
[0049] Figure 3 is a schematic diagram of an optional structure for cross-chip transmission in an NOC network provided in an embodiment of this application;
[0050] Figure 4 is a schematic diagram of another optional process of the memory access method provided in the embodiment of this application;
[0051] Figure 5 is a schematic diagram of another optional process of the memory access method provided in the embodiments of this application;
[0052] Figure 6 is a schematic diagram of an optional structure for cross-chip transmission of the prefetch network provided in an embodiment of this application;
[0053] Figure 7 is a schematic diagram of an optional structure of the memory access device provided in an embodiment of this application;
[0054] Figure 8 is a schematic diagram of another optional structure for CPU accessing memory provided in an embodiment of this application;
[0055] Figure 9 is an optional block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation
[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0057] As described in the background section, in the design of Network-on-Chip (NOC), the efficiency of data transfer between the processor (e.g., CPU) and memory, particularly bandwidth and latency, has always been a core challenge for designers. In recent years, with the slowdown in the pace of advancement in semiconductor manufacturing processes, chip sizes have continued to increase as processes mature, and many chip designs are approaching the physical limits of lithography technology. This large-scale chip design not only increases design complexity but also makes data latency issues increasingly significant, becoming a major bottleneck for processor access to memory. Against this backdrop, how to reduce latency to improve data transfer efficiency has become a technical problem that urgently needs to be solved by those skilled in the art.
[0058] Based on this, embodiments of this application provide a memory access method and related apparatus. The memory access method includes: acquiring a prefetch request, wherein the prefetch request is a prediction request generated by a processor, and / or an extraction request obtained by extracting an access request sent to memory by the processor; wherein the prefetch request includes at least a memory address and control information; reading prefetch data corresponding to the prefetch request from the memory based on the prefetch request; determining whether the memory address of the access request sent to memory by the processor matches the memory address of the prefetch request; if the memory address of the access request sent to memory by the processor matches the memory address of the prefetch request, then returning the prefetch data as feedback data of the access request to the processor.
[0059] As can be seen, the memory access method provided in this application reads prefetched data corresponding to the prefetch request from the memory based on the prefetch request. Then, when the processor issues an access request, it determines whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request. If the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, it means that the processor is requesting the prefetched data that has already been read from the memory. At this time, the prefetched data can be directly returned to the processor as feedback data of the access request without accessing the slower memory, thereby reducing latency and improving data transmission efficiency.
[0060] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.
[0061] Referring to Figure 1, Figure 1 is a schematic flowchart of an optional memory access method provided in an embodiment of this application. As shown in Figure 1, the memory access method may include the following steps:
[0062] Step S100: Obtain the prefetch request.
[0063] The prefetch request includes at least a memory address and control information. The memory address refers to the address of the memory that the prefetch request needs to access, specifying the exact location from which data needs to be read or written. The memory controller can perform data access operations on the memory region accessed by the prefetch request based on the memory address. The control information instructs the memory controller how to perform the data access operation. This information may include, but is not limited to, operation type, data size, priority, and control flags. The operation type indicates whether the request is a read or write operation. For the prefetch request in this application, the operation type is a read operation, because the purpose of prefetching is to obtain data in advance for later use. The data size specifies the amount of data to be read or written. For the prefetch request in this application, the data size is the amount of data to be read.
[0064] The prefetch request is a prediction request generated by the processor, and / or an extraction request obtained by extracting access requests sent to memory by the processor. Taking the processor as a CPU (Central Processing Unit) as an example, the prefetch request can originate from predictions within the CPU, or from normal access requests sent to memory by the CPU. These two sources are explained below:
[0065] CPU prediction refers to the process of predicting the next instruction while the CPU is executing the current instruction, using predictive techniques. This process can be performed by the CPU's internal predictor, which uses algorithms and historical data to predict the next instruction.
[0066] Referring to Figure 2, which is a schematic diagram of an optional CPU memory access structure provided in an embodiment of this application, the CPU memory access process can be as follows: The CPU initiates an access request, which first passes through multiple levels of cache to minimize the number of direct memory accesses, thereby improving access efficiency. Specifically, when the CPU needs to access data, it first searches the highest-level cache. If the data is not found (i.e., the required data is not in that level of cache), it continues to search the next level of cache, and so on, until the required data is found or the lowest-level memory (such as main memory) is reached. The relatively small storage units of the upper-level cache can map to the relatively large storage units of the lower-level cache, and the relatively small storage units of the lower-level cache can map to the relatively large storage units of main memory; furthermore, the upper-level cache has a faster data access speed than the lower-level cache, but its capacity is smaller.
[0067] In the CPU access process, if multiple cache levels fail to find the cache, the access request enters the NOC network (Network on Chip). This request is then routed to the memory access proxy module. The memory access proxy module is responsible for converting the access request into a format suitable for memory access and sending it to the memory controller. The memory controller then forwards the access request to memory for further processing. Access requests that undergo the above process can be referred to as normal access requests sent by the CPU to memory.
[0068] In some cases, if the required data resides on different chips, the NOC network also needs to handle cross-chip transmission. Specifically, referring to Figure 3, which is a schematic diagram of an optional structure for cross-chip transmission of the NOC network provided in an embodiment of this application, as shown in Figure 3, the NOC network in DIE 0 is connected to the NOC network in DIE 1 through a PHY (Physical Layer) port to enable communication between the NOC networks in DIE 0 and DIE 1.
[0069] In an optional implementation, the memory address of the prefetch request has a smaller bit width than the memory address of the access request transmitted over the NOC network; for example, the bit width of the memory address can range from 32 to 64 bits. A smaller bit width allows the prefetch request to occupy less bandwidth during transmission, thereby reducing the likelihood of network congestion and improving data transmission efficiency.
[0070] In this context, the prefetch request has a lower priority than the access request sent to memory by the processor. That is, when access requests A and B sent to memory by the processor, along with prefetch request B, both require memory access, the access request A sent to memory by the processor has a higher priority than the prefetch request B. Furthermore, the memory addresses of access requests A and B sent to memory by the processor are different from those of prefetch request B.
[0071] Compared to prefetch request B, access request A sent by the processor to memory is issued based on the exact current demand. Therefore, when the two conflict, to ensure correct program execution and performance, access request A sent by the processor to memory should be processed first.
[0072] Referring again to Figure 1, step S200 is executed, whereby prefetch data corresponding to the prefetch request is read from the memory based on the prefetch request.
[0073] By reading prefetched data corresponding to the prefetch request from the memory based on the prefetch request, and then returning the prefetched data as feedback data of the access request to the processor when the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, the latency is reduced and the data transmission efficiency is improved without accessing the slower memory.
[0074] Referring again to Figure 1, step S300 is executed to determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request.
[0075] If the memory address of the access request sent by the processor to memory matches the memory address of the prefetch request, then step S400 is executed, and the prefetched data is returned to the processor as feedback data of the access request. If the memory address of the access request sent by the processor to memory matches the memory address of the prefetch request, it means that the processor is requesting the prefetched data that has already been read from the memory. In this case, the prefetched data can be directly returned to the processor as feedback data of the access request without accessing the slower memory, thereby reducing latency and improving data transfer efficiency.
[0076] Furthermore, referring to Figure 4, which is another optional flowchart of the memory access method provided in this application embodiment, if the memory address of the access request sent by the processor to the memory does not match the memory address of the prefetch request, then step S500 is executed to discard the prefetched data. Specifically, when a new prefetch request arrives, and the cache space is insufficient to accommodate all the data that needs to be prefetched, some data that has already been prefetched but has not yet been used by the CPU can be discarded to free up space for the new prefetch request.
[0077] As can be seen, the memory access method provided in this application reads prefetched data corresponding to the prefetch request from the memory based on the prefetch request. Then, when the processor issues an access request, it determines whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request. If the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, it means that the processor is requesting the prefetched data that has already been read from the memory. At this time, the prefetched data can be directly returned to the processor as feedback data of the access request without accessing the slower memory, thereby reducing latency and improving data transmission efficiency.
[0078] In an optional example, refer to FIG5, which shows another optional flowchart of the memory access method provided by an embodiment of this application. As shown in FIG5, after the step of obtaining the prefetch request and before the step of reading the prefetch data corresponding to the prefetch request from the memory based on the prefetch request, the method may further include:
[0079] Step S101: Deduplicate prefetch requests with the same memory address.
[0080] Since the prefetch request is a prediction request generated by the processor, and / or an extraction request obtained by extracting the access request sent to memory by the processor, when the prefetch request is a prediction request generated by the processor and an extraction request obtained by extracting the access request sent to memory by the processor, prefetch requests with the same memory address may arrive one after another in a short period of time. Therefore, it is necessary to deduplicate prefetch requests with the same memory address. For example, the prefetch request that arrives first can be retained, and the prefetch request that arrives later can be discarded.
[0081] Step S102: Transmit the deduplicated prefetch request based on the prefetch network.
[0082] Since prefetch requests entering the prefetch network are for prefetching purposes, they aim to obtain data that may be needed in the future to reduce latency during actual access. However, due to the inherent uncertainties in the prefetching process (such as whether the data will actually be used, network conditions, etc.), a certain degree of packet loss is permissible during prefetching.
[0083] Furthermore, to optimize the latency of the prefetch network, the routing of the prefetch network can be simplified. By reducing the complexity and hierarchy of routing, the transmission latency of data packets in the network can be reduced. For example, a more direct routing path can be used, or the number of routing nodes can be reduced.
[0084] Furthermore, selectors can be used, such as "one-to-many" and "many-to-one" selectors, to optimize data transmission, reduce data branching and merging during transmission, and thus lower latency. A "one-to-many" selector means that one data source (or sender, source node) can simultaneously transmit data to multiple data receivers (or target nodes, terminals). A "many-to-one" selector means that multiple data sources can simultaneously transmit data to one data receiver.
[0085] In an optional implementation, the step of transmitting the deduplicated prefetch request based on the prefetch network may further include: caching the prefetch request. By caching the prefetch request, when the memory address of an access request sent by the processor to memory matches the memory address of the prefetch request, the prefetched data can be directly returned to the processor as feedback data of the access request, without needing to access slower memory, thereby reducing latency and improving data transmission efficiency.
[0086] In a specific implementation, the step of caching the prefetch request may include: discarding or replacing the new prefetch request when the cached prefetch request overflows; and discarding the prefetch request if it is not sent to memory within a preset time. That is, when a new prefetch request arrives, if the cache is full, these additional prefetch requests can be handled in two ways: One is to discard new prefetch requests exceeding the cache capacity. The other is to use a replacement algorithm (such as LRU, LFU, etc.) to replace existing prefetch requests in the cache. In some cases, prefetch requests may not be sent to memory within the expected time, causing them to become outdated or invalid. To avoid system resources being occupied by invalid requests, if a prefetch request fails to be successfully sent to memory within a preset time, it can be discarded.
[0087] The memory access method is adapted to multi-chip devices. In an optional implementation, the prefetch request is transmitted across chips based on a prefetch network cross-chip line. The step of transmitting the deduplicated prefetch request based on the prefetch network may include: the prefetch request is sent from the transmitting end of the prefetch network cross-chip line in one chip to the receiving end of the prefetch network cross-chip line in another chip. Specifically, referring to Figure 6, Figure 6 is a schematic diagram of an optional structure for cross-chip transmission of the prefetch network provided in an embodiment of this application. As shown in Figure 6, the prefetch request is sent from the transmitting end of the prefetch network cross-chip line in DIE 0 to the receiving end of the prefetch network cross-chip line in DIE 1, and from the transmitting end of the prefetch network cross-chip line in DIE 1 to the receiving end of the prefetch network cross-chip line in DIE 0. The transmitting and receiving ends in DIE 0 and DIE 1 can both be PHY (Physical Layer) ports.
[0088] In its implementation, the transmitting end of the prefetch network across chip lines is positioned closest to all processors within the chip where the transmitting end resides, while the receiving end of the prefetch network across chip lines is positioned close to the memory within the chip where the receiving end resides. Positioning the transmitting end closest to the processors reduces the physical distance between the processors and the prefetch network, thereby reducing data transmission latency. Positioning the receiving end close to the memory reduces memory access time, as data can be transferred from the prefetch network to memory more quickly.
[0089] Furthermore, to ensure data transmission efficiency and resource utilization, the number of cross-slice lines in the prefetch network can be set to match the size of the prefetch network. Too many cross-slice lines may lead to resource waste and increased complexity, while too few cross-slice lines may limit data transmission efficiency and speed.
[0090] To optimize the balance of the prefetch network, in one optional implementation, after the step of deduplicating prefetch requests with the same memory address and before the step of transmitting the deduplicated prefetch requests based on the prefetch network, the following may be included: configuring a preset tag for the prefetch requests entering the prefetch network based on a preset rule. In a specific implementation, a tag 0 or a tag 1 can be configured for the prefetch requests, wherein the allocation of the tag can be random or in a uniform distribution manner (such as alternating between 0 and 1). When a node in the prefetch network receives multiple prefetch requests from different sources that need to be sent to the same exit (i.e., when using a "many-to-one" selector), these tagged requests are first buffered in the input buffer of the "many-to-one" selector. At the exit, the prefetch requests are sent in sequence. For example, a prefetch request marked with 0 can be sent first, followed by a prefetch request marked with 1, and then a prefetch request marked with 0 can be sent again. This process is repeated. By cyclically selecting prefetch requests with different tags for sending, bandwidth can be distributed evenly in a probabilistic manner. This effectively avoids the problem of one or more sources occupying a large amount of bandwidth for a long time, causing other sources to not be able to obtain sufficient bandwidth.
[0091] In an alternative implementation, a weight selector can be configured for the prefetch network. This weight selector assigns different weight values to different links or nodes within the prefetch network. For example, each node in the prefetch network can be assigned a specific weight value. This weight value can reflect the node's processing power, storage resources, or other relevant performance metrics. When a prefetch request enters the network, the weight selector determines which node the request should be routed to based on these weight values, maximizing network performance and efficiency. In a concrete implementation, if a node experiences a performance bottleneck or overload, its weight value can be dynamically adjusted to reduce the number of prefetch requests sent to that node, thereby alleviating pressure on the prefetch network.
[0092] As can be seen, the memory access method provided in this application reads prefetched data corresponding to the prefetch request from the memory based on the prefetch request. Then, when the processor issues an access request, it determines whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request. If the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, it means that the processor is requesting the prefetched data that has already been read from the memory. At this time, the prefetched data can be directly returned to the processor as feedback data of the access request without accessing the slower memory, thereby reducing latency and improving data transmission efficiency.
[0093] The memory access apparatus provided in the embodiments of this application will be described below. The memory access apparatus described below can be considered as a software or hardware functional module required to implement the memory access method provided in the embodiments of this application. The content of the memory access apparatus described below can be referred to in correspondence with the content of the method described above.
[0094] In an optional implementation, FIG7 exemplarily illustrates an optional structural schematic diagram of the memory access device provided in the embodiments of this application. The memory access device is used to implement the memory access method provided in the embodiments of this application. As shown in FIG7, the memory access device may include:
[0095] Acquisition module 1 is used to acquire a prefetch request, wherein the prefetch request is a prediction request generated by the processor, and / or an extraction request obtained by extracting the access request sent to memory by the processor; wherein the prefetch request includes at least a memory address and control information;
[0096] The prefetch request control module 5 is used to read prefetch data corresponding to the prefetch request from the memory based on the prefetch request; determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request; wherein, if the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, the prefetch data is returned to the processor as feedback data of the access request.
[0097] Optionally, the prefetch request control module 5 includes: a reading module 51 and a judging module 52;
[0098] The reading module 51 is used to read prefetch data corresponding to the prefetch request from the memory based on the prefetch request;
[0099] The judgment module 52 is used to determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request; wherein, if the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, the prefetch data is returned to the processor as feedback data of the access request.
[0100] Optionally, if the memory address of the access request sent by the processor to memory does not match the memory address of the prefetch request, the prefetched data is discarded.
[0101] Optional, also includes:
[0102] Deduplication module 2 is used to deduplicatize the prefetch requests;
[0103] Transmission module 4 is used to transmit deduplicated prefetch requests based on the prefetch network.
[0104] Optional, also includes:
[0105] Tagging module 3 is used to configure preset tags for prefetch requests entering the prefetch network based on preset rules.
[0106] Optionally, the prefetch request is transmitted across chip lines based on the prefetch network; the transmission module 4 is used to transmit the deduplicated prefetch request based on the prefetch network, including:
[0107] The prefetch request is sent from the transmitter of the prefetch network across the chip line in one chip to the receiver of the prefetch network across the chip line in another chip.
[0108] The transmitting end of the prefetch network cross-chip line is located at the position closest to all processors in the chip where the transmitting end is located, and the receiving end of the prefetch network cross-chip line is located near the memory in the chip where the receiving end is located.
[0109] Optionally, the deduplication module 2, used for transmitting the deduplicated prefetch request based on the prefetch network, further includes:
[0110] The prefetch request is cached.
[0111] Optionally, caching the prefetch request includes:
[0112] When the cached prefetch requests overflow, new prefetch requests are discarded or replaced.
[0113] If the prefetch request is not sent to memory within a preset time, the prefetch request is discarded.
[0114] Optionally, the prefetch request has a lower priority than the access request sent to memory by the processor.
[0115] Optionally, the prefetch network includes a weight selector for assigning different weight values to different links or nodes in the prefetch network.
[0116] In an optional implementation, referring to Figure 8, which is a schematic diagram of another optional structure for CPU accessing memory provided in an embodiment of this application, as shown in Figure 8, the acquisition module first acquires the prediction request generated by the predictor inside the CPU, and the extraction request obtained by extracting the access request sent by the processor to memory. Then, the deduplication module deduplicates the prefetch requests with the same memory address. Next, the deduplicated prefetch requests are transmitted based on the prefetch network. Finally, the prefetch request control module reads the prefetch data corresponding to the prefetch request from the memory based on the prefetch request. Then, when the access request sent by the processor passes through the NOC network and the memory access proxy module and is sent to the memory controller, it is determined whether the memory address of the access request sent by the processor to memory matches the memory address of the prefetch request. If the memory address of the access request sent by the processor to memory matches the memory address of the prefetch request, it means that the processor is requesting the prefetch data that has been read from the memory before. At this time, the prefetch data can be directly returned to the processor as the feedback data of the access request, without having to access the slower memory, thereby reducing latency and improving data transmission efficiency.
[0117] In an alternative implementation, if the memory address of the access request sent by the processor to memory does not match the memory address of the prefetch request, the prefetched data is discarded.
[0118] In one alternative implementation, the memory address of the access request sent by the processor to memory matches the memory address of the prefetch request. However, if the prefetch data corresponding to the prefetch request has not yet been read, the processor can wait for the prefetch data to be read and then return the prefetch data as feedback data to the processor, without having to access the slower memory.
[0119] In an alternative implementation, prefetched data can be left uncached. If no matching access request is found when the prefetched data corresponding to a prefetch request has been read, the prefetched data can be discarded directly.
[0120] In one optional implementation, before transmitting deduplicated prefetch requests through the prefetch network, a tagging module can configure a preset tag for each prefetch request entering the prefetch network based on preset rules. This preset tag indicates the node information to which the prefetch request should be sent. By indicating the node information to which the prefetch request should be sent, prefetch requests can be managed and scheduled more effectively, ensuring load balancing and efficient operation of the prefetch network.
[0121] In an optional implementation, when the memory access proxy module returns feedback data to the processor, it can determine the memory state's busy level based on the memory's busyness and configure the busy level in the feedback data. This allows the predictor in the CPU and the extraction module (not shown in the figure) that extracts access requests sent to memory by the CPU to limit or stop the generation of prefetch requests based on the memory state's busy level fed back by the memory access proxy module. For example, when the memory state's busy level is greater than or equal to a preset busy level, the generation of prefetch requests is stopped.
[0122] It should be noted that the memory access method provided in this application is particularly applicable to situations where there is an objective delay in processor memory access, where the delay time is typically between 50 nanoseconds and 200 nanoseconds.
[0123] As can be seen, the memory access method provided in this application reads prefetched data corresponding to the prefetch request from the memory based on the prefetch request. Then, when the processor issues an access request, it determines whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request. If the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, it means that the processor is requesting the prefetched data that has already been read from the memory. At this time, the prefetched data can be directly returned to the processor as feedback data of the access request without accessing the slower memory, thereby reducing latency and improving data transmission efficiency.
[0124] This application also provides an electronic device that may include at least one memory and at least one processor. The memory stores one or more computer-executable instructions, and the processor invokes the one or more computer-executable instructions to execute the memory access method as described above.
[0125] As an optional implementation, referring to FIG9, FIG9 is an optional block diagram of an electronic device provided in an embodiment of the present application. As shown in FIG9, the electronic device may include: at least one processor 10, at least one communication interface 20, at least one memory 30, and at least one communication bus 40.
[0126] In this embodiment, the number of processor 10, communication interface 20, memory 30 and communication bus 40 is at least one, and processor 10, communication interface 20 and memory 30 communicate with each other through communication bus 40.
[0127] Optionally, the processor 10 may be a CPU (Central Processing Unit), GPU (Graphics Processing Unit), NPU (Neural-network Processing Unit), FPGA (Field Programmable Gate Array), TPU (Tensor Processing Unit), AI chip, ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.
[0128] Optionally, the communication interface 20 can be an interface for a communication module used for network communication.
[0129] The memory 30 may include high-speed RAM and may also include non-volatile memory, such as at least one disk storage device. The memory 30 stores one or more computer-executable instructions, which the processor 10 invokes to perform the memory access method described above.
[0130] This application also provides a storage medium that stores one or more computer-executable instructions, which, when executed, implement the memory access method described above.
[0131] This application also provides a computer program product that may include one or more computer-executable instructions, which, when executed, implement the memory access method described above.
[0132] The foregoing describes multiple embodiment schemes provided by the embodiments of this application. The optional methods described in each embodiment scheme can be combined and cross-referenced with each other without conflict, thereby extending to a variety of possible embodiment schemes. These can all be considered as the embodiment schemes disclosed and published by the embodiments of this application.
[0133] While the embodiments disclosed above are described in this application, this application is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.
Claims
1. A memory access method, wherein, include: Obtain a prefetch request, wherein the prefetch request is a prediction request generated by the processor, and / or an extraction request obtained by extracting the access request sent to memory by the processor; wherein the prefetch request includes at least a memory address and control information; Based on the prefetch request, read the prefetch data corresponding to the prefetch request from the memory; Determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request; If the memory address of the access request sent by the processor to memory matches the memory address of the prefetch request, the prefetched data is returned to the processor as feedback data of the access request.
2. The memory access method according to claim 1, wherein, If the memory address of the access request sent by the processor to memory does not match the memory address of the prefetch request, the prefetched data is discarded.
3. The memory access method according to claim 1, wherein, When the prefetch request is a prediction request generated by the processor, and an extraction request is obtained by extracting access requests sent to memory by the processor, after the step of obtaining the prefetch request and before the step of reading prefetch data corresponding to the prefetch request from the memory based on the prefetch request, the method further includes: Deduplicate prefetch requests with the same memory address; Based on the prefetch request after deduplication of the prefetch network transmission.
4. The memory access method according to claim 3, wherein, After the step of deduplicating prefetch requests with the same memory address, and before the step of transmitting the deduplicated prefetch requests based on the prefetch network, the method further includes: Based on preset rules, preset tags are configured for prefetch requests entering the prefetch network.
5. The memory access method according to claim 3, wherein, The memory access method is adapted to multi-chip devices, and the prefetch request is transmitted across chips based on the prefetch network across chip lines. The steps of transmitting deduplicated prefetch requests based on the prefetch network transmission include: The prefetch request is sent from the transmitter of the prefetch network across the chip line in one chip to the receiver of the prefetch network across the chip line in another chip. The transmitting end of the prefetch network cross-chip line is located at the position closest to all processors in the chip where the transmitting end is located, and the receiving end of the prefetch network cross-chip line is located near the memory in the chip where the receiving end is located.
6. The memory access method according to claim 3, wherein, The step of transmitting a deduplicated prefetch request based on the prefetch network transmission further includes: The prefetch request is cached.
7. The memory access method according to claim 6, wherein, The step of caching the prefetch request includes: When the cached prefetch requests overflow, new prefetch requests are discarded or replaced. If the prefetch request is not sent to memory within a preset time, the prefetch request is discarded.
8. The memory access method according to claim 1, wherein, The prefetch request has a lower priority than the access request sent to memory by the processor.
9. The memory access method according to claim 3, wherein, The prefetch network includes a weight selector for assigning different weight values to different links or nodes in the prefetch network.
10. A memory access device, wherein, include: An acquisition module is used to acquire a prefetch request, wherein the prefetch request is a prediction request generated by the processor, and / or an extraction request obtained by extracting the access request sent to memory by the processor; wherein the prefetch request includes at least a memory address and control information; A prefetch request control module is configured to read prefetch data corresponding to the prefetch request from the memory based on the prefetch request; determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request; wherein, if the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, the prefetch data is returned to the processor as feedback data of the access request.
11. The memory access device according to claim 10, wherein, The prefetch request control module includes: a reading module and a judgment module; The reading module is used to read prefetch data corresponding to the prefetch request from the memory based on the prefetch request; The judgment module is used to determine whether the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request; wherein, if the memory address of the access request sent by the processor to the memory matches the memory address of the prefetch request, the prefetch data is returned to the processor as feedback data of the access request.
12. The memory access device according to claim 11, wherein, If the memory address of the access request sent by the processor to memory does not match the memory address of the prefetch request, the prefetched data is discarded.
13. The memory access device according to claim 10, wherein, Also includes: A deduplication module is used to deduplicat the prefetch requests; The transmission module is used to transmit deduplicated prefetch requests based on the prefetch network.
14. The memory access device according to claim 10, wherein, Also includes: The tagging module is used to configure preset tags for prefetch requests entering the prefetching network based on preset rules.
15. An electronic device, wherein, It includes at least one memory and at least one processor, the memory storing one or more computer-executable instructions, and the processor invoking the one or more computer-executable instructions to perform the memory access method as described in any one of claims 1 to 9.
16. A storage medium, wherein, The storage medium stores one or more computer-executable instructions, which, when executed, implement the memory access method as described in any one of claims 1 to 9.
17. A computer program product, wherein, It includes one or more computer-executable instructions, which, when executed, implement the memory access method as described in any one of claims 1 to 9.