Memory access processing apparatus, method, processor, chip, board and electronic device

Abstract

Embodiments of the present disclosure provide a memory access processing apparatus, method, processor, chip, board card and electronic device. The memory access processing apparatus is used for processing a plurality of parallel memory access requests. The memory includes a plurality of memory regions, and each memory region includes a plurality of memory cells. The memory access processing apparatus includes: an arbitration unit configured to determine a plurality of first memory access requests accessing a same target memory region, and different first memory access requests are used to access different target memory cells in the target memory region; a control unit configured to request data of each memory cell in the target memory region from the memory; and a data return unit configured to return data in a corresponding target memory cell for each first memory access request after the memory returns data of each memory cell in the target memory region. Embodiments of the present disclosure improve the efficiency of a plurality of memory access requests accessing the memory.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the chip technical field, and particularly relates to a memory access processing apparatus and method, a processor, a chip, a board card and an electronic device. BACKGROUND

[0002] With the rapid development of artificial intelligence and other technologies, the data processing tasks required to be undertaken by a computing system are increasingly heavy, and higher requirements are put forward for high-performance computing. In order to improve processing efficiency, many processors will introduce hardware multithreading technology. For example, a graphics processing unit (GPU) can schedule multiple threads to form a thread group, and multiple threads under a thread group complete an overall computing task in parallel. Many computing tasks need to access the memory to read and write data, that is, the memory access request of the thread is needed to complete, and therefore the design of the memory access mechanism is particularly important. For example, for a large-capacity low-bandwidth external memory, when multiple parallel threads access the memory, if the addresses accessed by multiple memory access requests span a large range, the memory needs to span a long address to respond to the memory access requests of each thread, the delay of data transmission will greatly increase, and the response efficiency is low. SUMMARY

[0003] In a first aspect, an embodiment of the present disclosure provides a memory access processing apparatus for processing multiple parallel memory access requests, the memory including multiple memory regions, each memory region including multiple memory cells; the memory access processing apparatus comprising:

[0004] an arbitration unit configured to determine multiple first memory access requests accessing a same target memory region, different first memory access requests being configured to access different target memory cells in the target memory region;

[0005] a control unit configured to request data of each memory cell in the target memory region from the memory;

[0006] a data returning unit configured to return data in a corresponding target memory cell for each first memory access request after the memory returns data of each memory cell in the target memory region.

[0007] In an embodiment of the present disclosure, for multiple parallel memory access requests, the arbitration unit acquires multiple first memory access requests accessing a same target memory region, and requests data from the memory in the granularity of the memory region; for the memory, only the address-continuous memory region needs to be operated, and all data of each memory cell in the memory region is returned; subsequently, the data returning unit performs data distribution, thereby improving the execution efficiency of the multiple first memory access requests.

[0008] Optionally, each memory access request carries first identification information of a memory region accessed and second identification information of a memory cell accessed in the corresponding memory region.

[0009] The arbitration unit is configured to determine the plurality of first memory access requests based on the first identification information.

[0010] The data returning unit is configured to determine a target memory cell accessed by each first memory access request based on the second identification information.

[0011] Optionally, at least one of the first memory access requests is obtained by merging each second memory access request accessing a same memory cell in a same memory region.

[0012] The arbitration unit is configured to send the first memory access requests to the control unit through a plurality of instruction channels, wherein each instruction channel corresponds to a memory cell in a memory region and is configured to send a first memory access request accessing the memory cell corresponding to the instruction channel.

[0013] Optionally, at least one of the first memory access requests and / or at least one of the second memory access requests is obtained by splitting a memory access request accessing a plurality of continuous memory cells.

[0014] Optionally, the apparatus further comprises an arrangement unit configured to:

[0015] In a case where the plurality of first memory access requests satisfy a preset condition, add indication information in each first memory access request, and send each first memory access request carrying the indication information to the control unit; the indication information is configured to indicate that the first memory access request satisfies the preset condition, and the preset condition comprises that each first memory access request in the plurality of first memory access requests accesses a memory cell, and a thread sending the first memory access request and a memory cell accessed by the first memory access request satisfy a preset correspondence.

[0016] The data returning unit is configured to, after the memory returns data of each memory cell in the target memory region, return data in a corresponding target memory cell to each first memory access request based on the preset correspondence.

[0017] Optionally, each memory access request comprises bypass information, the bypass information comprises a first correspondence between a thread sending the memory access request and a memory cell accessed by the memory access request; and the memory access processing apparatus further comprises an arrangement unit and a bypass storage unit.

[0018] The arrangement unit is configured to extract the bypass information from each first memory access request and send the bypass information to the bypass storage unit for storage.

[0019] The data returning unit is configured to, after the memory returns the data in each storage unit in the target storage area, take out the bypass information from the bypass storage unit, and return the data in the corresponding target storage unit to each first memory access request based on the taken-out bypass information.

[0020] Optionally, in a case where the first memory access request is obtained by merging a plurality of second memory access requests accessing a same storage unit in a same storage area, the first correspondence relationship of the first memory access request comprises a correspondence relationship between each thread corresponding to each second memory access request and a target storage unit accessed by the first memory access request.

[0021] In a case where the first memory access request is obtained by splitting a third memory access request accessing a plurality of continuous storage units, the first correspondence relationship of the first memory access request comprises a correspondence relationship between a thread sending the third memory access request and a storage unit accessed by the first memory access request.

[0022] Optionally, the data returning unit is further configured to:

[0023] obtain an encoding corresponding to each first memory access request, the encoding corresponding to a first memory access request being used to determine a target storage unit in a target storage area accessed by the first memory access request, and a bit number of the encoding being less than a total number of storage units in the target storage area.

[0024] return data in the target storage unit accessed by the first memory access request to the first memory access request based on the encoding corresponding to the first memory access request.

[0025] Optionally, a length of the encoding is determined based on a logarithm of the total number of storage units in the target storage area.

[0026] Optionally, the dispatching unit is further configured to:

[0027] send the encoding corresponding to each first memory access request to the bypass storage unit for storage.

[0028] Optionally, the control unit is configured to:

[0029] obtain a plurality of fourth memory access requests, each fourth memory access request in the plurality of fourth memory access requests being a write request and carrying first data to be written into a target storage unit.

[0030] request the memory to access the target storage area, so as to write the first data carried by the plurality of fourth memory access requests into storage units in the target storage area.

[0031] Optionally, the apparatus further comprises a scheduling unit and a bypass information storage unit;

[0032] The scheduling unit is configured to extract the first data from each fourth memory access request and send the first data to the bypass storage unit for storage.

[0033] The control unit is configured to write the first data carried by the plurality of fourth memory access requests into the storage units in the target storage area after obtaining the first data from the bypass storage unit.

[0034] Optionally, the arbitration unit is configured to:

[0035] obtain a plurality of memory access request groups, each memory access request group comprising the plurality of first memory access requests, and the first memory access requests in different memory access request groups being configured to access different target storage areas;

[0036] arbitrate the priority of each memory access request group, so that the control unit requests the memory to access the target storage area accessed by the first memory access request in the memory access request group with the highest priority.

[0037] In a second aspect, the embodiments of the present disclosure provide a memory access processing apparatus for processing a plurality of parallel memory access requests, the memory comprising a plurality of storage areas, each storage area comprising a plurality of storage units; the memory access processing apparatus comprising:

[0038] an arbitration unit configured to determine a plurality of first memory access requests accessing a same target storage area, different first memory access requests being configured to access different target storage units in the target storage area;

[0039] a control unit configured to request the memory to access the target storage area, so as to write first data carried by the respective first memory access requests into the target storage units in the target storage area.

[0040] In the embodiments of the present disclosure, the arbitration unit, for the plurality of parallel memory access requests, obtains a plurality of first memory access requests accessing a same target storage area, so that the control unit requests the memory to access the target storage area, and the first data carried by the plurality of first memory access requests can be written into the storage units in the target storage area. Since the plurality of first memory access requests all access the target storage area, the memory does not need to write data across a long address, thereby improving the access efficiency of the memory.

[0041] Optionally, each memory access request carries first identification information of the accessed storage area and second identification information of the accessed storage unit in the corresponding storage area.

[0042] The arbitration unit is configured to determine the plurality of first memory access requests based on the first identification information.

[0043] The data returning unit is configured to determine a target storage unit accessed by each first memory access request based on the second identification information.

[0044] Optionally, the at least one first memory access request is obtained by splitting a memory access request that accesses a plurality of continuous storage units.

[0045] Optionally, the arbitration unit is configured to:

[0046] obtain a plurality of memory access request groups, each memory access request group including the plurality of first memory access requests, and the first memory access requests in different memory access request groups being configured to access different target storage regions;

[0047] arbitrate priorities of the memory access request groups, so that the control unit requests the memory to access a target storage region accessed by a first memory access request in a memory access request group with the highest priority.

[0048] In a third aspect, an embodiment of the present disclosure provides a processor, which includes the memory access processing apparatus according to any one of the embodiments of the present disclosure.

[0049] In a fourth aspect, an embodiment of the present disclosure provides a chip, which includes the processor according to any one of the embodiments of the present disclosure.

[0050] In a fifth aspect, an embodiment of the present disclosure provides a board card, which includes a packaging structure in which at least one chip according to any one of the embodiments of the present disclosure is packaged.

[0051] In a sixth aspect, an embodiment of the present disclosure provides an electronic device, which includes the chip according to any one of the embodiments of the present disclosure or the board card according to any one of the embodiments of the present disclosure.

[0052] In a seventh aspect, an embodiment of the present disclosure provides a memory access processing method, which is used for processing a plurality of parallel memory access requests to a memory, the memory including a plurality of storage regions, each storage region including a plurality of storage units; the method including:

[0053] determining a plurality of first memory access requests that access a same target storage region, different first memory access requests being configured to access different target storage units in the target storage region;

[0054] requesting data of each storage unit in the target storage region from the memory;

[0055] after the memory returns the data of each storage unit in the target storage region, returning data in a corresponding target storage unit for each first memory access request.

[0056] Optionally, each memory access request carries first identification information of a memory region accessed and second identification information of a memory cell accessed in the corresponding memory region;

[0057] The method further comprises:

[0058] The arbitration unit determines the plurality of first memory access requests based on the first identification information;

[0059] The data returning unit determines a target memory cell accessed by each first memory access request based on the second identification information.

[0060] Optionally, at least one of the first memory access requests is obtained by merging each second memory access request accessing a same memory cell in a same memory region;

[0061] The method further comprises: the arbitration unit sends the first memory access requests to the control unit through a plurality of instruction channels; each instruction channel corresponds to a memory cell in a memory region, and is used to send a first memory access request accessing the memory cell corresponding to the instruction channel.

[0062] Optionally, at least one of the first memory access requests and / or at least one of the second memory access requests is obtained by splitting a memory access request accessing a plurality of continuous memory cells.

[0063] Optionally, the method further comprises:

[0064] The dispatching unit adds indication information in each first memory access request when the plurality of first memory access requests meet a preset condition, and sends each first memory access request carrying the indication information to the control unit; the indication information is used to indicate that the first memory access request meets the preset condition, and the preset condition comprises: each first memory access request in the plurality of first memory access requests accesses a memory cell, and a thread sending the first memory access request meets a preset correspondence relationship with the memory cell accessed by the first memory access request;

[0065] The data returning unit returns data in a corresponding target memory cell to each first memory access request based on the preset correspondence relationship after the memory returns data of each memory cell in the target memory region.

[0066] Optionally, each memory access request comprises bypass information, the bypass information comprises a first correspondence relationship between a thread sending the memory access request and a memory cell accessed by the memory access request; the method further comprises:

[0067] extracting the bypass information from each first memory access request by the dispatching unit and sending the bypass information to the bypass storage unit for storage;

[0068] extracting the bypass information from the bypass storage unit by the data returning unit after the memory returns data of each storage unit in the target storage area, and returning data in the corresponding target storage unit to each first memory access request based on the extracted bypass information.

[0069] Optionally, in a case where the first memory access request is obtained by merging a plurality of second memory access requests accessing a same storage unit in a same storage area, the first correspondence relationship of the first memory access request comprises a correspondence relationship between each second memory access request and a thread corresponding to the target storage unit accessed by the first memory access request.

[0070] In a case where the first memory access request is obtained by splitting a third memory access request accessing a plurality of continuous storage units, the first correspondence relationship of the first memory access request comprises a correspondence relationship between a thread sending the third memory access request and the storage unit accessed by the first memory access request.

[0071] Optionally, the method further comprises:

[0072] acquiring, by the data returning unit, an encoding corresponding to each first memory access request, the encoding corresponding to a first memory access request being used to determine a target storage unit in a target storage area accessed by the first memory access request, and a bit number of the encoding being less than a total number of storage units in the target storage area;

[0073] returning data in the target storage unit accessed by the first memory access request to the first memory access request based on the encoding corresponding to the first memory access request.

[0074] Optionally, a length of the encoding is determined based on a logarithm of the total number of storage units in the target storage area.

[0075] Optionally, the method further comprises: sending, by the dispatching unit, the encoding corresponding to each first memory access request to the bypass storage unit for storage.

[0076] Optionally, the method further comprises:

[0077] acquiring, by the control unit, a plurality of fourth memory access requests, each fourth memory access request in the plurality of fourth memory access requests being a write request and carrying first data to be written into a target storage unit, and requesting the memory to access the target storage area to write the first data carried by the plurality of fourth memory access requests into storage units in the target storage area.

[0078] Optionally, the method further comprises:

[0079] The first data carried by each fourth access request is written into the target storage region by the control unit after the first data is obtained from the bypass storage unit.

[0080] The first data carried by each fourth access request is written into the target storage region by the control unit after the first data is obtained from the bypass storage unit.

[0081] Optionally, the method further comprises:

[0082] Each access request group includes the plurality of first access requests, and the first access requests in different access request groups are used to access different target storage regions.

[0083] The priority of each access request group is arbitrated, so that the control unit requests the memory to access the target storage region accessed by the first access request in the access request group with the highest priority.

[0084] In an eighth aspect, the embodiments of the present disclosure provide a memory access processing method, which is used to process a plurality of parallel memory access requests, and the memory includes a plurality of storage regions, each of which includes a plurality of storage units; the method comprises:

[0085] A plurality of first access requests accessing the same target storage region are determined, and different first access requests are used to access different target storage units in the target storage region.

[0086] The target storage region is requested to be accessed to write the first data carried by each first access request into the target storage unit in the target storage region.

[0087] Optionally, each access request carries first identification information of the accessed storage region and second identification information of the accessed storage unit in the corresponding storage region; the method further comprises:

[0088] The plurality of first access requests are determined by an arbitration unit based on the first identification information.

[0089] The target storage unit accessed by each first access request is determined by a data return unit based on the second identification information.

[0090] Optionally, at least one of the first access requests is obtained by splitting an access request accessing a plurality of continuous storage units.

[0091] Optionally, the method further comprises:

[0092] The arbitration unit obtains a plurality of memory access request groups, each memory access request group including the plurality of first memory access requests, and the first memory access requests in different memory access request groups are used to access different target storage areas; and the arbitration unit arbitrates the priority of each memory access request group, so that the control unit requests the memory to access the target storage area accessed by the first memory access request in the memory access request group with the highest priority.

[0093] In a ninth aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the method in any of the embodiments.

[0094] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0095] The accompanying drawings, which are incorporated into and form part of the specification, illustrate embodiments consistent with the present disclosure and, together with the specification, serve to explain the technical solutions of the present disclosure.

[0096] Figure 1 is a schematic diagram of a memory partitioning method of an embodiment of the present disclosure.

[0097] Figure 2 is a structural schematic diagram of a memory access processing apparatus of an embodiment of the present disclosure.

[0098] Figure 3A is a schematic diagram of a logical address of an embodiment of the present disclosure.

[0099] Figure 3B is a schematic diagram of two memory units of an embodiment of the present disclosure.

[0100] Figure 3C is a schematic diagram of thread processing of an embodiment of the present disclosure.

[0101] Figure 3D is a schematic diagram of processing of a memory access request of an embodiment of the present disclosure.

[0102] Figure 4 is a structural schematic diagram of a processor of an embodiment of the present disclosure.

[0103] Figure 5 is a structural schematic diagram of another memory access processing apparatus of an embodiment of the present disclosure.

[0104] Figure 6 is a structural schematic diagram of another processor of an embodiment of the present disclosure.

[0105] Figure 7 is a schematic diagram of a chip of an embodiment of the present disclosure.

[0106] Figure 8 is a schematic diagram of a board card of an embodiment of the present disclosure.

[0107] Figure 9A and Figure 9B is a schematic diagram of an electronic device of an embodiment of the present disclosure.

[0108] Figure 10 is a flowchart of a data processing method of an embodiment of the present disclosure.

[0109] Figure 11 is a flowchart of a data processing method of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0110] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, like reference numerals refer to like elements throughout the description. The following exemplary embodiments are not representative of all embodiments consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0111] The terminology used in the present disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used in the present disclosure and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. In addition, the term "at least one of' as used herein means any one of or any combination of one or more of the listed items.

[0112] It should be understood that although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a particular order or hierarchy. These terms are used merely to distinguish one type of information from another. For example, a first information can be termed a second information, and similarly, a second information can be termed a first information, without departing from the scope of the present disclosure. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to a determination."

[0113] In order to enable persons skilled in the art to better understand the technical solutions in the embodiments of the present disclosure, and to make the above-mentioned purposes, features and advantages of the embodiments of the present disclosure more apparent and easy to understand, the technical solutions in the embodiments of the present disclosure will be further described in detail below with reference to the drawings.

[0114] To achieve high-speed computing, many processors incorporate hardware multithreading technology. Taking GPUs as an example, some GPUs include multiple Streaming Multiprocessors (SMs). The unit of execution within an SM is a warp (thread group), and a warp can schedule multiple threads. A thread is equivalent to one instruction and is the smallest unit of execution for the processor. Thread groups can execute in parallel with the support of hardware resources. A warp requires one SM to run, and multiple warps need to take turns entering an SM. A common approach is to group 32 threads into one warp.

[0115] Memory access behavior of multiple threads within a thread group includes reading data from memory and writing data to memory. The latency caused by memory access is increasingly becoming a bottleneck in computing systems. In particular, external memory (sSD) located off-chip (not on the processor) is characterized by large capacity and low bandwidth, posing a significant performance constraint. Therefore, designing mechanisms for thread groups to access memory is crucial in chip designs related to image processing and high-performance computing.

[0116] like Figure 1 As shown, in this embodiment, the memory 10 is divided into multiple storage areas. Figure 1 The dashed boxes in the diagram represent storage areas, showing storage areas 100 to 10i. Each storage area includes M storage units (Banks), where the specific value of M can be determined based on the number of schedulable threads (N) in a warp. Figure 1 Taking 32 banks as an example. To ensure maximum efficiency and effective utilization of hardware resources, M equals N; of course, M being greater than N is also an option.

[0117] For example, external storage and other memory have characteristics such as large capacity, small bandwidth, and limited access paths. When N threads access memory, if some threads access addresses with large spans, such as... Figure 1 As shown, three memory access requests are executed in parallel, each requiring access to memory locations P1, P2, and P3. Because these three locations are far apart and have a large address span, the latency of data transfer in memory increases, resulting in low response efficiency for the memory access requests.

[0118] Based on this, embodiments of this disclosure provide a memory access processing apparatus, such as... Figure 2 As shown, the memory access processing device is used to process multiple parallel memory access requests to the memory 10, the memory being divided into multiple storage regions, each of which includes multiple storage cells;

[0119] The memory access processing device includes:

[0120] Arbitration unit 201 is used to determine multiple first memory access requests to access the same target storage area, and different first memory access requests are used to access different target storage units in the target storage area.

[0121] Control unit 202 is used to request data from each storage cell in the target storage area from the memory.

[0122] The data return unit 203 is used to return the data in the corresponding target storage unit for each of the first memory access requests after the memory returns the data of each storage unit in the target storage area.

[0123] The memory access processing apparatus of this disclosure can be applied to various types of multi-threaded processors such as GPUs, Neural Network Processing Units (NPUs), or CPUs. This disclosure does not limit the type of processor. The processor can schedule multiple threads to process data in parallel.

[0124] For example, the memory access processing device of this embodiment can be implemented using an integrated circuit. The device can be configured in a processor and can handle multiple parallel memory access requests. For example, the memory access processing device of this embodiment can be implemented in an existing processor with hardware multithreading. The memory access requests of multiple threads in the original thread group are processed by the memory access processing device of this embodiment before accessing the memory.

[0125] The memory in this embodiment includes memory accessible to all threads within a thread group. Taking a GPU as an example, it can include shared memory or global memory. Shared memory is on-chip memory; global memory is off-chip memory, which can be implemented using dynamic random access memory (DRAM; also known as video memory). Compared to global memory, shared memory has a smaller capacity, and the span of access locations by multiple threads is relatively small. The solution of this embodiment can also be applied as needed, and the effect of this embodiment is more significant when applied to external memory with large capacity but low bandwidth.

[0126] This embodiment of the disclosure divides the memory into multiple storage regions, each of which includes multiple storage units. When processing multiple parallel memory access requests in a thread group, the arbitration unit can identify multiple first memory access requests that access the same target memory region. The control unit requests data from the memory in units of the target memory region. The memory only needs to read the data of each storage unit in the target memory region once and return it. The data return unit returns the data in the corresponding target memory unit to each first memory access request, thereby ensuring the continuity of the memory read data address and improving the memory access speed.

[0127] For example, each storage area includes N storage units, and the thread group has 5 threads whose memory access requests need to be processed in parallel. These 5 memory access requests may access locations that are far apart. The arbitration unit in this embodiment can determine that 2 of the memory access requests are accessing different storage units in the same target storage area Area1, and the other 3 memory access requests are accessing different storage units in another target storage area Area2. Figure 2 As shown, for two first memory access requests to the target memory area Area1, the control unit can request data from N memory cells in Area1. In the diagram, M is 32 for example. Then, the data return unit returns the data from the corresponding target memory cells to each of the two first memory access requests. For example... Figure 2 The diagram shows the data from Bank31 and Bank5 being returned to the two first memory access requests, respectively.

[0128] Therefore, the memory access processing device in this embodiment does not directly execute multiple memory access requests of the thread group in parallel. Instead, it obtains multiple first memory access requests to access the same target memory area and requests data from the memory at the granularity of the memory area. For the memory, its processing capability is relatively weak. It only needs to operate on the memory areas with contiguous addresses and return all the data stored in each memory cell in the memory area. Subsequently, the data return unit performs data distribution, thereby improving the execution efficiency of multiple first memory access requests.

[0129] For a write request, the memory access request carries first data that needs to be written to the target memory cell. In this embodiment, the control unit can request access to the target memory region from the memory when the memory access request is a write request, so as to write the first data carried by the plurality of first memory access requests to the memory cells in the target memory region. Since the plurality of first memory access requests all access the target memory region, the memory does not need to write data across long addresses, thus improving the memory access efficiency.

[0130] In some examples, each storage region of the memory corresponds to identification information, each storage unit in each storage region corresponds to identification information, and each memory access request carries first identification information of the accessed storage region and second identification information of the accessed storage unit in the corresponding storage region, so that the arbitration unit can be used to determine the plurality of first memory access requests based on the first identification information, and the data return unit can be used to determine the target storage unit accessed by each first memory access request based on the second identification information. The identification information of each storage region and the identification information of each storage unit can be flexibly configured as needed, and the present embodiment does not limit this.

[0131] The number of storage units in each storage region can be set as needed. As an example, each storage region in the memory includes M storage units Bank, and the same position Bank in each storage region can use the same identification, for example, using numbers 0 to (M-1) and the like to represent; of course, those skilled in the art will clearly understand that other ways can be used to represent the identification according to actual conditions, which will not be described here.

[0132] The data bit width and the storage bit width of the Bank accessed by each memory access request can be designed according to the smallest storage unit of the memory. For example, the memory stores data with a byte as the smallest storage unit, and for example, the data size of each memory access request operation can be designed to be only one byte, and the size of the Bank can also be one byte. The amount of information of one byte is small, and if multiple bytes of data need to be accessed, multiple requests are needed, and each request needs to be executed one by one, resulting in low execution efficiency. Alternatively, the data bit width accessed by the memory access request of the thread can be greater than 1 byte. The data bit width accessed by the memory access request of the thread and the storage bit width of the Bank can be the same or different. If the data bit width accessed by the memory access request is greater than the storage bit width of the Bank, the storage unit accessed by one memory access request will have multiple; if the data bit width is less than or equal to the storage bit width, the data accessed by one memory access request will fall into one Bank or cross two Banks. Both are designed to be the same, and the data accessed by some memory access requests is just in one Bank, and the processing of these memory access requests will be relatively efficient.

[0133] According to the storage bit width of the Bank, the Bank can be further divided into multiple storage positions, for example, the storage bit width of the Bank is multiple bytes, and for example, 4 bytes, so that the Bank can be further divided into 4 storage positions, and each storage position stores one byte of data.

[0134] The first identification information and the second identification information carried by the access request can be understood as a logical address of a location of a memory to be accessed by the access request. The logical address has a mapping relationship with a physical address of the memory. According to the division of the memory, the logical address can be used to determine a storage unit in a storage region to be accessed by the access request. In the case where the storage unit is divided into a plurality of storage locations, the logical address can also be used to determine a storage location of the data to be accessed by the access request in the storage unit.

[0135] It is clear to those skilled in the art that the logical address can have various implementations. As an example, the logical address can include an identification of a storage region (which can be referred to as a base address of a Bank), an identification of a storage unit in the storage region, and an identification of a storage location in the storage unit. As shown in Figure 3A the logical address is represented by 40 bits of data, in which the 39th bit to the 7th bit are used to represent the identification of the storage region, and the 6th bit to the 2nd bit are used to represent the identification of the Bank in the storage region; taking the storage bit width of the Bank as 4 bytes and the storage bit width of each storage location as 1 byte, the 1st bit to the 0th bit in the logical address can be used to represent the storage location of the data to be accessed by the access request in the Bank.

[0136] If the data to be operated by the access request is a plurality of bytes, as an example, the identification of the storage location in the logical address can indicate the storage location of the first byte of the data to be accessed by the access request, and the plurality of bytes of data to be operated by the access request can be accessed from the storage location of the first byte and the storage locations above it.

[0137] As an example, please see Figure 3B , Figure 3B (a) shows that Bank1 and Bank2 each include 4 bytes. The data bit width to be accessed by the access request is also 4 bytes.

[0138] If the data to be accessed by the access request is 4 bytes in Bank1, the logical address points to the 0th byte in Bank1, and the 1st bit to the 0th bit in the address can be represented by “00”, that is, Figure 3B (b) uses gray blocks to represent the 4 bytes above the 1st byte in Bank1.

[0139] If the data to be accessed by the access request is from the 1st byte in Bank1 to the 0th byte in Bank2, the logical address points to the 1st byte in Bank1, the identification of the storage unit in the address is the identification of Bank1, and the 1st bit to the 0th bit in the logical address needs to be represented by “01”, that is, Figure 3B(c) the 4 bytes above the 1st byte of Bankl, which spans to the 1st byte of Bank2, are represented by a gray scale block. Thus the memory access request accesses locations across two banks, which is the address misalignment problem, and will affect the subsequent execution efficiency. For example, between the processor obtains the thread group to the final memory access, the thread memory access request may need to be parsed or address conversion or compression and other processing, if the memory access request across two banks, will bring more computational load to the processing of memory access request. For example, in some scenarios, the memory access request is issued from the arbitration unit, and then needs to pass through many buses and caches and other units on the processor to reach the memory, some bus interfaces require address alignment to improve the processing efficiency of memory access request.

[0140] Based on this, in some examples, the memory access processing device can include an alignment unit, which can determine from the plurality of memory access requests that the memory access request accesses a plurality of continuous memory units, and split it to obtain a memory access request that accesses any memory unit in the plurality of continuous memory units.

[0141] For example, the data accessed by the thread memory access request Ql is from the 2nd storage location of Bankl to the 1st storage location of Bank2. The two memory access requests obtained by splitting the memory access request Ql in this embodiment can be: a memory access request Ql 1 accessing the 2nd to 4th storage locations of Bankl, and a memory access request Ql 2 accessing the 1st storage location of Bank2.

[0142] In the case where the memory unit includes a plurality of storage locations, the split memory access request accesses part of the storage locations in the memory unit. For example, the logical address carried by the split memory access request can indicate the first storage location of the memory unit accessed by the memory access request, and carry position identification information for indicating the valid position.

[0143] For example, the logical address of the memory access request Ql 1 includes the identification of the memory unit Bankl, and the 1st to 0th bits of the logical address are "00", indicating that the first storage location of Bankl accessed by the memory access request; the memory access request also carries position identification information for indicating that the valid position is the 2nd to 4th storage locations of Bankl.

[0144] The logical address of the memory access request Ql 2 includes the identification of the memory unit Bank2, and the 1st to 0th bits of the logical address are "00", indicating that the first storage location of Bank2 accessed by the memory access request; the memory access request also carries position identification information for indicating that the valid position is the 1st storage location of Bank2.

[0145] Therefore, after the alignment processing, the first to the zeroth bits of all the memory access requests are "00", so that the byte identifiers of the logical addresses in the memory access requests all indicate the first storage locations of the storage units, i.e., the addresses are all aligned.

[0146] For example, the alignment units can be one or more, for example, in order to process the memory access requests in parallel, the number of the alignment units can be the same as the number of the memory access requests processed in parallel in the thread group, and one alignment unit is used to determine whether one memory access request needs to be split.

[0147] For the plurality of memory access requests processed in parallel, the arbitration unit distinguishes the target storage areas accessed by the memory access requests, so that the plurality of memory access requests processed in parallel can be divided into one or more memory access request groups, each memory access request group includes a plurality of first memory access requests, and the first memory access requests in different memory access request groups are used to access different target storage areas. In the case of a plurality of memory access request groups, the arbitration unit can send the plurality of memory access request groups to the control unit in sequence. As an example, the arbitration unit can also arbitrate the priorities of the memory access request groups, so that the control unit requests the storage to access the target storage area accessed by the first memory access request in the memory access request group with the highest priority. In the foregoing example, the five memory access requests are divided into two memory access request groups, and the two memory access request groups can be executed in sequence. Those skilled in the art will understand that the priorities of the memory access request groups can be arbitrated according to any rule, for example, randomly, or according to the addresses of the target storage areas accessed by the first memory access requests in the memory access request groups, etc. The storage with large capacity and low bandwidth is the slowest link in the execution of the thread group. Although the thread group is divided into a plurality of groups to access the storage in sequence in the embodiment, the execution efficiency is reduced to a certain extent, but for the plurality of first memory access requests in one memory access request group, the storage obtains the data of each storage unit in one storage area with continuous addresses and returns once, without the need to operate across a very long address, so that the data return speed is greatly improved, and thus the execution efficiency of the thread group is still improved.

[0148] For example, the arbitration unit can send the memory access request groups to the control unit in sequence according to the priorities, so that the control unit requests the storage to receive the target storage area accessed by the first memory access request in the memory access request group when receiving the memory access request group. The plurality of first memory access requests in one memory access request group can be sent in parallel, for example, the memory access processing apparatus can include a plurality of instruction paths, each instruction path is used to send a memory access request for accessing a storage unit corresponding to the instruction path.

[0149] As an example, each memory region includes M banks, and at least M instruction channels are provided accordingly, one instruction channel corresponding to one bank, for sending a memory access request for accessing the corresponding bank. As an example, the banks at the same position in each memory region can have the same identification, for example, the identification of the M banks under each memory region can be represented by numbers such as 0 to (M-1). The instruction channels can also be numbered, for example, instruction channel 0 sends instructions for accessing bank 0, instruction channel 1 sends instructions for accessing bank 1, and so on. Based on this, as many memory access requests as possible are responded to by as many instruction channels as possible, and the parallel sending of multiple memory access requests is achieved.

[0150] However, in the above design, the instruction channel can only deliver one memory access request at a time, and if multiple first memory access requests access the same bank, each first memory access request needs to be sent by the corresponding instruction channel one by one, and the blocking of the instruction channel leads to a decrease in execution efficiency.

[0151] For example, as shown in FIG. 1, Figure 3C is a schematic diagram of thread processing in an embodiment of the present disclosure, Figure 3C In (a), each thread of the 32 threads accesses the 32 banks in order, and thus is sent in parallel by the corresponding instruction channel; Figure 3C In (b), although each thread of the 32 threads does not access in order, it is still sent in parallel by the 32 instruction channels; Figure 3C In (c), there are multiple threads accessing the same bank 0, and thus the instruction channel of bank 0 is blocked, and multiple threads need to wait for the instruction channel of bank 0 to respond one by one.

[0152] Based on this, in the present embodiment, the arbitration unit can merge each second memory access request accessing the same memory unit in the same memory region, and thus multiple first memory access requests access different memory units in the same memory region, so that the blocking of the instruction channel is avoided when sending, and the execution efficiency of the memory access request is improved.

[0153] As an example, after the memory access request splitting in the foregoing embodiment is performed, the merging of the memory access request can be performed, and then the division of the memory access request group is performed, and thus at least one of the first memory access request and / or at least one of the second memory access request is obtained by splitting a memory access request accessing multiple continuous memory units, so that the split memory access request can be merged with other memory access requests to reduce the blocking of the instruction channel.

[0154] The types of the threads in one warp are usually the same, in the memory access scenario of the embodiment, are all read types or all write types, and the type information can be carried in the memory access request. The merging of the memory access request of the embodiment can be applied to the two types of memory access requests. For simplicity, two requests are taken as examples, which access the same storage unit in the same storage area, and the following cases can exist:

[0155] ①Both memory access requests access the same storage location in one storage unit; for example, both request the data of the same byte in the storage unit.

[0156] ②Both memory access requests access different storage locations in one storage unit; for example, one memory access request operates the data of the first byte stored in Bank 1, and the other memory access request operates the data of the second byte stored in Bank 1.

[0157] ③The memory locations accessed by the two memory access requests have overlap; for example, one memory access request operates the data of the first byte and the second byte stored in Bank 1, and the other memory access request operates the data of the second byte and the third byte in Bank 1.

[0158] As described above, the above cases can be merged into one memory access request. The new memory access request obtained by merging can be obtained by splicing the original two memory access requests, or a new memory access request can be generated, and the new memory access request carries the information of the original two memory access requests.

[0159] As an example, the memory access requests of the current thread group to be processed are Q A to Q P , which are sent by 16 threads, as shown in FIG. 5, for simplicity, five memory access requests are shown. Figure 3D

[0160] Q A accesses two continuous storage units in the same storage area, and is therefore split into Q a1 and Q a2 .

[0161] Q B and Q a2 access the same storage unit in the same storage area, and are therefore merged into Q a2-B .

[0162] Q C and Q D access the same storage unit in the same storage area, and can be merged into Q C-D .

[0163] The other memory access requests Q F to Q P ​The same processing can also be performed to divide the 16 memory access requests into one or more memory access request groups. Each memory access request group can be sent to the control unit in order according to the priority by the sending unit, so that when the control unit receives a memory access request group, it requests the memory to access the data of the storage area accessed by the first memory access request in the memory access request group.

[0164] The control unit requests data from the memory in the granularity of a storage area, and the data returning unit returns the data in the corresponding target storage unit to each first memory access request after the memory returns the data of each storage unit in the target storage area.

[0165] For example, for Q a1 , Q a2-B , Q C-D and Q E , since Q E is not combined or split, the data returning unit returns data to Q E , which is equivalent to responding to the thread sending Q E .

[0166] For Q C-D , which is obtained by combining Q C and Q D sent by two threads respectively, both of which access data of the same Bank, therefore, when the data returning unit returns data to Q C-D , it can respond to the thread sending Q C and the thread sending Q D .

[0167] For Q a2-B , which is obtained by splitting and then combining, therefore, when the data returning unit returns data to Q a2-B , it can respond to the corresponding thread sending Q a2 and the thread sending Q B .

[0168] For Q a1 , when the data returning unit returns data to Q a1 , it can respond to the corresponding thread Q a1 ; wherein Q a1 and Q a2 are obtained by splitting the thread Q A , and the data returning unit can respond to the thread Q A according to the corresponding data of Q a1 and the corresponding data of Qa2.

[0169] When the data return unit returns data to each first access request in the access request group, the data return unit needs to determine the correspondence between the bank accessed by each first access request and the thread of the first access request. In some examples, the correspondence can be carried in the first access request, and the data return unit determines the correspondence by reading the information carried in the first access request. In other examples, an access request group is sent from the arbitration unit to the control unit, and when the control unit requests access to the memory according to the access request group, the access request can still need to be processed, such as parsing or address conversion, and some information carried in the access request is irrelevant to the request of the control unit to access the memory, such as the correspondence described above, which is needed by the data return unit when returning data. Based on this, the access processing apparatus of the present embodiment further includes a bypass information storage unit that can be used to temporarily store bypass information in the access request. The bypass information indicates temporarily unnecessary information that can be temporarily stored. In actual applications, one or more information in the access request can be set as bypass information by those skilled in the art as needed, and the present embodiment does not limit this. Since the bypass information in the access request can be temporarily stored and does not need to be sent down, other information in the access request is sent down for other processing, thereby improving the sending speed and execution efficiency of the access request.

[0170] As an example, the bypass information can include a first correspondence between a thread sending the access request and a storage unit accessed by the access request; the allocation unit is configured to extract the bypass information from each first access request and send the bypass information to the bypass storage unit for storage; and the data return unit is configured to take the bypass information from the bypass storage unit after the memory returns data of each storage unit in the target storage area, and return data in the corresponding target storage unit to each first access request based on the taken bypass information. For example, the arbitration unit divides a plurality of access requests of a thread group into one or more access request groups, and each access request group is sent to the allocation unit in turn. The allocation unit can store the first correspondence of each access request group.

[0171] Those skilled in the art will appreciate that the storage of the first correspondence can have various implementation manners. For example, the thread sending the access request corresponds to a thread identifier, and the access request carries second identification information of the accessed storage unit in the corresponding storage area. The storage of the first correspondence can be to store the storage thread identifier and the second identification information of each access request.

[0172] As the aforementioned read merge processing, there are cases where multiple threads correspond to one storage unit. For example, the identification of the M storage units under the storage region is represented by M numbers from 0 to M-1. Assuming that the thread group has 32 threads, the memory access requests of thread 31, thread 30 and thread 29 all access Bank 0, and these three memory access requests can be merged. In the first correspondence, Bank 0, thread 31, thread 30 and thread 29 can be represented as:

[0173] Bank 0: 1110…0;

[0174] That is, 32 bits are needed to record the threads corresponding to Bank 0, of which 3 bits are set to 1, indicating thread 31, thread 30 and thread 29 corresponding to Bank 0. From the perspective of Bank 0, three threads need to be corresponded. Moreover, since the number of threads in the thread group that access the same storage unit in the same storage region is unknown, the data bits of the storage corresponding to the threads need to be the same as the number of threads in the thread group, and the larger M is, the larger the data bits are.

[0175] However, from the perspective of the thread, the number of Banks accessed by the thread can be determined. By dividing the storage bit width of the storage unit and setting the data bit width accessed by the memory access request of the thread, the number of storage units accessed by the thread can be determined. For example, in the case where the storage bit width of the storage unit is the same as the data bit width accessed by the memory access request, the Bank accessed by the memory access request of the thread is one or two.

[0176] Therefore, the storage of the correspondence between the two dimensions can be compressed from the perspective of the thread. For example, each storage unit in the storage region can correspond to a code, and the number of bits of the code can be less than the total number of storage units in the storage region. When the Bank accessed by the memory access request of the thread is one, each thread corresponds to only one Bank, and the storage of data is significantly reduced.

[0177] For example, the code of the M storage units in the storage region can be determined according to the logarithm of M. If M is 2 raised to the power of n, the length of the code of the storage unit can be n bits at the minimum; if M is not 2 raised to the power of n, the length of the code of the storage unit can be obtained by taking the logarithm of M and rounding up. Those skilled in the art will understand that the length of the code can be adjusted as needed in actual application, which is not limited in the present embodiment.

[0178] For example, the storage region includes 32 storage units, and the storage unit identification is represented by numbers from 0 to 31. The length of the code of each storage unit identification can be reduced to 5 bits at the minimum.

[0179] For example, from the perspective of the thread, the correspondence between thread 31, thread 30 and thread 29 and Bank 0 can be represented as:

[0180] Thread 31: 0000...1;

[0181] Thread 30: 0000...1;

[0182] Thread 29: 0000...1;

[0183] Wherein the 0th bit is set to 1, indicating Bank0. After encoding of the 32 Banks, the encoding of Bank0 can be 00000. The encoding of other Banks is similar, for example, the encoding of Bank1 can be 00001, and so on, which will not be repeated here. Therefore, by encoding, the amount of data can be reduced, and the bypass information storage unit can be implemented in hardware using a smaller area of circuit, also reducing the hardware power consumption.

[0184] A memory access request group includes memory access requests of multiple threads, and by implementing the encoding of the Banks, the storage of the first corresponding relationship of each first memory access request in the memory access request group can be the storage of the corresponding relationship between the thread identifier of each first memory access request in the memory access request group and the encoding of the Bank.

[0185] Alternatively, the identifier of the thread can also not be stored, for example, since the memory access requests of the multiple threads of the thread group are divided into multiple memory access request groups, only part of the threads can be included in a memory access request group, and the threads not included do not need to respond, the multiple threads of the thread group can be stored in a set order with mark information indicating whether the thread needs to respond. The mark information can have two types, respectively indicating the state of needing to respond and the state of not needing to respond. For example, at least 1 bit of data can be used to indicate, and binary data "0" and "1" can be used to respectively indicate not needing to respond and needing to respond. Of course, those skilled in the art will understand that the mark information will not be listed one by one here. Further, for the threads that need to respond, the corresponding Bank encoding can be stored.

[0186] Alternatively, an array can also be used for storage, the array including multiple elements equal to the number of threads, each element corresponding to a thread, and the threads that do not need to respond can be marked by mark information such as 0; the threads that need to respond store the corresponding Bank encoding. Wherein, the Bank corresponding to the thread can also have multiple, for example, the memory access request of a thread accesses multiple storage units and is split into multiple, and the Bank encoding corresponding to the thread has multiple, as an example, the multiple Bank encodings can be stored by splicing, which will increase the data bit width in the case of splicing storage. Optionally, the arbitration unit can divide the multiple memory access requests obtained by splitting the memory access request of a thread into different memory access request groups, increase the number of memory access request groups, so that in each memory access request group, the Bank corresponding to the thread in the multiple first memory access requests has only one, and the amount of data can be reduced when storing the encoding.

[0187] The decoding process is the same. When the data returning unit returns the data of each first access request for one access request group, the bypass information storage unit can obtain the encoding corresponding to each first access request based on the bypass information storage unit, so as to determine the target storage unit accessed by each first access request in the target storage area, and return the data in the target storage unit accessed by the first access request to the first access request.

[0188] In addition to the above-mentioned first correspondence, in some examples, the bypass information of the access request can also include position identification information, etc. For example, in the above-mentioned embodiment requiring address alignment, the first access request split out only needs to operate part of the storage positions under one Bank, and the first access request carries position identification information, which is used to indicate the effective storage position operated by the access request in the corresponding storage unit. Alternatively, the access request sent by the thread can also have this case, for example, the storage bit width of the storage unit is greater than 1 byte, the storage unit can be divided into multiple storage positions, and the access request sent by the thread only needs to operate part of the storage positions under one Bank, and also carries position identification information. Therefore, the bypass information of the access request stored by the bypass information storage unit can also include position identification information. When the data returning unit returns the data of the corresponding target storage unit to the first access request, the position identification information can be taken out from the bypass information storage unit, and the access request is responded according to the position identification information.

[0189] For the first correspondence between the thread of the access request and the storage unit accessed by the access request, in some examples, it can be further optimized to reduce the amount of data stored by the bypass information storage unit. The allocation unit is configured to: in the case that the plurality of first access requests meet a preset condition, add indication information in each of the first access requests, and send each of the first access requests carrying the indication information to the control unit; the indication information is used to indicate that the first access request meets the preset condition, and the preset condition includes that each of the plurality of first access requests accesses one storage unit, and the thread sending the first access request and the storage unit accessed by the first access request meet a preset correspondence; and the data returning unit is configured to, after the memory returns the data of each storage unit in the target storage area, return the data in the corresponding target storage unit to each of the first access requests based on the preset correspondence.

[0190] The carrying of indication information can be achieved by expanding the signal width of the memory access request; or in some scenarios, some data bits are reserved in the memory access request, which can be used to mark and represent specific meanings as needed, such as the tag signal in the memory access request. By writing a specific tag, such as binary 0 or 1, into the data bit, it can represent the preset condition.

[0191] The preset correspondence can be flexibly determined as needed. For example, it can be determined based on the correspondence between threads and the memory units accessed by the first memory access request, which often occurs in practical applications. There can be one or more preset correspondences, and this embodiment does not limit them.

[0192] For example, a common scenario in practical applications is that there are no address misalignments among the M threads in a thread group (i.e., the first thread does not exist in the thread group), and the memory unit identifiers of the logical addresses in each memory access request correspond one-to-one with M memory units in a memory region. For example, the preset correspondence could be... Figure 3C In cases such as (a) and 3C(b), Threads 0 to 31 access a memory cell in a memory region one by one. The 32 threads form a memory access request group. In this one-to-one correspondence, there is no need to store the first correspondence relationship. The allocation unit can assign each first memory access request an indication information. Optionally, a specific flag, such as binary 0 or 1, can be written into the tag signal of the memory access request to represent the preset condition. Based on this indication information of the first memory access request, the data return unit can determine that the memory access request group to which the first memory access request belongs meets the preset condition and directly use the preset correspondence relationship to return the data in the corresponding target memory cell to each first memory access request. Therefore, when the preset condition is met, there is no need to write the first correspondence relationship to the bypass information storage unit, which reduces the amount of data stored in the bypass information storage unit, optimizes its power consumption and area, and allows the data return unit to respond to each request more quickly.

[0193] For write-type memory access requests, bypass information can be temporarily stored in a bypass information storage unit to improve request execution efficiency. For example, a write-type memory access request carries first data that needs to be written to the target storage unit. As mentioned earlier, in some scenarios, the memory access request does not directly reach the control unit to access the memory after being issued by the arbitration unit. The data to be written to the memory by the memory access request is only needed when accessing the memory. Optionally, in this embodiment, the allocation unit can be used to extract the first data from each first memory access request and send it to the bypass storage unit for storage. The control unit is used to obtain the first data from the bypass information storage unit and then write the first data carried by the multiple first memory access requests to the storage units in the target storage area.

[0194] For example, the dispatch unit receives a group of memory access requests sent by the sending unit, each of which carries first data and other information. The dispatch unit can write the first data of each memory access request to the bypass information storage unit, and continue to send the other information of each memory access request, which can be processed. After the processed other information of each memory access request reaches the control unit, the control unit retrieves the first data of each memory access request from the bypass information storage unit, combines the processed other information of each memory access request, and requests the memory to write data. In this way, unnecessary data transmission can be reduced for write requests, improving the efficiency of sending memory access requests and the efficiency of processing other information in subsequent memory access requests.

[0195] Optionally, different bypass information storage units can be set for different types of memory access requests. For example, the bypass information storage unit can include a read operation bypass information storage unit for storing bypass information of read-type memory access requests and a write operation bypass information storage unit for storing bypass information of write-type memory access requests.

[0196] Next, an embodiment will be described. Referring to Figure 4 FIG. 1 is a schematic diagram of a processor according to an embodiment of the present disclosure. The memory access processing apparatus according to the embodiment can be applied to the processor. As an example, the processor can include a thread block control unit, a strong synchronization unit, and the memory access processing apparatus according to the embodiment.

[0197] (1) The thread block control unit is configured to send a plurality of threads of a thread group warp to the memory access processing apparatus, each thread being a memory access request.

[0198] (2) The thread group synchronization unit is configured to: control each thread (i.e., a thread-level instruction) in a warp, specifically, schedule N threads in a thread group at a time, and obtain the next thread group after waiting for all threads in the thread group to respond. The thread group obtained by the thread group synchronization unit is sent to the address alignment unit.

[0199] (3) The address alignment unit is configured to: determine whether each thread group has the address misalignment problem described above, and split the memory access request of a thread with address misalignment into two memory access requests corresponding to the thread. Optionally, the address alignment unit can have N to process the address misalignment problem of N threads in parallel.

[0200] (4) The arbitration unit can be configured to:

[0201] ① read merge, that is, if the access request is a read request, it is determined whether there are multiple second access requests of the same storage unit under the same storage area, and if so, the multiple second access requests are merged into one first access request.

[0202] ② distinguish access request groups, that is, multiple access requests are divided into one or more access request groups.

[0203] In the case of multiple access request groups, the priority of each access request group can be arbitrated, and each access request group is sent through the instruction path in turn. Multiple first access requests of the same access request group are sent in parallel.

[0204] ③ send access request groups, that is, the data access instruction of the jth Bank is sent through the jth instruction path, and j is an integer from 0 to M-1. In a batch, threads may access a Bank at the same time, for example, write type instructions are not merged. In the case of Bank blocking, the sending order can be set according to the size of the thread id, for example, the thread with a smaller id is sent first.

[0205] (5) Each instruction path, when receiving an access request group issued by the Bank access arbitration unit, sends one or more first access requests of an access request group in parallel.

[0206] (6) The dispatch unit is configured to receive the first access requests of an access request group sent by the multiple instruction paths.

[0207] The dispatch unit can include one or more selectors SEL for dispatching the information carried by the first access request.

[0208] The dispatch unit further includes a read operation bypass information storage unit and a write operation bypass information storage unit.

[0209] For example, through the selector SEL, the dispatch unit stores the bypass information of the read request to the read operation bypass information storage unit when receiving a read request, and stores the bypass information of the write request to the write operation bypass information storage unit when receiving a write request.

[0210] After the access request is extracted from the bypass information, other information of the access request can be used as an instruction and sent to other units downstream through the instruction queue. For example, the access processing device can implement N' instruction queues, each instruction queue is used to send an instruction, thereby realizing parallel processing of N' instructions. The instruction is finally sent to the control unit, and the control unit accesses the memory according to one or more instructions. In order to simplify, Figure 4The other units after the instruction queue are not shown in the figure, and those skilled in the art can understand that in actual application, the instruction can also pass through one or more other functional units before reaching the control unit according to actual needs.

[0211] For a read type access request, the bypass information stored in the read operation bypass information storage unit can include the first correspondence relationship in the foregoing embodiments; or, in the case that the access request carries position identification information, the position identification information and the like can be stored. The storage of the first correspondence relationship can adopt the encoding mode of the foregoing embodiments to reduce the data amount. Wherein, the control unit requests data from the memory, the memory sends data to the data return unit (not shown in the figure) according to the request, and the bypass information of the read operation bypass information storage unit is taken out by the data return unit, which can be decoded by using the decoding mode of the foregoing embodiments when taken out. Figure 4

[0212] The write operation bypass information storage unit is used for storing bypass information of a write type access request. For example, the access request carries data to be written into the memory, which can be temporarily stored in the unit. Other information of the access request is sent to the control unit as an instruction. After the control unit receives the instruction, the data corresponding to the instruction is taken out from the write operation bypass information storage unit and written into the memory.

[0213] Referring to Figure 5 The embodiment of the disclosure provides another access processing device for processing a plurality of parallel access requests to a memory, the memory comprising a plurality of storage regions, each storage region comprising a plurality of storage units; the access processing device comprising:

[0214] An arbitration unit 501 is configured to determine a plurality of first access requests accessing a same target storage region, different first access requests being configured to access different target storage units in the target storage region.

[0215] A control unit 502 is configured to request access to the target storage region from the memory, so as to write first data carried by the respective first access requests into the target storage units in the target storage region.

[0216] Optionally, each access request carries first identification information of the accessed storage region and second identification information of the accessed storage unit in the corresponding storage region.

[0217] The arbitration unit is configured to determine the plurality of first access requests based on the first identification information.

[0218] The data return unit is configured to determine the target storage unit accessed by each first access request based on the second identification information. ​

[0219] Optionally, the at least one first memory access request is obtained by splitting a memory access request that accesses a plurality of continuous memory cells.

[0220] Optionally, the arbitration unit is configured to: obtain a plurality of memory access request groups, each memory access request group including the plurality of first memory access requests, and the first memory access requests in different memory access request groups being configured to access different target memory regions; and arbitrate the priority of each memory access request group, so that the control unit requests the memory to access the target memory region accessed by the first memory access request in the memory access request group with the highest priority.

[0221] Referring to Figure 6 The embodiment of the present disclosure also provides a processor, which comprises the memory access processing apparatus according to any one of the foregoing embodiments.

[0222] Referring to Figure 7 The embodiment of the present disclosure also provides a chip 700, which comprises a processor 701, and the processor 701 can be the processor according to any one of the foregoing embodiments. In some examples, the chip comprises a memory 702, which is connected to the processor 701, and the memory comprises a plurality of memory regions, each of which comprises a plurality of memory cells. Details of the embodiment of the present disclosure are described in the foregoing embodiments, and will not be described here.

[0223] In addition, the embodiment of the present disclosure also provides a board card, which comprises a packaging structure in which at least one chip is packaged. Referring to Figure 8 which provides an exemplary board card 800, the board card 800 comprises the chip 700, and can further comprise other components, including but not limited to: a memory 802, an interface device 804 and a processor 806.

[0224] The memory is connected to the chip in the chip packaging structure through a bus, and is configured to store data. The memory can comprise a plurality of memory regions, each of which comprises a plurality of memory cells, for example, a DDR SDRAM (English: Double Data Rate SDRAM, double data rate synchronous dynamic random access memory) and the like. The memory is connected to the chip through a bus.

[0225] The interface device is electrically connected to the chip in the chip packaging structure. The interface device is configured to realize data transmission between the chip and an external device 808 (for example, a terminal, a server, a camera and the like). In an embodiment, the interface device can comprise a PCIE interface, and can also be a network interface or other interfaces, which are not limited by the present disclosure. Details of the embodiment of the present disclosure are described in the foregoing embodiments, and will not be described here.

[0226] Referring to Figure 9A The electronic device includes a chip 901 and a memory 902. In some examples, the electronic device includes a memory 902 connected to a processor 9011 of the chip 901. Referring to Figure 9B The electronic device includes a board card 800. Details of the electronic device are described in the foregoing embodiments, which will not be repeated here.

[0227] Referring to Figure 10 The method can be applied to the processor of the foregoing embodiments. The method is used for processing a plurality of parallel memory access requests to a memory, the memory including a plurality of memory regions, each memory region including a plurality of memory cells. The method can include:

[0228] Step 1001, determining a plurality of first memory access requests accessing a same target memory region. Different first memory access requests are used to access different target memory cells in the target memory region.

[0229] Step 1002, requesting data of each memory cell in the target memory region from the memory.

[0230] Step 1003, after the memory returns the data of each memory cell in the target memory region, returning data in the corresponding target memory cell for each first memory access request.

[0231] Optionally, in the method, the plurality of first memory access requests accessing the same target memory region are determined by an arbitration unit, and different first memory access requests are used to access different target memory cells in the target memory region.

[0232] The data of each memory cell in the target memory region is requested from the memory by a control unit.

[0233] After the memory returns the data of each memory cell in the target memory region, data in the corresponding target memory cell is returned for each first memory access request by a data returning unit.

[0234] Optionally, each memory access request carries first identification information of the accessed memory region and second identification information of the accessed memory cell in the corresponding memory region.

[0235] The method further includes:

[0236] The plurality of first memory access requests are determined by an arbitration unit based on the first identification information.

[0237] The data returning unit determines a target storage unit accessed by each first memory access request based on the second identification information.

[0238] Optionally, the at least one first memory access request is obtained by merging each second memory access request accessing a same storage unit in a same storage region.

[0239] The method further includes: sending, by the arbitration unit, the first memory access requests to the control unit through a plurality of instruction channels; each instruction channel corresponds to a storage unit in a storage region, and is used to send a first memory access request accessing the storage unit corresponding to the instruction channel.

[0240] Optionally, the at least one first memory access request and / or the at least one second memory access request is obtained by splitting a memory access request accessing a plurality of continuous storage units.

[0241] Optionally, the method further includes:

[0242] The dispatching unit adds indication information in each first memory access request in a case where the plurality of first memory access requests meet a preset condition, and sends each first memory access request carrying the indication information to the control unit; the indication information is used to indicate that the first memory access request meets the preset condition, and the preset condition includes that each first memory access request in the plurality of first memory access requests accesses a storage unit, and a thread sending the first memory access request and a storage unit accessed by the first memory access request meet a preset correspondence relationship.

[0243] The data returning unit returns data in a corresponding target storage unit to each first memory access request based on the preset correspondence relationship after the memory returns data of each storage unit in the target storage region.

[0244] Optionally, each memory access request includes bypass information, and the bypass information includes a first correspondence relationship between a thread sending the memory access request and a storage unit accessed by the memory access request; the method further includes:

[0245] The dispatching unit extracts the bypass information from each first memory access request and sends the bypass information to the bypass storage unit for storage.

[0246] The data returning unit takes out the bypass information from the bypass storage unit after the memory returns data of each storage unit in the target storage region, and returns data in a corresponding target storage unit to each first memory access request based on the taken-out bypass information.

[0247] Optionally, in a case that the first memory access request is obtained by merging a plurality of second memory access requests accessing a same memory cell in a same memory region, the first correspondence relationship of the first memory access request comprises a correspondence relationship between a thread corresponding to each of the second memory access requests and a target memory cell accessed by the first memory access request.

[0248] In a case that the first memory access request is obtained by splitting a third memory access request accessing a plurality of continuous memory cells, the first correspondence relationship of the first memory access request comprises a correspondence relationship between a thread sending the third memory access request and a memory cell accessed by the first memory access request.

[0249] Optionally, the method further comprises:

[0250] obtaining, by a data returning unit, an encoding corresponding to each first memory access request, the encoding corresponding to a first memory access request being used to determine a target memory cell in a target memory region accessed by the first memory access request, wherein a bit number of the encoding is less than a total number of memory cells in the target memory region;

[0251] returning, by the data returning unit, data in the target memory cell accessed by the first memory access request to the first memory access request based on the encoding corresponding to the first memory access request.

[0252] Optionally, a length of the encoding is determined based on a logarithm of the total number of memory cells in the target memory region.

[0253] Optionally, the method further comprises: sending, by the dispatching unit, the encoding corresponding to each first memory access request to the bypass memory unit for storage.

[0254] Optionally, the method further comprises:

[0255] obtaining, by the control unit, a plurality of fourth memory access requests, each of the fourth memory access requests being a write request and carrying first data to be written into a target memory cell, and requesting, by the control unit, the memory to access the target memory region to write the first data carried by each of the fourth memory access requests into a memory cell in the target memory region.

[0256] Optionally, the method further comprises:

[0257] extracting, by the dispatching unit, the first data from each of the fourth memory access requests and sending the first data to the bypass memory unit for storage;

[0258] writing, by the control unit, the first data carried by each of the fourth memory access requests into a memory cell in the target memory region after obtaining the first data from the bypass memory unit.

[0259] Optionally, the method further comprises:

[0260] acquiring, by an arbitration unit, a plurality of memory access request groups, each memory access request group comprising the plurality of first memory access requests, and the first memory access requests in different memory access request groups being used to access different target storage regions;

[0261] arbitrating priorities of the memory access request groups, so that the control unit requests the memory to access the target storage region accessed by the first memory access request in the memory access request group with the highest priority.

[0262] With reference to Figure 11 The embodiments of the present disclosure further provide a memory access processing method, which can be applied to the processor of the foregoing embodiments, and is used for processing a plurality of parallel memory access requests. The memory comprises a plurality of storage regions, and each storage region comprises a plurality of storage units. The method comprises the following steps:

[0263] In step 1102, a plurality of first memory access requests accessing a same target storage region are determined. Different first memory access requests are used to access different target storage units in the target storage region.

[0264] In step 1104, the memory is requested to access the target storage region, so as to write first data carried by the respective first memory access requests into the target storage units in the target storage region.

[0265] Optionally, in the method, the arbitration unit determines a plurality of first memory access requests accessing a same target storage region, and different first memory access requests are used to access different target storage units in the target storage region.

[0266] The control unit requests the memory to access the target storage region, so as to write first data carried by the respective first memory access requests into the target storage units in the target storage region.

[0267] Optionally, each memory access request carries first identification information of a storage region to be accessed and second identification information of a storage unit to be accessed in a corresponding storage region. The method further comprises the following steps:

[0268] The arbitration unit determines the plurality of first memory access requests based on the first identification information.

[0269] The data return unit determines a target storage unit accessed by each first memory access request based on the second identification information.

[0270] Optionally, at least one of the first memory access requests is obtained by splitting a memory access request accessing a plurality of continuous storage units.

[0271] Optionally, the method further comprises:

[0272] acquiring, by the arbitration unit, a plurality of memory access request groups, each memory access request group comprising the plurality of first memory access requests, and the first memory access requests in different memory access request groups being used to access different target storage areas; arbitrating the priority of each memory access request group, so that the control unit requests the memory to access the target storage area accessed by the first memory access request in the memory access request group with the highest priority.

[0273] The embodiments of the present disclosure further provide a computer readable storage medium, which stores a computer program. The program is executed by a processor to implement the memory access processing method in any of the foregoing embodiments.

[0274] Computer readable media includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device. According to the definition herein, computer readable media does not include transitory computer readable media, such as modulated data signals and carriers.

[0275] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of the present specification can be implemented by means of software plus necessary general hardware platforms. Based on such understanding, the technical solutions of the embodiments of the present specification can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in various embodiments or some parts of the embodiments of the present specification.

[0276] The systems, apparatuses, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, and the specific forms of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0277] The various embodiments described in the specification are progressive, and the same or similar parts among the various embodiments can be mutually referred to. Each embodiment focuses on the differences from other embodiments. In particular, the device embodiments are described more simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the part of the method embodiments. The device embodiments described above are merely illustrative, and the modules described as separate components can or can not be physically separated, and the functions of each module can be implemented in one or more software and / or hardware in the implementation of the embodiments of the specification. Some or all of the modules can be selected to achieve the purpose of the embodiments of the specification according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0278] The above is only a specific implementation of the embodiments of the specification, and it should be noted that those skilled in the art can make several improvements and refinements without departing from the principles of the embodiments of the specification, and these improvements and refinements should also be considered within the protection scope of the embodiments of the specification.

Claims

1. A memory access processing apparatus, characterized in that, This is used to process M parallel memory access requests to memory from a thread group containing M threads. The memory includes multiple storage regions, each storage region includes M storage units, and the storage bit width of each storage unit is multiple bytes. The data bit width accessed by the memory access request is the same as the storage bit width of the storage unit. The memory access processing device includes: The alignment unit is used to: determine from the M memory access requests whether there is a memory access request to be split that accesses multiple consecutive memory units; if so, split the memory access request to be split so that each split memory access request accesses one of the multiple consecutive memory units. An arbitration unit is used to determine multiple first memory access requests to access the same target storage area, with different first memory access requests used to access different target storage units in the target storage area; each first memory access request is sent to the control unit through M instruction paths; wherein, at least one first memory access request is obtained by merging various second memory access requests to access the same storage unit in the same storage area; each instruction path corresponds to a storage unit in the storage area and is used to send a first memory access request to access the storage unit corresponding to that instruction path; The control unit is used to initiate an access to the memory with the target storage area as the access granularity, so that the memory reads the entire block of the target storage area with contiguous addresses and returns the data of each storage cell in the target storage area; The data return unit is used to return the data in the corresponding target storage unit for each of the first memory access requests after the memory returns the data of each storage unit in the target storage area.

2. The apparatus according to claim 1, characterized in that, Each memory access request carries the first identification information of the accessed storage area and the second identification information of the accessed storage unit in the corresponding storage area; The arbitration unit is used to determine the plurality of first memory access requests based on the first identification information; The data return unit is used to determine the target storage unit accessed by each first memory access request based on the second identification information.

3. The apparatus according to claim 1, characterized in that, At least one of the first memory access requests and / or at least one of the second memory access requests are obtained by splitting memory access requests that access multiple consecutive memory units.

4. The apparatus according to claim 3, characterized in that, The device further includes a dispensing unit for: When the plurality of first memory access requests meet the preset conditions, an indication information is added to each of the first memory access requests, and each of the first memory access requests carrying the indication information is sent to the control unit. The indication information is used to indicate that the first memory access request meets the preset conditions. The preset conditions include: each of the plurality of first memory access requests accesses a storage unit, and the thread that sends the first memory access request and the storage unit accessed by the first memory access request satisfy a preset correspondence. The data return unit is used to return the data in the corresponding target storage unit to each of the first memory access requests based on the preset correspondence after the memory returns the data of each storage unit in the target storage area.

5. The apparatus according to any one of claims 1 to 4, characterized in that, Each memory access request includes bypass information, which includes a first correspondence between the thread sending the memory access request and the memory unit accessed by the memory access request; the memory access processing device further includes a scheduling unit and a bypass storage unit: The allocation unit is used to extract the bypass information from each first memory access request and send it to the bypass storage unit for storage; The data return unit is configured to retrieve the bypass information from the bypass storage unit after the memory returns the data of each storage unit in the target storage area, and based on the retrieved bypass information, return the data in the corresponding target storage unit to each of the first memory access requests.

6. The apparatus according to claim 5, characterized in that, When the first memory access request is obtained by merging multiple second memory access requests that access the same memory unit in the same memory region, the first correspondence of the first memory access request includes: the correspondence between the thread corresponding to each second memory access request and the target memory unit accessed by the first memory access request; When the first memory access request is split into a third memory access request that accesses multiple consecutive memory units, the first correspondence of the first memory access request includes: the correspondence between the thread that sends the third memory access request and the memory unit accessed by the first memory access request.

7. The apparatus according to claim 5, characterized in that, The data return unit is also used for: Obtain the code corresponding to each first memory access request. The code corresponding to a first memory access request is used to determine the target storage unit in the target storage area accessed by the first memory access request. The number of bits in the encoding is less than the total number of storage cells in the target storage area; Based on the encoding corresponding to the first memory access request, the data in the target storage unit accessed by the first memory access request is returned to the first memory access request.

8. The apparatus according to claim 7, characterized in that, The length of the encoding is determined based on the logarithm of the total number of storage cells in the target storage region.

9. The apparatus according to claim 7, characterized in that, The allocation unit is also used for: The code corresponding to each first memory access request is sent to the bypass storage unit for storage.

10. The apparatus according to any one of claims 1 to 4, characterized in that, The control unit is used for: Obtain multiple fourth memory access requests, each of which is a write request and carries first data to be written to the target storage unit. The memory requests access to the target storage region in order to write the first data carried by the plurality of fourth memory access requests into the storage cells in the target storage region.

11. The apparatus according to claim 10, characterized in that, The device also includes a dispatching unit and a bypass information storage unit; The allocation unit is used to extract the first data from each fourth memory access request and then send it to the bypass storage unit for storage; The control unit is configured to obtain the first data from the bypass storage unit and then write the first data carried by the plurality of fourth memory access requests into the storage unit in the target storage area.

12. The apparatus according to any one of claims 1 to 4, characterized in that, The arbitration unit is used for: Multiple memory access request groups are obtained, each memory access request group includes the multiple first memory access requests, and the first memory access requests in different memory access request groups are used to access different target storage areas; Arbitrate the priorities of each memory access request group so that the control unit requests the memory to access the target memory area accessed by the first memory access request in the highest priority memory access request group.

13. A memory access processing apparatus, characterized in that, This is used to process M parallel memory access requests to memory from a thread group containing M threads. The memory includes multiple storage regions, each storage region includes M storage units, and the storage bit width of each storage unit is multiple bytes. The data bit width accessed by the memory access request is the same as the storage bit width of the storage unit. The memory access processing device includes: The alignment unit is used to: determine from the M memory access requests whether there is a memory access request to be split that accesses multiple consecutive memory units; if so, split the memory access request to be split so that each split memory access request accesses one of the multiple consecutive memory units. An arbitration unit is used to determine multiple first memory access requests to access the same target storage area, with different first memory access requests used to access different target storage units in the target storage area; each first memory access request is sent to the control unit through M instruction paths; wherein, at least one first memory access request is obtained by merging various second memory access requests to access the same storage unit in the same storage area; each instruction path corresponds to a storage unit in the storage area and is used to send a first memory access request to access the storage unit corresponding to that instruction path; The control unit is configured to initiate an access to the memory at the target storage region as the access granularity, so as to write the first data carried by each of the first memory access requests into the target storage cell in the target storage region.

14. A processor, characterized in that, The processor includes the memory access processing apparatus according to any one of claims 1 to 13.

15. A chip, characterized in that, The chip includes the processor of claim 14.

16. The chip according to claim 15, characterized in that, The chip includes a memory connected to the processor; the memory includes multiple storage regions, and each storage region includes multiple storage cells.

17. A circuit board, characterized in that, The board includes a package structure that encapsulates at least one chip as described in claim 15 or 16.

18. An electronic device, characterized in that, The electronic device includes the chip as described in any one of claims 15 or 16, or the board as described in claim 17.

19. The electronic device according to claim 18, characterized in that, The electronic device includes a memory connected to the processor of the chip; the memory includes multiple storage areas, and each storage area includes multiple storage cells.

20. A memory access processing method, characterized in that, The method is used to process M parallel memory access requests to memory from a thread group containing M threads. The memory includes multiple storage regions, each storage region includes M storage units, and the storage width of each storage unit is multiple bytes. The data width accessed by each memory access request is the same as the storage width of the storage unit. The method includes: Determine from the M memory access requests whether there is a memory access request to be split that accesses multiple consecutive memory units; if so, split the memory access request to be split so that each split memory access request accesses one of the multiple consecutive memory units. Multiple first memory access requests are identified to access the same target storage area, and different first memory access requests are used to access different target storage units in the target storage area; each first memory access request is sent through M instruction paths; wherein, at least one first memory access request is obtained by merging various second memory access requests to access the same storage unit in the same storage area; each instruction path corresponds to a storage unit in the storage area and is used to send a first memory access request to access the storage unit corresponding to the instruction path. Using the target storage area as the access granularity, an access to the memory is initiated, causing the memory to read the entire block of the target storage area with contiguous addresses, and return the data of each storage cell in the target storage area; After the memory returns the data of each storage cell in the target storage area, it returns the data in the corresponding target storage cell for each of the first memory access requests.

21. A memory access processing method, characterized in that, The method is used to process M parallel memory access requests to memory from a thread group comprising M threads. The memory includes M storage regions, each storage region including multiple storage units. The storage width of each storage unit is multiple bytes, and the data width accessed by each memory access request is the same as the storage width of the storage unit. The method includes: Determine from the M memory access requests whether there is a memory access request to be split that accesses multiple consecutive memory units; if so, split the memory access request to be split so that each split memory access request accesses one of the multiple consecutive memory units. Multiple first memory access requests are identified to access the same target storage area, and different first memory access requests are used to access different target storage units in the target storage area; each first memory access request is sent through M instruction paths; wherein, at least one first memory access request is obtained by merging various second memory access requests to access the same storage unit in the same storage area; each instruction path corresponds to a storage unit in the storage area and is used to send a first memory access request to access the storage unit corresponding to the instruction path. Using the target storage region as the access granularity, an access to the memory is initiated to write the first data carried by each of the first memory access requests into the target storage cell in the target storage region.

22. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method of claim 20 or 21.

Images