Method for processing multi-thread memory access request, memory access control unit and corresponding processor

Abstract

The invention provides a method for processing a multi-thread memory access request, a memory access control unit and a corresponding processor. Wherein for a plurality of memory access addresses generated for a plurality of threads which are executed concurrently, an expected mode is selected from one or more preset modes by comparing the memory access addresses of adjacent threads; according to the selected expected mode, a basic address is determined, and an expected address corresponding to each thread in the multiple threads is calculated based on the basic address; and in response to determining that the memory access address of each thread is the same as the expected address, generating a compressed memory access request for the plurality of threads. According to the scheme, resource occupation of the address information in transmission and storage processes is reduced, and the utilization rate of internal resources of a processor chip is improved.

Description

Technical Field

[0001] The present application relates to the field of parallel computing, and in particular to the processing of multi-threaded parallel memory access requests. Background Art

[0002] The statements in this section are merely intended to provide background information related to the technical solution of the present application to aid understanding, and they do not necessarily constitute prior art for the technical solution of the present application.

[0003] General-Purpose Graphics Processing Unit (GPGPU) has been widely used in the field of high-performance computing due to its powerful parallel computing capabilities. In GPGPU, a single instruction multiple thread architecture or a single instruction multiple data architecture is usually used to implement parallel data processing, allowing multiple threads executed concurrently to access data at different addresses at the same time, which will generate a large number of memory access requests at the same time. Effectively processing memory access requests from multiple threads will improve the performance of GPGPU chips.

[0004] It should be noted that the above content is only used to help understand the technical solution of the present application, and is not used as a basis for evaluating the prior art of the present application. Summary of the invention

[0005] The present application provides a solution for effectively processing memory access requests from multiple threads by compressing memory access addresses.

[0006] According to a first aspect of an embodiment of the present application, a method for processing multi-threaded memory access requests is provided, comprising: for multiple memory access addresses generated for multiple threads executed concurrently, selecting an expected mode from one or more pre-set modes by comparing the memory access addresses of adjacent threads; according to the selected expected mode, determining a base address and calculating the expected address corresponding to each thread in the multiple threads based on the base address; and in response to determining that the memory access address of each thread is the same as the expected address, generating a compressed memory access request for the multiple threads, the compressed memory access request including at least an identifier of the base address and the expected mode.

[0007] In some embodiments, the method may further include, in response to receiving the compressed memory access request, determining a memory access address corresponding to each thread according to a base address and an identifier of an expected mode included in the compressed memory access request.

[0008] In some embodiments, the compressed memory access request is a request for a first-level cache, which also includes an offset field in the memory access address of each thread.

[0009] In some embodiments, the one or more pre-set modes may include at least:

[0010] The first mode: the memory access addresses of each thread are equal;

[0011] Second mode: the difference between the memory access addresses of adjacent threads is equal;

[0012] The third mode: two adjacent threads form a group, the address difference within each group is the same, the address difference between each group is also equal, but the address difference within a group is different from the address difference between groups;

[0013] The fourth mode: two adjacent threads form a small group, and the address difference within each group is the same; and every four adjacent threads form a large group, and the address difference between the two small groups in each large group is also equal, but the address difference within the group, the address difference between the small groups, and the difference between the starting addresses of two adjacent large groups are all different.

[0014] In some embodiments, the one or more pre-set modes may also include a fifth mode: the memory access addresses of each thread have some of the same bits; and the method may also include directly selecting the fifth mode as the expected mode in response to determining that the multiple memory access addresses are addresses for the first-level cache.

[0015] In some embodiments, the one or more pre-set modes may also include a combination mode: the memory access address of each thread is divided into multiple groups, the difference between the starting addresses of two adjacent groups is equal, the addresses in some groups conform to the second mode, the addresses in some groups conform to the third mode, and the addresses in the remaining groups conform to the fourth mode.

[0016] In some embodiments, according to the selected expected mode, determining a base address and calculating an expected address corresponding to each thread in the plurality of threads based on the base address includes:

[0017] When the expected mode is the first mode: any one memory access address is selected from the multiple memory access addresses as the base address, and the expected address corresponding to each thread is set to the base address;

[0018] When the expected mode is the second mode: taking the memory access address of the first thread as the base address, and taking the address difference between the first thread and the second thread as the unit, the expected address corresponding to each thread is calculated in descending order;

[0019] When the expected mode is the third mode: the minimum address or the maximum address is selected from the memory access addresses of each thread as the base address, and starting from the base address, an arithmetic sequence is constructed with the address difference between the first thread and the second thread as a unit; and two adjacent elements in the constructed arithmetic sequence are respectively exchanged to obtain the expected address corresponding to each thread;

[0020] When the expected mode is the fourth mode: the minimum address or the maximum address is selected from the memory access addresses of each thread as the base address, and starting from the base address, an arithmetic sequence is constructed with the address difference between the first thread and the second thread as the unit; and if the address of the first thread is greater than the address of the second thread, two adjacent elements are grouped as a group, and the two adjacent groups are exchanged in the constructed arithmetic sequence to obtain the expected address corresponding to each thread, otherwise the two adjacent groups are exchanged in the constructed arithmetic sequence, and the two adjacent elements are exchanged in each group to obtain the expected address corresponding to each thread.

[0021] In some embodiments, the method may also include determining whether the multiple memory access addresses are for a first-level cache or a shared memory, and in response to determining that the multiple memory access addresses are for a first-level cache, directly generating a compressed memory access request, the compressed memory access request comprising a common address and an offset field in the memory access address of each thread, the common address being a tag field and an index field common to the memory access addresses of each thread.

[0022] According to a second aspect of an embodiment of the present application, there is provided a memory access control unit for processing multi-threaded memory access requests, comprising an address calculation module and a mode compression mode. The address calculation module is used to generate multiple memory access addresses for multiple threads executed concurrently. The mode compression module is configured to execute the method described in the first aspect of an embodiment of the present application.

[0023] According to a third aspect of an embodiment of the present application, a general-purpose graphics processor is provided, comprising a plurality of processing units that can be executed in parallel, wherein each processing unit comprises a memory access control unit according to the second aspect of an embodiment of the present application.

[0024] According to the fourth aspect of the embodiment of the present application, a general graphics processor including multiple processing units is provided, wherein the processing unit includes a memory access control unit and a shared memory. The memory access control unit is configured to: for multiple memory access addresses generated for shared memory for multiple threads, select an expected mode from one or more pre-set modes by comparing the memory access addresses of adjacent threads; determine the base address according to the selected expected mode and calculate the expected address corresponding to each thread in the multiple threads based on the base address; in response to determining that the memory access address of each thread is the same as the expected address, generate a compressed memory access request for the multiple threads, and the compressed memory access request includes at least the base address and the identifier of the expected mode. And the controller of the shared memory is configured to: in response to receiving the compressed memory access request, determine the memory access address corresponding to each thread according to the base address and the identifier of the expected mode contained in the compressed memory access request.

[0025] In some embodiments, the processing unit may also include a first-level cache, wherein the memory access control unit may also be configured to: for multiple memory access addresses generated for the first-level cache for multiple threads, in response to determining that the multiple memory access addresses have the same tag field and index field, generate a compressed memory access request, the compressed memory access request includes a common address and an offset field in the memory access address of each thread, the common address is a tag field and an index field shared by the memory access addresses of each thread. And wherein the controller of the first-level cache is configured to: in response to receiving the compressed memory access request, restore the memory access address corresponding to each thread according to the common address contained in the compressed memory access request and the offset field in the memory access address of each thread.

[0026] In an embodiment of the present application, a method of compressing multiple memory access address information is provided to process memory access requests of multiple threads. After calculating the memory access address corresponding to each thread, it is detected whether the relationship between these memory access addresses conforms to some specific patterns or rules; if so, such multiple memory access address information can be compressed based on these patterns or rules, thereby reducing the resource occupation of the address information during transmission and storage, thereby improving the utilization rate of the internal resources of the processor chip.

[0027] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings herein are incorporated into the specification and constitute a part of the specification, showing some embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0029] Figure 1 A schematic diagram of a typical structure of a processing unit in a GPGPU;

[0030] Figure 2 A flowchart of a method for processing a multi-threaded memory access request according to an embodiment of the present application;

[0031] Figure 3 Schematic diagram of the structure of a memory access control unit according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of this application more clear, the present application is further described in detail by specific embodiments in conjunction with the accompanying drawings. It should be understood that the described embodiments are part of the embodiments of this application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0033] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0034] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0035] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0036] General-purpose graphics processors (GPGPUs) have been widely used in the field of high-performance computing with their powerful parallel computing capabilities. From a hardware perspective, GPGPUs include multiple processing units that can execute independently in parallel; each processing unit includes multiple computing units; each computing unit (also referred to as a core) executes one thread. The types of computing units may include integer computing units (ALUs, Arithmetic Logic Units), floating-point computing units (FPUs, Floating Point Units), special computing units (SFUs, Special Function Units), tensor computing units, or a combination thereof. From a software perspective, GPGPUs can support concurrent execution of multiple threads, and their scheduling unit is usually a thread block. Each thread block includes multiple threads that execute concurrently, and a thread block will only be scheduled to one processing unit, and multiple threads in the same thread block will only run on the same processing unit.

[0037] Figure 1 A typical structural diagram of a processing unit in a GPGPU is given. Figure 1 As shown in FIG. 1 , the processing unit mainly includes an instruction cache, a scheduler, multiple computing units, a register file, a memory access control unit (Load / Store Unit, LSU, also referred to as a memory access unit), an L1 cache (also referred to as a Level 1 cache), a shared memory, and the like. The instruction cache is used to store the instructions to be executed by the processing unit. The scheduler is equivalent to the controller of the processing unit, which controls the process of reading, decoding, transmitting, executing, and writing back instructions. Each instruction is executed at the granularity of a thread block, and multiple threads of the same thread block are simultaneously loaded to multiple computing units of the processing unit for execution. Each thread can directly access the data in the register, while the access to memory data is performed through the memory access control unit LSU. In GPGPU, each processing unit contains its own dedicated register file, L1 cache, and shared memory. The register file can also be called a register stack, which includes multiple registers, and the register is the fastest storage unit. The L1 cache is designed for situations where registers cannot meet storage requirements, and is mainly used to store large arrays and variables. Shared memory is a storage space that allows multiple threads in the same thread block to access. When each thread block starts executing, a certain amount of shared memory is allocated to it. The address space of this shared memory is shared by all threads in the thread block.

[0038] In such a processing unit, each operation unit can directly read and write registers, while the data in the L1 cache and shared memory must be loaded into the corresponding registers through the memory access control unit before operation. When the processing unit reads data from external storage, it must first load the relevant data from the external storage to the L1 cache or shared memory of the processing unit through the cache system. Correspondingly, when the processing unit writes data to external storage, it also needs to write it into the L1 cache or shared memory first, and then save it to the external storage through the cache system.

[0039] Continue to refer Figure 1After receiving the instruction that needs to access memory, the memory access control unit LSU reads the operand related to the instruction from the corresponding register, calculates the corresponding memory access address according to the obtained operand, and generates the corresponding memory access request based on the calculated memory access address, and sends it to the L1 cache or shared memory. The memory access control unit LSU can determine whether to access the L1 cache or the shared memory according to the operand related to the instruction. In GPGPU, a single instruction multiple threads (SIMT, Single Instruction Multiple Threads) architecture or a single instruction multiple data (SIMD, Single Instruction Multiple Data) architecture is usually used to implement parallel data processing, allowing multiple threads executed concurrently to access data at different addresses at the same time. Therefore, the instructions received by the memory access control unit LSU will involve requests from multiple threads at the same time. When performing address calculation, memory access addresses related to multiple threads will be obtained at the same time, thereby generating multiple memory access requests. The memory access request for the L1 cache is transmitted to the L1 cache controller through the bus between the LSU and the L1 cache. The access request for the shared memory is transmitted to the shared memory controller through the bus between the LSU and the shared memory. The L1 cache controller or shared memory controller will temporarily store these memory access requests and process them in sequence. For example, if the current L1 cache or shared memory has the data required for the corresponding memory access request, the data will be returned to the LSU; if not, it is necessary to wait for the cache system to load the relevant data before processing the corresponding memory access request. It can be seen that these concurrent multiple memory access requests not only increase the consumption of transmission resources between the LSU and the L1 cache or shared memory, but also increase the storage overhead of the L1 cache and shared memory, which is a very unfavorable factor for processor chips with limited internal resources.

[0040] The inventors discovered in the design practice of GPGPU chips that the memory access addresses of multiple threads executed concurrently often have a certain degree of correlation or conform to or follow certain specific rules to a certain extent. Therefore, in an embodiment of the present application, a method of compressing these memory access address information is provided to process the memory access requests of multiple threads. For example, after the LSU calculates the memory access address corresponding to each thread, it detects whether the relationship between these memory access addresses conforms to certain specific patterns or rules; if so, such multiple memory access address information can be compressed based on these patterns or rules, thereby reducing the resource occupation of the address information during transmission and storage, thereby improving the utilization rate of the internal resources of the processor chip.

[0041] Figure 2A flow chart of a method for processing multi-thread memory access requests according to an embodiment of the present application is provided. The method mainly comprises: step S201, for multiple memory access addresses generated for multiple threads executed concurrently, selecting an expected mode of the multiple memory access addresses from one or more pre-set modes by comparing the memory access addresses of adjacent threads; step S202, determining a base address according to the selected expected mode and calculating the expected address corresponding to each thread in the multiple threads based on the base address; step S203, in response to determining that the memory access address of each thread is the same as the expected address, generating a compressed memory access request for the multiple threads, wherein the compressed memory access request at least includes the base address and the identifier of the expected mode. The execution subject of the method is the memory access control unit LSU in the processing unit. After generating the compressed memory access request, the memory access control unit LSU transmits it to the L1 cache controller or the shared memory controller. The L1 cache controller or the shared memory controller that receives the compressed memory access request can calculate the memory access address corresponding to each thread according to the base address and the identifier of the expected mode contained in the compressed memory access request. This method can combine memory access requests from multiple threads into a compressed memory access request, reducing the transmission overhead between the memory access control unit and the L1 cache or shared memory. Accordingly, the L1 cache controller or shared memory controller that receives the compressed memory access request no longer needs to reserve space to store multiple memory access requests, but only needs a small space to store the compressed memory access request, thereby reducing the storage overhead of the L1 cache and shared memory.

[0042] In the embodiments of the present application, the pre-set mode refers to the association relationship or regularity between memory access addresses, including but not limited to:

[0043] The first mode or mode a: the memory access addresses of each thread are equal.

[0044] The second mode or mode b: the differences of the memory access addresses of adjacent threads are equal (that is, the addresses are in an arithmetic sequence), for example, 0x0000 0x0010 0x0020 0x0030 0x0040 0x0050 0x0060 0x0070.

[0045] The third mode or mode c: the memory access addresses of each thread are divided into multiple arithmetic groups, two adjacent threads form a group, the address difference within each group is the same, and the difference between the starting addresses of two adjacent groups (which can be referred to as the difference between groups) is also equal, but the difference between the addresses within each group is different from the difference between the starting addresses of two adjacent groups. The address difference between groups is also the same, but the difference between the addresses within the group is different from the difference between the addresses between groups, for example, 0x0010 0x0000 0x0030 0x0020 0x0050 0x00400x0070 0x0060, where the difference within the group is 10, the order within the group is decreasing, and the difference between groups is 20. It can be seen that mode c is actually based on the arithmetic sequence of mode b, and the two adjacent elements of each group are exchanged two by two. Here, the starting address of each group refers to the memory access address corresponding to the first thread in the group.

[0046] The fourth mode or mode d: the memory access addresses of each thread are divided into multiple arithmetic groups, two adjacent threads form a small group, and the address difference within each group is the same; and every four adjacent threads form a large group, and the difference between the starting addresses of two adjacent small groups in each large group (which can be referred to as the difference between groups) is also equal, but the difference between the addresses within the small group, the difference between the small groups, and the difference between the starting addresses of two adjacent large groups are all different. For example, 0x0020 0x0030 0x0000 0x0010 0x0060 0x0070 0x0040 0x0050, where the difference within the small group is 10, the order within the small group is increasing, the difference between the small groups is 20, but the difference between the starting addresses of the two large groups (0x0420 and 0x0460) is 40. It can be seen that mode d is actually obtained by exchanging the positions of two adjacent groups on the basis of the arithmetic sequence of mode b. For another example, 0x0030 0x0020 0x0010 0x0000 0x0070 0x0060 0x0050 0x0040, where the difference within the group is 10, the order within the group is decreasing, the difference between the groups is 20, but the difference between the start addresses of the two large groups (0x0030 and 0x0070) is 40. It can be seen that pattern d can also be obtained by swapping the positions of two adjacent groups based on the sequence of pattern c.

[0047] The fifth mode or mode 0: the memory access addresses of each thread have some identical bits.

[0048] From the above examples, it can be seen that the address sequences in mode c and mode d can be obtained by exchanging positions based on the arithmetic sequence in mode b, such as exchanging two adjacent elements, exchanging the positions of two adjacent small groups of elements, etc. In other words, the address sequence corresponding to mode c or d can be obtained by making appropriate transformations based on the arithmetic sequence (increasing or decreasing order) of the address.

[0049] The inventors found in actual research that the access to the L1 cache is more inclined to the access of local data. Usually, multiple addresses access the data in the same cache block, so the memory access address mostly presents the situation of mode 0. The access to the shared memory is more inclined to the distributed and discrete situation. The modes that appear more frequently in the multiple access addresses to the shared memory are modes b, c, and d. In GPGPU, in order to obtain high bandwidth during parallel access, the shared memory is divided into equal-sized storage bodies (called banks) that can be accessed simultaneously. Since different banks can work simultaneously without interfering with each other, access to multiple addresses located on different banks can be performed simultaneously, but the same bank cannot respond to multiple requests at the same time. Therefore, if multiple memory access addresses from multiple threads are located in the same bank, then these requests must be processed serially, thereby reducing the memory access efficiency, which is a bank conflict. Since the shared memory is a software-controllable high-speed cache, some specific address crossing, mapping and other methods are usually used in application programming to minimize bank conflicts, so that the memory access addresses of multiple concurrent threads are distributed on different banks as much as possible, rather than mapped to the same bank. Therefore, modes b, c, and d are more common, but mode a may also occur. How to resolve bank conflicts belongs to the prior art and will not be described here. The present application is concerned with detecting the internal relationship or regularity of multiple memory access addresses generated.

[0050] In addition to the above-mentioned modes a, b, c, and d, the inventors have also found that in accessing shared memory, a combination mode often occurs, that is, the memory access addresses of each thread are divided into multiple groups, and the difference between the starting addresses of two adjacent groups is equal, but the addresses in some groups conform to the second mode (mode b), the addresses in some groups conform to the third mode (mode c), and the addresses in the remaining groups conform to the fourth mode (mode d). The combination mode usually occurs when the number of concurrent threads is 32 or more. Taking the common number of concurrently executed threads as 32 as an example, the memory access addresses are shown in Table 1:

[0051] Table 1

[0052]

[0053] The memory access addresses of the 32 threads are divided into 4 groups (8 in each group). If the 8 addresses in each group are arranged from small to large or from large to small, they are all arithmetic sequences. However, in fact, some positions are exchanged or crossed, and the starting addresses between groups are arithmetic. The first group of addresses 0 to 8 are normal arithmetic sequences without position exchange (in mode b); the 8th to 15th addresses are obtained by exchanging two adjacent elements in the arithmetic sequence (in mode c); the 16th to 23rd addresses are obtained by exchanging two adjacent groups in the arithmetic sequence (in mode d); the 24th to 31st addresses are obtained by exchanging two adjacent groups in the arithmetic sequence (in mode d). The starting addresses of each group are 0x0000, 0x0210, 0x0420, and 0x0630, which are also arithmetic. For the convenience of description, the following description uses 32 concurrent threads corresponding to 32 memory access addresses as an example, but it should be understood that no quantity limitation is imposed.

[0054] The fifth mode or mode 0 often appears between multiple memory access addresses for the L1 cache. The memory access address for the L1 cache is different from the memory access address for the shared memory. As mentioned above, the memory access control unit LSU can determine whether it is a memory access operation for the L1 cache or the shared memory according to the operand related to the instruction, thereby generating a corresponding memory access address. The memory access address for the L1 cache usually includes three parts: a tag, an index, and an offset. The offset is used to address a certain data in the cache block, the index is used to locate a certain group in the L1 cache, and the tag is used to compare with the corresponding tags of each cache block contained in the group specified by the index to determine whether it hits. In this way, when the L1 cache controller receives a memory access request, it can locate the corresponding group through the index field of the address in the memory access request, take the tag field in the memory access address and compare it with the tags of each cache block contained in the group. If there is a matching tag, it means that the cache hits, and the corresponding data in the cache block corresponding to the matched tag can be extracted according to the offset field in the memory access address and returned to the LSU. Therefore, multiple memory access addresses of the L1 cache often conform to the fourth mode (mode 0), that is, the memory access addresses of each thread have some identical bits (for example, the tag field and the index field are the same, but the offset field is different).

[0055] refer to Figure 2More specifically, in step S201, after generating multiple memory access addresses for multiple threads executed concurrently, the memory access control unit can further detect whether the relationship between these memory access addresses conforms to one or more pre-set patterns. If a certain pattern is met, the pattern can be selected as the expected pattern of these memory access addresses. If no matching pattern is found, a corresponding memory access request is generated for the memory access address of each thread in a conventional manner.

[0056] In one embodiment, by comparing the memory access addresses of adjacent threads, one or two modes can be selected from one or more pre-set modes as the expected mode of the generated multiple memory access addresses. For example, the memory access addresses of thread 0 and thread 1 are compared. If the two are the same, mode a is selected as the expected mode; if the memory access address of thread 1 is greater than the memory access address of thread 0, mode b and / or mode d are selected as the expected mode; if the address of thread 0 is greater than the memory access address of thread 1, mode c and / or mode d can be selected as the expected mode, that is, mode d or mode c can be selected as the expected mode, or mode c and mode d can be selected as the expected mode. The inventor found in actual research that most of the address crossing methods used to solve bank conflicts are often two-by-two crossing or two-by-two position exchange. Therefore, in this embodiment, by comparing two adjacent threads in this way, the expected mode can be quickly determined or selected for multiple memory access addresses.

[0057] In another embodiment, in order to more accurately determine the regularity that may exist between multiple memory access addresses, the memory access addresses of a portion of the threads (for example, at least 3 or more) can be detected. In fact, the memory access addresses of adjacent threads are also compared, but the difference is that the memory access addresses of at least 3 or more consecutive adjacent threads are compared. For example, the memory access addresses of thread 0, thread 1, thread 2, and thread 3 are compared. If all are the same, mode a is selected as the expected mode; if they increase or decrease in sequence, mode b is selected as the expected mode; if the memory access address of thread 0 is greater than the memory access address of thread 1, the memory access address of thread 3 is greater than the memory access address of thread 2, and the address difference between thread 0 and thread 3 is greater than the address difference between thread 0 and thread 1, mode c is selected as the expected mode, otherwise mode d can be selected as the expected mode. In another embodiment, the memory access addresses of at least three large groups (a total of 12 adjacent threads) can also be checked in groups of four. If any of modes a, b, c, and d are not met, the expected mode is directly set to the combined mode.

[0058] It should be understood that in the above embodiments, modes a, b, c, d and the combination mode are for memory access addresses of shared memory, while for multiple memory access addresses of L1 cache, mode 0 can be directly selected as the expected mode.

[0059] Continue to refer Figure 2 , in step S202, according to the expected mode selected in step S201, the base address is determined and the expected address corresponding to each thread is calculated based on the base address. If the selected expected mode is mode a (i.e., the memory access addresses of each thread are equal), any memory access address can be selected from the current multiple memory access addresses as the base address, and the expected address corresponding to each thread is set to the base address. If the selected expected mode is mode b (i.e., the difference in memory access addresses of adjacent threads is equal), the memory access address of the first thread is used as the base address. If the address of the first thread is greater than the address of the second thread, starting from the base address, the address difference between the first thread and the second thread is used as the unit to sequentially decrease in sequence to calculate the expected address corresponding to each thread; otherwise, the expected address corresponding to each thread is calculated by sequentially increasing in sequence.

[0060] If the expected mode selected is mode c (for example, two threads are a group, and the address difference within the group is different from the address difference between the groups), the address difference within the group and the order within the group (for example, descending order) are determined by comparing the addresses of the first thread and the second thread, the address difference between the groups is determined by comparing the addresses of the first thread and the third thread, and the minimum address or the maximum address is selected from the memory access addresses of each thread as the base address. Starting from the base address, the expected addresses of adjacent threads belonging to the same group as the thread corresponding to the base address are first constructed based on the address difference within the group and the order within the group, and then the expected addresses of each thread of another group adjacent to the group to which the base address belongs are constructed based on the address difference between the groups. According to this mode c, the expected addresses corresponding to each thread are constructed in sequence. For example, taking 8 memory access addresses 0x0210 0x0200 0x0230 0x0220 0x0250 0x0240 0x0270 0x0260 corresponding to 8 threads as an example, two threads are in a group, the address difference within the group is 10 and the order within the group is decreasing, and the address difference between groups is 20. First, the smallest address 0x0200 is selected as the base address. The adjacent thread that belongs to the same group is the first thread. Since the order within the group is decreasing, the expected address of the first thread is 0x0210; and according to the address difference between groups, the expected starting address of another adjacent group is calculated to be 0x0230 (that is, the expected address of the third thread), and since the order within the group is decreasing, the expected address of the fourth thread belonging to the adjacent group is 0x0220, and so on. For another example, the minimum address or the maximum address may be selected from the memory access addresses of each thread as the base address, and starting from the base address, an arithmetic sequence may be constructed with the address difference within the group as a unit, and then the positions of two adjacent addresses in the constructed arithmetic sequence may be swapped to obtain the expected address corresponding to each thread.

[0061] If the expected mode selected is mode d (two adjacent threads form a small group, and every four adjacent threads form a large group), by comparing the addresses of the first thread and the second thread, determine the address difference within the group and the order within the group (for example, increasing or decreasing order), select the minimum address or the maximum address from the memory access address of each thread as the base address, and start from the base address to construct an arithmetic sequence with the address difference within the group as the unit. If the order within the group is increasing order, then the positions of the two adjacent groups are exchanged in the constructed arithmetic sequence to obtain the expected address corresponding to each thread. If the order within the group is decreasing order, then the positions of the two adjacent groups are exchanged in the constructed arithmetic sequence, and the two adjacent addresses are exchanged in the group to obtain the expected address corresponding to each thread.

[0062] If the expected mode selected is the combined mode (usually occurs when the number of concurrent threads exceeds 32 and is an integer multiple of 8), the 8 threads are divided into a group, and the first two or three threads of each group are tested to determine which mode a, b, c, d is applicable to this group to generate the expected address. After determining the applicable mode for each group, the corresponding expected address is generated for each group in the manner described above.

[0063] If the selected expected mode is mode 0, the same part of the memory access address of each thread (ie, the tag field and the index field) is used as the base address, and the memory access address of each thread is directly set to the expected address corresponding to each thread.

[0064] In step S203, for the expected addresses corresponding to the threads calculated in step S202, the expected addresses of each thread are compared with the memory access addresses of the thread. If the expected addresses of one or more threads are different from their actual memory access addresses, corresponding memory access requests are generated based on the memory access addresses of each thread. If the expected addresses of each of these threads are the same as their corresponding memory access addresses, a compressed memory access request is generated, which includes at least the base address determined in step S202 and an identifier of the expected mode. The compressed memory access request for the L1 cache also includes different parts of the memory access addresses of each thread (i.e., the offset field).

[0065] In another embodiment, the method for processing multi-threaded memory access requests may further include the step of calculating the memory access address corresponding to each thread in response to receiving a compressed memory access request. For the compressed memory access request of the shared memory, the memory access address corresponding to each thread can be calculated based on the base address and the identifier of the expected mode contained in the compressed memory access request. The method is the same as the method for calculating the expected address corresponding to each thread introduced in conjunction with step S202 above, and will not be repeated here. For the compressed memory access request of the L1 cache, the memory access address of each thread can be restored respectively according to the base address (tag field and index field) contained therein and the offset field corresponding to each thread.

[0066] In another embodiment, the method for processing multi-thread memory access requests described above in conjunction with steps S201-203 may also be used only to process multiple memory access requests for shared memory, wherein one or more pre-set modes include modes a, b, c, d, and a combination mode, but does not include mode 0. For multiple memory access addresses of the L1 cache, when it is detected that the same index field and tag field exist in each memory access address, a compressed memory access request may be directly generated, and the compressed memory access request includes a common address (i.e., the tag field and index field shared by each thread memory access address) and an offset field in each thread memory access address.

[0067] Figure 3 The schematic diagram of the structure of the memory access control unit according to an embodiment of the present application is shown. The memory access control unit LSU mainly includes an address calculation module and a mode compression module. As mentioned above, the memory access control unit LSU receives an instruction that needs to be accessed from the scheduler, reads an operand related to the instruction from the register file, and the address calculation module is used to calculate the memory access address corresponding to each thread according to the obtained operand. The mode compression module selects an expected mode of multiple memory access addresses from one or more pre-set modes by comparing the memory access addresses of adjacent threads, determines a base address according to the selected expected mode, and calculates the expected address corresponding to each thread in the multiple threads based on the base address, and generates compressed memory access requests for the multiple threads in response to determining that the memory access address of each thread is the same as the expected address. The above process is similar to the process described above in conjunction with steps S201-S203, and will not be repeated here. The memory access control unit LSU can determine whether to access the L1 cache or the shared memory according to the instruction-related operand. The memory access request for the L1 cache is transmitted to the controller of the L1 cache via the bus between the LSU and the L1 cache. The access request for the shared memory is transmitted to the shared memory controller through the bus between the LSU and the shared memory. The L1 cache and the shared memory controller will temporarily store these memory access requests and process them in sequence.

[0068] In another embodiment of the present application, a general-purpose graphics processor is provided, comprising a plurality of processing units that can be executed in parallel, wherein each processing unit comprises the above-mentioned Figure 3 The memory access control unit introduced.

[0069] In another embodiment of the present application, there is also a general graphics processor including multiple processing units, wherein the processing unit includes a memory access control unit and a shared memory. The memory access control unit is configured to: for multiple memory access addresses generated for the shared memory for multiple threads, select an expected mode from one or more pre-set modes by comparing the memory access addresses of adjacent threads; determine the base address according to the selected expected mode and calculate the expected address corresponding to each thread in the multiple threads based on the base address; in response to determining that the memory access address of each thread is the same as the expected address, generate a compressed memory access request for the multiple threads, and the compressed memory access request includes at least the base address and the identifier of the expected mode. The process of selecting the expected mode and calculating the expected address of each thread is similar to the process described above in conjunction with steps S201 and S202, and will not be repeated here. In addition, the controller of the shared memory is configured to determine the memory access address corresponding to each thread according to the base address and the identifier of the expected mode contained in the compressed memory access request as described above in response to receiving the compressed memory access request.

[0070] In another embodiment, the processing unit may also include an L1 cache, wherein the memory access control unit may also be configured to: for multiple memory access addresses generated for the L1 cache for multiple threads, in response to determining that the multiple memory access addresses have the same tag field and index field, generate a compressed memory access request, the compressed memory access request includes a common address and an offset field in the memory access address of each thread, the common address is a tag field and an index field shared by the memory access addresses of each thread. And wherein the controller of the L1 cache is configured to, in response to receiving the compressed memory access request, restore the memory access address corresponding to each thread according to the common address contained in the compressed memory access request and the offset field in the memory access address of each thread.

[0071] It should be understood that, for the steps, processes or methods mentioned in this article, in addition to being implemented in a purely computer-readable program code, it is entirely possible to implement the same functions by logically programming the corresponding functional modules, processes or steps so that these modules are in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the module or device implemented in this way can be considered as a hardware component, and the submodules included therein for implementing various functions can also be regarded as the internal structure of the hardware component. Alternatively, the device for implementing various functions can be regarded as both a software module for implementing the relevant process or method steps and a structure within the hardware component.

[0072] References in this specification to "various embodiments," "some embodiments," "one embodiment," or "an embodiment," etc. refer to a particular feature, structure, or property described in conjunction with the above-described embodiment being included in at least one embodiment. Thus, the appearance of the phrases "in various embodiments," "in some embodiments," "in one embodiment," or "in an embodiment," etc., in various places throughout the specification do not necessarily refer to the same embodiment. Furthermore, particular features, structures, or properties may be combined in any suitable manner in one or more embodiments. Thus, particular features, structures, or properties shown or described in conjunction with one embodiment may be combined in whole or in part with features, structures, or properties of one or more other embodiments without restriction, as long as the combination is not illogical or inoperable.

[0073] The expressions of "including", "having" and similar meanings in this specification are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices. "One" or "an" does not exclude multiple situations. In addition, the elements in the drawings of this application are for illustrative purposes only and are not drawn to scale.

[0074] Although the present application has been described through the above-mentioned embodiments, the present application is not limited to the embodiments described herein, and also includes various changes and modifications made without departing from the scope of the present application.

Claims

1. A method for processing a multi-threaded memory access request, comprising: For a plurality of memory access addresses generated for a plurality of threads executing concurrently, selecting an expected pattern from one or more pre-set patterns by comparing memory access addresses of adjacent threads; Determine a base address according to the selected expected mode and calculate an expected address corresponding to each thread in the plurality of threads based on the base address; In response to determining that the memory access address of each thread is the same as the expected address, a compressed memory access request is generated for the multiple threads, wherein the compressed memory access request includes at least a base address and an identification of the expected mode.

2. The method according to claim 1 further comprises, in response to receiving the compressed memory access request, determining the memory access address corresponding to each thread according to the base address and the identifier of the expected mode contained in the compressed memory access request. 3 . The method according to claim 1 , wherein the compressed memory access request is a request for a first-level cache, and further comprises an offset field in a memory access address of each thread.

4. The method according to claim 1, wherein the one or more pre-set modes at least include: The first mode: the memory access addresses of each thread are equal; Second mode: the difference between the memory access addresses of adjacent threads is equal; The third mode: two adjacent threads form a group, the address difference within each group is the same, the address difference between each group is also equal, but the address difference within a group is different from the address difference between groups; The fourth mode: two adjacent threads form a small group, and the address difference within each group is the same; and every four adjacent threads form a large group, and the address difference between the two small groups in each large group is also equal, but the address difference within the group, the address difference between the small groups, and the difference between the starting addresses of two adjacent large groups are all different.

5. According to the method described in any one of claims 1-4, the one or more pre-set modes also include a fifth mode: the memory access addresses of each thread have some identical bits; and the method also includes directly selecting the fifth mode as the expected mode in response to determining that the multiple memory access addresses are addresses for the first-level cache.

6. According to the method described in claim 4, the one or more pre-set modes also include a combination mode: the memory access addresses of each thread are divided into multiple groups, the difference between the starting addresses of two adjacent groups is equal, the addresses in some groups conform to the second mode, the addresses in some groups conform to the third mode, and the addresses in the remaining groups conform to the fourth mode.

7. The method according to claim 4, wherein selecting the expected mode from one or more preset modes by comparing the memory access addresses of adjacent threads comprises: In response to determining that the memory access addresses of the first thread and the second thread are the same, selecting the first mode as the expected mode; In response to determining that the memory access address of the first thread is greater than the memory access address of the second thread, selecting the second mode and / or the fourth mode as the expected mode; In response to determining that the memory access address of the first thread is smaller than the memory access address of the second thread, the third mode and / or the fourth mode is selected as the expected mode.

8. The method according to claim 6, wherein selecting the expected mode from one or more preset modes by comparing the memory access addresses of adjacent threads comprises comparing the memory access addresses of at least 12 adjacent threads, and if the expected mode does not conform to any one of the first mode, the second mode, the third mode and the fourth mode, directly setting the expected mode to the combined mode.

9. The method according to claim 4, wherein according to the selected expected mode, determining a base address and calculating the expected address corresponding to each thread in the plurality of threads based on the base address comprises: When the expected mode is the first mode: any one memory access address is selected from the multiple memory access addresses as the base address, and the expected address corresponding to each thread is set to the base address; When the expected mode is the second mode: taking the memory access address of the first thread as the base address, and taking the address difference between the first thread and the second thread as the unit, the expected address corresponding to each thread is calculated in descending order; When the expected mode is the third mode: the minimum address or the maximum address is selected from the memory access addresses of each thread as the base address, and starting from the base address, an arithmetic sequence is constructed with the address difference between the first thread and the second thread as a unit; and two adjacent elements in the constructed arithmetic sequence are respectively exchanged to obtain the expected address corresponding to each thread; When the expected mode is the fourth mode: the minimum address or the maximum address is selected from the memory access addresses of each thread as the base address, and starting from the base address, an arithmetic sequence is constructed with the address difference between the first thread and the second thread as the unit; and if the address of the first thread is greater than the address of the second thread, two adjacent elements are grouped as a group, and the two adjacent groups are exchanged in the constructed arithmetic sequence to obtain the expected address corresponding to each thread, otherwise the two adjacent groups are exchanged in the constructed arithmetic sequence, and the two adjacent elements are exchanged in each group to obtain the expected address corresponding to each thread.

10. The method according to claim 1 further includes determining whether the multiple memory access addresses are for a first-level cache or a shared memory, and in response to determining that the multiple memory access addresses are for a first-level cache, directly generating a compressed memory access request, the compressed memory access request comprising a common address and an offset field in the memory access address of each thread, the common address being a tag field and an index field common to the memory access addresses of each thread.

11. A memory access control unit for processing multi-threaded memory access requests, comprising an address calculation module and a mode compression mode, An address calculation module, used to generate multiple memory access addresses for multiple threads executed concurrently; Pattern compression module for: Selecting an expected pattern from one or more pre-set patterns by comparing memory access addresses of adjacent threads; Determine a base address according to the selected expected mode and calculate an expected address corresponding to each thread in the plurality of threads based on the base address; In response to determining that the memory access address of each thread is the same as the expected address, a compressed memory access request is generated for the multiple threads, wherein the compressed memory access request includes at least a base address and an identification of the expected mode.

12. A general-purpose graphics processor, comprising a plurality of processing units capable of executing in parallel, wherein each processing unit comprises the memory access control unit according to claim 11.

13. A general-purpose graphics processor comprising a plurality of processing units, wherein the processing units include a memory access control unit and a shared memory, and the memory access control unit is configured as follows: For a plurality of memory access addresses for a shared memory generated for a plurality of threads, selecting an expected pattern from one or more pre-set patterns by comparing memory access addresses of adjacent threads; Determine a base address according to the selected expected mode and calculate an expected address corresponding to each thread in the plurality of threads based on the base address; In response to determining that the memory access address of each thread is the same as the expected address, generating a compressed memory access request for the plurality of threads, the compressed memory access request including at least a base address and an identification of the expected mode; And wherein the shared memory controller is configured to: in response to receiving the compressed memory access request, determine the memory access address corresponding to each thread according to the base address and the identifier of the expected mode contained in the compressed memory access request.

14. The general purpose graphics processor according to claim 13, wherein the processing unit further comprises a level 1 cache, wherein the memory access control unit is further configured to: For a plurality of memory access addresses generated for a first-level cache for a plurality of threads, in response to determining that the plurality of memory access addresses have the same tag field and index field, generating a compressed memory access request, the compressed memory access request comprising a common address and an offset field in the memory access addresses of each thread, the common address being the tag field and the index field common to the memory access addresses of each thread; And the controller of the first-level cache is configured to: in response to receiving the compressed memory access request, restore the memory access address corresponding to each thread according to the common address contained in the compressed memory access request and the offset field in the memory access address of each thread.

Images