Private memory access method and device, computer equipment, medium and product

Abstract

The invention relates to a private memory method and device, computer equipment, a medium and a product. The method comprises the steps that when thread bundles are started, at least one unused thread bundle ID is applied to be obtained, and the thread bundle ID is identification information used for distinguishing different thread bundles; after a thread corresponding to the unused thread bundle ID is started, reading a first instruction from an instruction cache; when the first instruction is a read or write operation for a private memory, determining an address of the private memory; and reading the data from the address corresponding to the private memory, or writing the data into the address corresponding to the private memory. Therefore, when register files are not enough in the thread bundle execution process, the data can be temporarily stored in the private memory firstly, execution of an instruction pipeline does not need to be interrupted, the instruction execution speed is increased, the storage space of the private memory is fully utilized for data storage, the number of times of accessing the private memory is reduced, and the access efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, apparatus, computer device, computer-readable storage medium and computer program product for accessing a private memory. Background Art

[0002] The memory in the graphics processing unit (GPU) can be mainly divided into the following categories according to the functional definition: local memory, register file, cache, global memory, constant memory, texture memory and private memory. Among them, private memory is the name in OpenCL (full name Open Computing Language), and it is called local memory in CUDA (universal parallel computing architecture). Each processing unit has a private memory, and the processing units cannot access each other. Generally, when there are too many register files used by the processing unit, resulting in no register file to store the result after the operation, the data in some register files will be temporarily saved in the private memory. When the used register files are released or the data is needed, the data will be taken out of the private memory and written to the register file.

[0003] In traditional technology, when a program needs to use private memory, private storage space needs to be allocated to each processing unit, so this memory may be very large. Therefore, a space is generally allocated from the global memory outside the chip as private memory.

[0004] However, when the processing unit uses too many register files and there is no register file to store the calculation results in the subsequent operation process, the instruction pipeline has to stop and wait for the register file to be released until there are enough register files. The instruction pipeline will restart to execute the subsequent instructions, which will result in too low chip execution efficiency. Summary of the invention

[0005] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for accessing a private memory that can improve chip execution efficiency without interrupting the execution of the instruction pipeline in order to address the above technical problems.

[0006] In a first aspect, the present application provides a method for accessing a private memory, comprising:

[0007] When a thread warp is started, applying for obtaining at least one unused thread warp ID, where the thread warp ID is identification information used to distinguish different thread warps;

[0008] After starting the thread corresponding to the unused thread warp ID, reading a first instruction from an instruction cache;

[0009] When the first instruction is a read or write operation for a private memory, determining an address of the private memory;

[0010] Data is read from the address corresponding to the private memory, or data is written into the address corresponding to the private memory.

[0011] In one embodiment, before reading the first instruction from the instruction cache module, the method further includes:

[0012] Apply for a total private storage space from the global memory as private memory through the driver;

[0013] When the array used in the kernel function is larger than a preset value, or the compiler cannot statically determine the size of the array, a private storage space is applied for each processing unit of the thread warp in the private memory through the driver.

[0014] In one embodiment, before applying for a private storage space for each processing unit of the thread warp in the private memory through the driver, the method further includes:

[0015] Memory arrangements of private storage spaces of the first-type warps and the second-type warps are determined respectively.

[0016] In one embodiment, determining the memory arrangement of the private storage space of the first type of warp and the second type of warp respectively includes:

[0017] The size of the private storage space of each processing unit is recorded as stride; wherein the total number of strides is a multiple of 4, and the unit of stride is byte;

[0018] Set the size of the private storage space occupied by the first type of thread warp to 32 times the stride, and set the size of the private storage space occupied by the second type of thread warp to 64 times the stride.

[0019] The stride is split into components of equal size, each component being 4 bytes in size; wherein the components of the processing unit are not contiguous in memory.

[0020] In one embodiment, after applying for a private storage space for each processing unit of the warp in the private memory through the driver, the method further includes:

[0021] If the thread warp is finished running, the private storage space occupied by each processing unit of the thread warp is released;

[0022] The freed private storage space is allocated to the newly started warp by the driver.

[0023] In one embodiment, determining the address of the private memory includes:

[0024] The first instruction is parsed by a compiler to obtain predicate register indication information, target address register indication information, address offset information of a private storage space of a processing unit, and data format information for writing or reading a private memory, wherein the predicate register indication information is used to determine a processing unit in a thread warp that executes the first instruction, the target address register indication information is used to determine the number of channels for writing or reading a private memory, and the address offset information of the private storage space of the processing unit is used to indicate a size of a deviation from a starting position of the private storage space of the processing unit;

[0025] Determine the base address of the private memory, the thread warp index, the processing unit index, the private memory stride, the index channel, and the data format bit width;

[0026] The address at which data is stored in the private memory is determined according to the base address of the private memory, the thread bundle index, the processing unit index, the private memory stride, the address offset information of the private storage space of the processing unit, the channel index, and the data format bit width; wherein the base address is used to indicate the starting position of the private memory, the thread bundle index is used to indicate the index number of the thread bundle, the processing unit index is used to indicate the index number of each processing unit in the thread bundle, the private memory stride is used to indicate the size of the private storage space accessed by the processing unit, the channel index is used to indicate the index value of the data channel, and the data format bit width is used to indicate data formats of different bit widths.

[0027] In a second aspect, the present application further provides a device for accessing a private memory, the device comprising:

[0028] A thread warp acquisition module, used for applying for obtaining at least one unused thread warp ID when the thread warp is started, wherein the thread warp ID is identification information used to distinguish different thread warps;

[0029] An instruction reading module, after a user starts a thread corresponding to the unused thread warp ID, reads a first instruction from an instruction cache;

[0030] an address determination module, configured to determine the address of the private memory when the first instruction is a read or write operation for the private memory;

[0031] The read / write module is used to read data from the address corresponding to the private memory, or write data into the address corresponding to the private memory.

[0032] In a third aspect, the present application further provides a private memory access system, the system comprising:

[0033] A shader thread constructor, used for sending a request for a warp ID to a warp constructor when the warp is started, where the warp ID is identification information for distinguishing different warps;

[0034] A warp builder, configured to allocate at least one unused warp ID to the shader thread builder according to the request for applying for the warp ID;

[0035] A thread scheduler, configured to obtain an unused thread bundle ID from the shader thread constructor, and after starting a thread corresponding to the unused thread bundle ID, read a first instruction from an instruction cache to an instruction execution unit;

[0036] The instruction execution unit is configured to bypass the first instruction to the memory access scheduler when the first instruction is a read or write operation for the private memory;

[0037] The memory access scheduler is used to determine the address of the private memory, and read data from the address corresponding to the private memory, or write data into the address corresponding to the private memory.

[0038] In a fourth aspect, the present application further provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0039] When a thread warp is started, applying for obtaining at least one unused thread warp ID, where the thread warp ID is identification information used to distinguish different thread warps;

[0040] After starting the thread corresponding to the unused thread warp ID, reading a first instruction from an instruction cache;

[0041] When the first instruction is a read or write operation for a private memory, determining an address of the private memory;

[0042] Data is read from the address corresponding to the private memory, or data is written into the address corresponding to the private memory.

[0043] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0044] When a thread warp is started, applying for obtaining at least one unused thread warp ID, where the thread warp ID is identification information used to distinguish different thread warps;

[0045] After starting the thread corresponding to the unused thread warp ID, reading a first instruction from an instruction cache;

[0046] When the first instruction is a read or write operation for a private memory, determining an address of the private memory;

[0047] Data is read from the address corresponding to the private memory, or data is written into the address corresponding to the private memory.

[0048] In a sixth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0049] When a thread warp is started, applying for obtaining at least one unused thread warp ID, where the thread warp ID is identification information used to distinguish different thread warps;

[0050] After starting the thread corresponding to the unused thread warp ID, reading a first instruction from an instruction cache;

[0051] When the first instruction is a read or write operation for a private memory, determining an address of the private memory;

[0052] Data is read from the address corresponding to the private memory, or data is written into the address corresponding to the private memory.

[0053] The above-mentioned private memory access method, device, computer equipment, computer-readable storage medium and computer program product, by applying for obtaining at least one unused thread bundle ID when the thread bundle is started, the thread bundle ID is identification information used to distinguish different thread bundles; so that the thread bundle can be used as the execution unit to perform subsequent private memory access, thereby improving access efficiency. After starting the thread corresponding to the unused thread bundle ID, the first instruction is read from the instruction cache; so that the thread resources can be fully utilized to read the instruction, so as to efficiently distribute the instruction to the instruction execution unit. When the first instruction is a read or write operation for the private memory, the address of the private memory is determined, and data is read from the address corresponding to the private memory, or data is written to the address corresponding to the private memory. Therefore, when the register file is insufficient during the execution of the thread bundle, the data can be temporarily stored in the private memory first, without interrupting the execution of the instruction pipeline, speeding up the execution speed of the instruction, and making full use of the storage space of the private memory for data storage. In addition, in an optional implementation scheme, the memory arrangement of the thread bundle in the private storage space is also designed, so that the number of reading and writing private memory can be effectively reduced, and the efficiency of data access can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A schematic diagram of a method for accessing a private memory in one embodiment;

[0056] Figure 2 A schematic diagram of a flow chart of a method for accessing a private memory in one embodiment;

[0057] Figure 3 is a flowchart of a method for accessing a private memory in another embodiment;

[0058] Figure 4 is one of the schematic diagrams of the layout of the private memory of a warp in one embodiment;

[0059] Figure 5 FIG2 is a second schematic diagram of the layout of the private memory of a warp in one embodiment;

[0060] Figure 6 A schematic diagram of a process for determining a private memory address in one embodiment;

[0061] Figure 7 A schematic diagram of a process for determining a private memory address of a SIMD64 warp in one embodiment;

[0062] Figure 8 A schematic diagram of a process for determining a private memory address of a SIMD32 warp in one embodiment;

[0063] Fig. 9 is a structural block diagram of a device for accessing a private memory in one embodiment;

[0064] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] Figure 1 FIG. 1 is a schematic diagram showing a method for accessing a private memory in an embodiment. Figure 1The computer architecture shown may include: a wave constructor, a shader thread constructor, a wave scheduler, an instruction cache, an instruction execution unit, a load store scheduler, and a private memory. Among them, the wave constructor is mainly used to: allocate wave resources, such as the wave ID and the register space used by the wave; release the wave resources. The shader thread constructor is used to process vertex data in the graphics pipeline or work group splitting in the computing pipeline. The thread scheduler is used to manage and schedule the control unit of each thread, including selecting threads, fetching instructions, decoding, and issuing instructions. The instruction cache is a high-speed cache used to store instruction information, process instruction fetch requests from the wave scheduler, and return instructions to the wave scheduler. The instruction execution unit is used to execute ALU (arithmetic logic unit) instructions and bypass PM instructions (an instruction for reading data from private memory and writing it to registers, and / or writing data in register files to private memory) to the memory scheduler. The memory scheduler is used to write register data to global and private memory; read global and private memory and write data to registers. The private memory is used to store data sent by the memory scheduler and process the read data request of the memory scheduler.

[0067] For example, Figure 1 As shown, first the shader thread constructor starts the thread bundle. When the thread bundle is started, it applies for the thread bundle ID from the thread bundle constructor. After receiving the request from the shader thread constructor, the thread bundle constructor checks whether the register file and other resources meet the needs of the thread bundle. If the resources are sufficient, it allocates resources to the thread bundle and finds an unused thread bundle ID to send to the shader thread constructor. If the resources are insufficient, it waits for the already started thread bundle to finish running until the resources are sufficient. It finds an unused thread bundle ID and returns it to the shader thread constructor. After obtaining the thread ID, the shader thread constructor notifies the thread bundle scheduler that a new thread bundle has been started. The thread bundle scheduler sends a read instruction request to the instruction cache and sends the instruction returned by the instruction cache to the instruction execution unit. The instruction execution unit is responsible for processing ALU instructions. When it receives a PM instruction, it bypasses it to the memory access scheduler. The memory access scheduler is responsible for calculating the address of the data in the private memory. If it is a read instruction, the address is sent to the private memory, and the private memory takes the data out of these addresses and returns it to the memory access scheduler; if it is a write instruction, both the address and data are sent to the private memory, and the private memory stores the data in these addresses.

[0068] In an exemplary embodiment, Figure 2 As shown, a method for accessing a private memory is provided, and the method is applied to Figure 1 The computer architecture in FIG. 1 is taken as an example to illustrate the method, which includes the following steps 201 to 204. Among them:

[0069] Step 201 : when a warp is started, apply to obtain at least one unused warp ID.

[0070] The thread warp ID is identification information used to distinguish different thread warps.

[0071] In this embodiment, Figure 1 In the computer architecture shown, a warp can be started by a shader thread builder, and when starting a warp, a request for a warp ID is sent to the warp builder. After receiving the warp ID request from the shader thread builder, the warp builder allocates at least one unused thread ID to the shader thread builder.

[0072] In one possible case, checking the register file and other resources may not meet the needs of the thread warp, that is, the resources are insufficient. At this time, it is necessary to wait for the started thread warp to finish running until the resources are sufficient, and then find an unused thread warp ID to return to the shader thread constructor.

[0073] Step 202 : After starting the thread corresponding to the unused warp ID, read the first instruction from the instruction cache.

[0074] In this embodiment, the first instruction refers to a command for a processing unit, which includes an access instruction for a private memory. Exemplarily, the private memory is accessed by designing two instructions, PM_LD and PM_ST. After calculating the private memory address of each processing unit, the private memory can be accessed using the process of accessing the global memory. Among them, PM_LD is responsible for reading data from the private memory and writing it to the register; PM_ST is responsible for writing data in the register file to the private memory.

[0075] Step 203: when the first instruction is a read or write operation for the private memory, determine the address of the private memory.

[0076] In this embodiment, when the first instruction is a read or write operation for the private memory, the storage address of the data in the private memory must first be determined, so that the data can be accurately read from the private memory or accurately stored in the corresponding address of the private memory.

[0077] Exemplarily, the first instruction can be parsed by a compiler to obtain predicate register indication information, target address register indication information, address offset information of a private storage space of a processing unit, and data format information for writing or reading a private memory, wherein the predicate register indication information is used to determine a processing unit in a thread warp that executes the first instruction, the target address register indication information is used to determine the number of channels for writing or reading a private memory, and the address offset information of the private storage space of the processing unit is used to indicate the size of the deviation from the starting position of the private storage space of the processing unit. Furthermore, the base address of the private memory, the thread bundle index, the processing unit index, the private memory stride, the index channel, and the data format bit width are determined; the address at which the data is stored in the private memory is determined according to the base address of the private memory, the thread bundle index, the processing unit index, the private memory stride, the address offset information of the private storage space of the processing unit, the channel index, and the data format bit width; wherein the base address is used to indicate the starting position of the private memory, the thread bundle index is used to indicate the index number of the thread bundle, the processing unit index is used to indicate the index number of each processing unit in the thread bundle, the private memory stride is used to indicate the size of the private storage space accessed by the processing unit, the channel index is used to indicate the index value of the data channel, and the data format bit width is used to indicate data formats of different bit widths.

[0078] Exemplarily, the following instruction formats are designed for the PM_LD and PM_ST instructions respectively:

[0079] (Pn)PM_LD Rd[.xyzw],PM[imm],FMT

[0080] (Pn)PM_ST Rd[.xyzw],PM[imm],FMT

[0081] Among them, (Pn)PM_LD can be understood as the instruction code, and the compiler will translate PM_LD into the corresponding opcode. Pn is the predicate register, SIMD32 thread bundle is 32 bits, and SIMD64 thread bundle is 64 bits. As mentioned earlier, the thread bundle is packaged by 32 or 64 processing units, so each bit in the predicate register can control whether a processing unit works. For example, the Pn value is 0x0000FFFF, indicating that only the first 16 processing units work for the current instruction. The definition of (Pn)PM_ST is similar to the above, and will not be repeated here.

[0082] Among them, Rd[.xyzw] represents the target address register, which is used to temporarily save the data read from the private memory. .xyzw indicates how many channels of data need to be read. Here, .x / .xy / .xyz / .xyzw can be used to indicate reading one, two, three, or four channels of data, respectively.

[0083] Among them, PM[Imm] represents the address offset of the private storage space of the processing unit.

[0084] Among them, FMT represents the data format of writing or reading private memory, which can be an unsigned 8-bit integer (uint8), an unsigned 16-bit integer (uint16), or an unsigned 32-bit integer (uint32).

[0085] For example, take the instruction: (P5) PM_LD R4.xyzw, PM[1], uint32 as an example, assuming that the P5 value is 0x0000FFFF, and only the first 16 processing units (lanes) are working. First, according to the base address of the private memory, the thread warp offset address, the processing unit private storage offset value 1 and the data format uint32, the address in the private memory where the data is stored is calculated, and the data is read from the address and written to the register files R4 to R7.

[0086] For example, in the private memory, each processing unit has its own private storage space, so it is necessary to calculate the address to be accessed for the processing unit. The private memory address is calculated by the base address (BaseAddr), thread bundle index (WaveId), processing unit index (LaneId), private memory stride (Stride), private storage offset (Offset), channel index (ChnId) and data format bit width (ChnByte).

[0087] In this embodiment, the base address is the starting position of the memory allocated by the driver for the private memory when the program starts; the thread bundle index refers to the index number allocated by the thread bundle builder to each thread bundle; the processing unit index is used to indicate the index number of each processing unit in the thread bundle; the private memory stride (Stride) represents the private storage space that the processing unit can access; the private storage offset is the third item in the PM instruction, which is used to characterize the size of the deviation from the starting position of the processing unit private space; the channel index is used to indicate that there can be a maximum of 4 channels in the PM instruction, and the index value is 0 to 3; the data format bit width is used to indicate the three data formats supported by the PM instruction: uint8, uint16, and uint32, with bit widths of 1 bit, 2 bits, and 4 bits, respectively.

[0088] Exemplarily, the calculation formula of the address of the private memory is as follows:

[0089] PmAddr=BaseAddr+((WaveId(simd64?64:32))+((LaneId≥32)?32

[0090] ∶0))*Stride+((((Offset+ChnId)*ChnByte)>>2)

[0091] <<7)+((LaneId&0x1f)<<2)+(((Offset+ChnId)*ChnByte)&0x3)

[0092] Among them, PmAddr indicates the address of the private memory, BaseAddr indicates the starting address of the private memory, ((WaveId*(simd64?64∶32))+((LaneId≥32)?32∶0))*Stride indicates the offset size of the thread bundle, WaveId indicates the thread bundle index, SIMD64 indicates whether it is a SIMD64 thread bundle, LaneId is the processing unit index, and Stride is the stride size. When the Stride is equal, the memory space occupied by the SIMD64 thread bundle is twice that of the SIMD32 thread bundle; in the thread bundle of SIMD64, when LandId is greater than or equal to 32, the offset is the thread bundle offset plus the space occupied by the first 32 processing units.

[0093] Among them, ((((Offset+ChnId)*ChnByte)>>2)<<7) represents the row in the thread warp private space.

[0094] Wherein, ((LaneId&0x1f)<<2) represents the position of the processing unit in a row in the private memory.

[0095] Among them, (((Offset+ChnId)*ChnByte)&0x3) indicates the position of the calculation in a certain component.

[0096] Step 204: read data from the address corresponding to the private memory, or write data into the address corresponding to the private memory.

[0097] In this embodiment, after the address of the private memory is obtained, data can be read from the address corresponding to the private memory, or data can be written into the address corresponding to the private memory, through the memory access scheduler.

[0098] The above-mentioned method for accessing the private memory applies for obtaining at least one unused thread bundle ID when the thread bundle is started, and the thread bundle ID is identification information used to distinguish different thread bundles; thus, the thread bundle can be used as the execution unit to perform subsequent access to the private memory, thereby improving access efficiency. After starting the thread corresponding to the unused thread bundle ID, the first instruction is read from the instruction cache; thus, the thread resources can be fully utilized to read the instructions, so as to efficiently distribute the instructions to the instruction execution unit. When the first instruction is a read or write operation for the private memory, the address of the private memory is determined, and data is read from the address corresponding to the private memory, or data is written to the address corresponding to the private memory. Thus, when the register file is insufficient during the execution of the thread bundle, the data can be temporarily stored in the private memory without interrupting the execution of the instruction pipeline, thereby speeding up the execution of the instruction and making full use of the storage space of the private memory for data storage.

[0099] In another exemplary embodiment, Figure 3 As shown, a method for accessing a private memory is provided, and the method is applied to Figure 1 The computer architecture in FIG. 1 is taken as an example to illustrate, and the method includes the following steps 301 to 306. Among them:

[0100] Step 301: When a warp is started, apply to obtain at least one unused warp ID.

[0101] The thread warp ID is identification information used to distinguish different thread warps.

[0102] For the specific implementation process and technical effect of step 301 in this embodiment, please refer to Figure 2 The description of step 201 in the illustrated method embodiment will not be repeated here.

[0103] Step 302: Apply for a total private storage space from the global memory as a private memory through the driver.

[0104] In this embodiment, in order to improve the computing performance, the GPU usually uses a thread bundle (Wave) as the minimum execution unit, wherein the thread bundle is composed of multiple processing units (Lane) and performs reading, writing and calculation in a single instruction multiple data stream (SIMD) manner. Exemplarily, 32 or 64 processing units are usually packaged into a thread bundle, called a SIMD32 thread bundle or a SIMD64 thread bundle, so that each time an instruction is executed, the thread bundle can process 32 or 64 calculations or memory access operations at a time. In addition to a certain number of register files, each processing unit has a certain number of register files. When a larger array is used in the kernel function or the compiler cannot statically determine the array size, the driver will also apply for a private storage space for each processing unit. The driver applies for a total private storage space from the global memory according to the maximum number of thread bundles that the chip can run.

[0105] Exemplarily, before executing step 302 , memory arrangements of private storage spaces of the first-type warps and the second-type warps are determined respectively.

[0106] In an optional implementation, the size of the private storage space of each processing unit is recorded as a stride; wherein the total number of strides is a multiple of 4, and the unit of the stride is byte; the size of the private storage space occupied by the first type of thread warp is set to 32 times the stride, and the size of the private storage space occupied by the second type of thread warp is set to 64 times the stride, and the stride is split into components of the same size, and the size of each component is 4 bytes; wherein the components of the processing unit are not continuous in the memory.

[0107] Exemplarily, the first type of thread warp may be a SIMD32 thread warp, and the second type of thread warp may be a SIMD64 thread warp. When the thread warp ends, the private storage space occupied by the thread warp will be allocated to a new thread warp. The size of the private memory space of each processing unit is called stride, in bytes. Exemplarily, the stride can be set to a multiple of 4, so the size of the private storage space occupied by each SIMD32 thread warp is 32*Stride bytes, and the size of the private storage space occupied by the SIMD64 thread warp is 64*Stride bytes. The stride can be split into groups of 4 bytes, which can be split into Stride / 4 parts, each of which is called a component. For SIMD32 thread warps, the same components of 32 processing units are arranged together in memory, and then the next component of the 32 processing units is arranged. If the memory is regarded as a two-dimensional arrangement, each row size is 128 bytes, and Stride / 4 is the number of rows when the private storage space of each SIMD32 thread warp is arranged in two dimensions.

[0108] For example, Figure 4As shown in FIG. 1 , the private storage space of a SIMD64 thread bundle is divided into two parts: the first part is the storage space of the first 32 processing units, and the second part is the storage space of the second 32 processing units. Figure 4 It can be seen from the figure that when the stride is the same, the private storage space size of each SIMD64 thread warp is twice that of the SIMD32 thread warp. i C j Represents the jth component of the i-th Lane of the thread warp, i and j are 0-based. i represents the number of columns, j represents the number of rows, P i C j It represents the j+1th component of the i+1th processing unit. In the SIMD32 thread warp, it is located at the i+1th column and j+1th row of the thread warp. In the SIMD64 thread warp, if i is less than 32, it is located at the i+1th column and j+1th row of the thread warp. If i is greater than or equal to 32, it is located at the i+32+1th column and j+32+1th row of the thread warp.

[0109] It should be noted that when the above memory arrangement is adopted, the components of the processing unit are not continuous in the memory, but are separated by 128 bytes. However, the same component of the 32 processing units is continuous in the memory, that is, Figure 4 Each row in .

[0110] For example, in Figure 5 In this case, the same component is stored in memory continuously with the stride of the same processing unit. For example, the bandwidth of the private memory is 64 bytes, that is, each time the private memory is read or written, a maximum of 64 bytes of data is read or written, the stride is 64 bytes, the component size is 4 bytes, and the data size is 4 bytes. When reading a component data, the data in the private memory is stored in the stride of 4 bytes. Figure 4 When the data is stored, it is continuous in the memory. At this time, the private memory can return the component data of 16 processing units at a time, and the entire data of the SIMD32 thread bundle can be returned twice. Figure 5 When the private memory is stored in the same way, the component data of different processing units are not continuous in the memory. When the private memory returns data, it returns data in a continuous address space each time. Because the bandwidth and stride are both 64 bytes, only one of the 64 bytes of data returned by the private memory each time is the required data, and the rest are not needed. In this way, it takes 32 times to return all the data. Figure 4 The data placement designed in is more efficient when reading and writing private memory.

[0111] Step 303: when the array used in the kernel function is larger than a preset value, or the compiler cannot statically determine the size of the array, a private storage space is applied for each processing unit of the thread warp in the private memory through the driver.

[0112] Exemplarily, after executing step 303, if the thread warp ends, the private storage space occupied by each processing unit of the thread warp is released; and the released private storage space is allocated to the newly started thread warp through the driver.

[0113] In this embodiment, the private storage space can be used cyclically, that is, when a thread warp ends, the private storage space released by the thread warp is allocated to other new thread warps.

[0114] It should be noted that, in this embodiment, step 302 and step 303 may also be pre-executed before step 301 .

[0115] Step 304 : After starting the thread corresponding to the unused warp ID, read the first instruction from the instruction cache.

[0116] Step 305: when the first instruction is a read or write operation for the private memory, determine the address of the private memory.

[0117] Step 306: read data from the address corresponding to the private memory, or write data to the address corresponding to the private memory.

[0118] For the specific implementation process and technical effects of steps 304 to 306 in this embodiment, please refer to Figure 2 The descriptions of steps 202 to 204 in the illustrated method embodiment will not be repeated here.

[0119] For example, Figure 6 As shown, in Figure 6 The process of determining the private memory address is shown in Figure 1. Arrow 1 points to the starting position of the private memory, arrow 2 points to the offset of thread bundle n in the private memory, arrow 3 points to the offset row in thread bundle n, arrow 4 points to the offset position of the processing unit in a row, and arrow 5 points to the offset in the component. When the data format is uint32, arrow 5 can only point to the beginning of the component; when it is uint16, it can point to 0 or 2; when it is uint8, it can point to any position in the 4-byte component.

[0120] In an exemplary embodiment, Figure 7 As shown, in Figure 7The process of determining the private memory address of a SIMD64 warp is given in . Assume that the SIMD64 warp BaseAddr value is 0x20000, WaveId is 10, LaneId is 33, Stride size is 32 bytes, Offset is 2, ChnId is 1, and ChnByte is 2 bytes. At this time, the PmAddr value is: 0x20000+((10*64)+32)*32+((((2+1)*2)>>2)<<7)+((33&0x1f)<<2)+(((2+1)*2)&0x3)=0x20000+0x5400+0x80+0x4+0x2=0x25486.

[0121] In this embodiment, Figure 7 The position pointed by arrow 1 in the figure is the private memory BaseAddr; the position pointed by arrow 2 is the starting position of Wave10 in the private memory. Since the space occupied by each SIMD64 thread bundle is 64*Stride, the starting address of Wave10 is 0x20000+(10*64*32). Since LanId is 33, which is greater than or equal to 32, the space occupied by the first 32 lanes needs to be added, that is, Figure 7 The marker range of Wave10 is 32*32 bytes. The position pointed by arrow 3 is calculated based on the third part of the formula, (((2+1)*2)>>2)<<7 is equal to 128 bytes. Since each row has 128 bytes, it is offset by one row. The position pointed by arrow 4 is calculated based on the fourth part of the formula, (33&0x1f)<<2 is equal to 4, which is 4 bytes offset from the beginning of the row, i.e. P in the figure. 33 C 1 ; The position pointed by arrow 5 is calculated by the fifth part of the formula, ((2+1)*2)&0x3 is equal to 2, and at this time, two 2-byte offsets are made in the component.

[0122] In another exemplary embodiment, Figure 8 As shown, in Figure 8 The process of determining the private memory address of a SIMD32 warp is given in . Assume that the SIMD32 warp BaseAddr value is 0x20000, WaveId is 10, LaneId is 17, Stride size is 32 bytes, Offset is 2, ChnId is 1, and ChnByte is 2 bytes. At this time, the PmAddr value is: 0x20000+((10*32)+0)*32+((((2+1)*2)>>2)<<7)+((17&0x1f)<<2)+(((2+1)*2)&0x3)=0x20000+0x2800+0x80+0x44+0x2=0x228c6.

[0123] In this embodiment, Figure 8 The position pointed by arrow 1 in the figure is the private memory BaseAddr; the position pointed by arrow 2 is the starting position of Wave10 in the private memory, so the starting address of Wave10 is 0x20000+((10*32)+0)*32; the position pointed by arrow 3 is calculated according to the third part of the formula, (((2+1)*2)>>2)<<7 is equal to 128 bytes, because each row of memory space is 128 bytes, so it is offset by one row. The position pointed by arrow 4 is calculated according to the fourth part of the formula, (17&0x1f)<<2 is equal to 68, which is 68 bytes offset from the beginning of the row, that is, P in the figure. 17 C 1 ; The position pointed by arrow 5 is calculated by the fifth part of the formula, ((2+1)*2)&0x3 is equal to 2, and at this time, two 2-byte offsets are made in the component.

[0124] In this embodiment, by designing the memory arrangement of the thread warp in the private storage space, the number of times of reading and writing the private memory can be effectively reduced, thereby improving the efficiency of data access.

[0125] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0126] Based on the same inventive concept, the embodiment of the present application also provides a private memory access device for implementing the private memory access method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more private memory access devices provided below can refer to the limitations of the private memory access method above, and will not be repeated here.

[0127] In an exemplary embodiment, Fig. 9 As shown, a device for accessing a private memory is provided, the device comprising: a warp acquisition module 901, an instruction reading module 902, an address determination module 903 and a read / write module 904, wherein:

[0128] A warp acquisition module 901 is used to apply for obtaining at least one unused warp ID when a warp is started, where the warp ID is identification information used to distinguish different warps;

[0129] The instruction reading module 902 reads the first instruction from the instruction cache after the user starts the thread corresponding to the unused thread warp ID;

[0130] An address determination module 903, configured to determine the address of the private memory when the first instruction is a read or write operation for the private memory;

[0131] The read / write module 904 is used to read data from the address corresponding to the private memory, or write data to the address corresponding to the private memory.

[0132] Exemplarily, the above device further includes:

[0133] The private storage space application module 905 is used to apply for a total private storage space from the global memory as a private storage through the driver program;

[0134] The private storage space allocation module 906 is used to apply for a private storage space for each processing unit of the thread warp in the private memory through the driver program when the array used in the kernel function is larger than a preset value or the compiler cannot statically determine the size of the array.

[0135] Exemplarily, the above device further includes:

[0136] The memory arrangement mode setting module 907 is used to determine the memory arrangement modes of the private storage spaces of the first type of thread warps and the second type of thread warps respectively.

[0137] Exemplarily, the memory arrangement mode setting module 907 is specifically used to:

[0138] The size of the private storage space of each processing unit is recorded as stride; wherein the total number of strides is a multiple of 4, and the unit of stride is byte;

[0139] Set the size of the private storage space occupied by the first type of thread warp to 32 times the stride, and set the size of the private storage space occupied by the second type of thread warp to 64 times the stride.

[0140] The stride is split into components of equal size, each component being 4 bytes in size; wherein the components of the processing unit are not contiguous in memory.

[0141] Exemplarily, the private storage space allocation module 906 is further configured to release the private storage space occupied by each processing unit of the thread warp when the thread warp ends running; and allocate the released private storage space to the newly started thread warp through the driver.

[0142] Exemplarily, the address determination module 903 is specifically used to: parse the first instruction through a compiler to obtain predicate register indication information, target address register indication information, address offset information of a private storage space of a processing unit, and data format information for writing or reading a private memory, wherein the predicate register indication information is used to determine a processing unit in a thread warp that executes the first instruction, the target address register indication information is used to determine the number of channels for writing or reading a private memory, and the address offset information of the private storage space of the processing unit is used to indicate the size of the deviation from the starting position of the private storage space of the processing unit;

[0143] Determine the base address of the private memory, the thread warp index, the processing unit index, the private memory stride, the index channel, and the data format bit width;

[0144] The address at which data is stored in the private memory is determined according to the base address of the private memory, the thread bundle index, the processing unit index, the private memory stride, the address offset information of the private storage space of the processing unit, the channel index, and the data format bit width; wherein the base address is used to indicate the starting position of the private memory, the thread bundle index is used to indicate the index number of the thread bundle, the processing unit index is used to indicate the index number of each processing unit in the thread bundle, the private memory stride is used to indicate the size of the private storage space accessed by the processing unit, the channel index is used to indicate the index value of the data channel, and the data format bit width is used to indicate data formats of different bit widths.

[0145] Each module in the above-mentioned private memory access device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0146] In an exemplary embodiment, a private memory access system is provided, comprising: a shader thread builder, a thread bundle builder, a thread scheduler, an instruction execution unit, and a memory access scheduler, wherein:

[0147] A shader thread constructor, used for sending a request for a warp ID to a warp constructor when the warp is started, where the warp ID is identification information for distinguishing different warps;

[0148] A warp builder, configured to allocate at least one unused warp ID to the shader thread builder according to the request for applying for the warp ID;

[0149] A thread scheduler, configured to obtain an unused thread bundle ID from the shader thread constructor, and after starting a thread corresponding to the unused thread bundle ID, read a first instruction from an instruction cache to an instruction execution unit;

[0150] The instruction execution unit is configured to bypass the first instruction to the memory access scheduler when the first instruction is a read or write operation for the private memory;

[0151] The memory access scheduler is used to determine the address of the private memory, and read data from the address corresponding to the private memory, or write data into the address corresponding to the private memory.

[0152] The system in this embodiment can implement the steps in the above-mentioned various embodiments, and its specific implementation process and technical effects are not described in detail here.

[0153] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for accessing a private memory is implemented.

[0154] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0155] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0156] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0157] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0158] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0159] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0160] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0161] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for accessing a private memory, characterized in that: The method comprises: When a thread warp is started, applying for obtaining at least one unused thread warp ID, where the thread warp ID is identification information used to distinguish different thread warps; After starting the thread corresponding to the unused thread warp ID, reading a first instruction from an instruction cache; When the first instruction is a read or write operation for a private memory, determining an address of the private memory; Data is read from the address corresponding to the private memory, or data is written into the address corresponding to the private memory.

2. The method according to claim 1, characterized in that Before reading the first instruction from the instruction cache module, the method further includes: Apply for a total private storage space from the global memory as private memory through the driver; When the array used in the kernel function is larger than a preset value, or the compiler cannot statically determine the size of the array, a private storage space is applied for each processing unit of the thread warp in the private memory through the driver.

3. The method according to claim 2, characterized in that Before applying for a private storage space for each processing unit of the thread warp in the private memory through the driver, the method further includes: Memory arrangements of private storage spaces of the first-type warps and the second-type warps are determined respectively.

4. The method according to claim 3, characterized in that The determining of the memory arrangement of the private storage spaces of the first type of thread warps and the second type of thread warps respectively includes: The size of the private storage space of each processing unit is recorded as stride; wherein the total number of strides is a multiple of 4, and the unit of stride is byte; Set the size of the private storage space occupied by the first type of thread warp to 32 times the stride, and set the size of the private storage space occupied by the second type of thread warp to 64 times the stride. The stride is split into components of equal size, each component being 4 bytes in size; wherein the components of the processing unit are not contiguous in memory.

5. The method according to claim 2, characterized in that: After applying for a private storage space for each processing unit of the thread warp in the private memory through the driver, the method further includes: If the thread warp is finished running, the private storage space occupied by each processing unit of the thread warp is released; The freed private storage space is allocated to the newly started warp by the driver.

6. The method according to any one of claims 1 to 5, characterized in that: The determining the address of the private memory comprises: The first instruction is parsed by a compiler to obtain predicate register indication information, target address register indication information, address offset information of a private storage space of a processing unit, and data format information for writing or reading a private memory, wherein the predicate register indication information is used to determine a processing unit in a thread warp that executes the first instruction, the target address register indication information is used to determine the number of channels for writing or reading a private memory, and the address offset information of the private storage space of the processing unit is used to indicate a size of a deviation from a starting position of the private storage space of the processing unit; Determine the base address of the private memory, the thread warp index, the processing unit index, the private memory stride, the index channel, and the data format bit width; The address at which data is stored in the private memory is determined according to the base address of the private memory, the thread bundle index, the processing unit index, the private memory stride, the address offset information of the private storage space of the processing unit, the channel index, and the data format bit width; wherein the base address is used to indicate the starting position of the private memory, the thread bundle index is used to indicate the index number of the thread bundle, the processing unit index is used to indicate the index number of each processing unit in the thread bundle, the private memory stride is used to indicate the size of the private storage space accessed by the processing unit, the channel index is used to indicate the index value of the data channel, and the data format bit width is used to indicate data formats of different bit widths.

7. A device for accessing a private memory, characterized in that: The device comprises: A thread warp acquisition module, used for applying for obtaining at least one unused thread warp ID when the thread warp is started, wherein the thread warp ID is identification information used to distinguish different thread warps; An instruction reading module, configured to read a first instruction from an instruction cache after starting a thread corresponding to the unused thread warp ID; an address determination module, configured to determine the address of the private memory when the first instruction is a read or write operation for the private memory; The read / write module is used to read data from the address corresponding to the private memory, or write data into the address corresponding to the private memory.

8. A private memory access system, characterized in that: The system comprises: A shader thread constructor, used for sending a request for a warp ID to a warp constructor when the warp is started, where the warp ID is identification information for distinguishing different warps; A warp builder, configured to allocate at least one unused warp ID to the shader thread builder according to the request for applying for the warp ID; A thread scheduler, configured to obtain an unused thread bundle ID from the shader thread constructor, and after starting a thread corresponding to the unused thread bundle ID, read a first instruction from an instruction cache to an instruction execution unit; The instruction execution unit is configured to bypass the first instruction to the memory access scheduler when the first instruction is a read or write operation for the private memory; The memory access scheduler is used to determine the address of the private memory, and read data from the address corresponding to the private memory, or write data into the address corresponding to the private memory.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images