Data processing method, device and equipment for accessing shared cache across working groups
By implementing the method of accessing shared cache across workgroups in GPU, threads from different workgroups are allowed to access shared caches with each other, solving the problem of high data transmission cost in AI computing scenarios and improving data utilization.
Patent Information
- Application Number
- CN202410918331.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-09
- Publication Date
- 2024-11-08
AI Technical Summary
In AI computing scenarios, when different working groups need to access the same data, each working group in the prior art needs to read and write from external memory, resulting in high data transmission costs and low data utilization.
By introducing a method of accessing shared cache across workgroups in the GPU, threads of different workgroups are allowed to access shared caches with each other. By obtaining target access information, checking the status of associated workgroups, obtaining shared cache address information and performing data access actions, the same Data sharing between workgroups in the workspace.
Improve data utilization, reduce read and write requests of external memory, and reduce data transmission costs.
Smart Images

Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, apparatus, computer equipment, storage medium and computer program product for accessing a shared cache across workgroups. Background Art
[0002] GPU (Graphics Processing Unit) is used for graphics and image processing. When performing general computing tasks, each work group needs to access a large amount of data. In order to improve data utilization and reduce read and write requests from external memory, some SRAM components are currently added to the GPU for each work group to cache data, which is called shared cache (SMem, Shared Memory).
[0003] The shared cache can be used by all threads of the corresponding workgroup, but cannot be accessed by threads of other workgroups. However, in scenarios such as AI computing, different workgroups may need to input the same data, and the shared caches of multiple workgroups store common data. This requires each workgroup to read and write from external memory when obtaining common data stored by other workgroups, increasing data transmission costs and making data utilization low. Summary of the invention
[0004] Based on this, it is necessary to provide a data processing method, device, computer equipment, computer-readable storage medium and computer program product for accessing shared cache across workgroups, which can allow threads belonging to different workgroups to access each other's shared caches, thereby improving data utilization.
[0005] In a first aspect, the present application provides a data processing method for accessing a shared cache across workgroups, comprising:
[0006] Acquire target access information, and determine a target workgroup according to the target access information, wherein the target workgroup and the current workgroup are in the same work grid, and the work grid includes a plurality of the workgroups;
[0007] Checking the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information;
[0008] When the working states of all the associated working groups are in the preparation state, obtaining the shared cache address information of the target working group;
[0009] According to the shared cache address information, an action is performed by the current working group to access the shared cache data of the target working group.
[0010] In one embodiment, the target access information includes at least a target workgroup index and a target data address; and obtaining the shared cache address information of the target workgroup includes:
[0011] Acquire configuration mapping information corresponding to the work grid where the current work group is located, wherein the configuration mapping information is used to characterize the mapping configuration relationship between various work groups;
[0012] According to the target workgroup index, searching the configuration mapping information for a shared cache start address of the target workgroup;
[0013] The shared cache address information of the target working group is calculated according to the shared cache start address and the target data address.
[0014] In one embodiment, before obtaining the target access information, the method further includes:
[0015] Get the data to be processed;
[0016] Loading the data to be processed into the shared caches of the plurality of working groups in the current working grid respectively;
[0017] When the working group is loaded, setting the working state of the working group to a standby state;
[0018] Stop the data modification action on the shared cache of the work group.
[0019] In one embodiment, after accessing the shared cache data of the target work group according to the shared cache address information, the method further includes:
[0020] Setting the working status of the current working group and the associated working group to the access-end status;
[0021] When the working status of all the associated workgroups in the current work grid is the end-of-access status, the working status of the current workgroup and the associated workgroup is set to a data change status, wherein the data change status is a status in which the workgroup can perform a shared cache data change operation.
[0022] In one embodiment, the work group includes a main thread and at least one sub-thread; when the work group is loaded, setting the work state of the work group to a ready state includes:
[0023] In response to a preparation command issued by the main thread of the work group, setting the working state of the work group to a preparation state;
[0024] Accordingly, the action of stopping the data change action on the shared cache of the work group includes:
[0025] Stop the data modification action on the shared cache in the main thread;
[0026] The data modification action on the shared cache in each of the sub-threads is stopped through a thread synchronization command.
[0027] In one embodiment, the configuration mapping information includes execution location information and data address information of the work group; the method further includes:
[0028] Acquire a data processing task, and divide the global space into a plurality of working grids according to the data processing task;
[0029] According to the three-dimensional structural data of each working cell, the working cell is divided into a plurality of working groups;
[0030] The execution position information and the data address information corresponding to each of the working groups are determined according to the three-dimensional structure data of the working group.
[0031] In a second aspect, the present application further provides a data processing device for accessing a shared cache across workgroups, comprising:
[0032] A data acquisition module, used for acquiring target access information, and determining a target workgroup according to the target access information, wherein the target workgroup and the current workgroup are in the same work grid, and the work grid includes a plurality of the workgroups;
[0033] A status checking module, used for checking the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information;
[0034] An address acquisition module, used for acquiring the shared cache address information of the target work group when the working states of all the associated work groups are in the preparation state;
[0035] A data access module is used to execute the action of the current working group accessing the shared cache data of the target working group according to the shared cache address information.
[0036] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:
[0037] Acquire target access information, and determine a target workgroup according to the target access information, wherein the target workgroup and the current workgroup are in the same work grid, and the work grid includes a plurality of the workgroups;
[0038] Checking the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information;
[0039] When the working states of all the associated working groups are in the preparation state, obtaining the shared cache address information of the target working group;
[0040] According to the shared cache address information, an action is performed by the current working group to access the shared cache data of the target working group.
[0041] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:
[0042] Acquire target access information, and determine a target workgroup according to the target access information, wherein the target workgroup and the current workgroup are in the same work grid, and the work grid includes a plurality of the workgroups;
[0043] Checking the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information;
[0044] When the working states of all the associated working groups are in the preparation state, obtaining the shared cache address information of the target working group;
[0045] According to the shared cache address information, an action is performed by the current working group to access the shared cache data of the target working group.
[0046] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:
[0047] Acquire target access information, and determine a target workgroup according to the target access information, wherein the target workgroup and the current workgroup are in the same work grid, and the work grid includes a plurality of the workgroups;
[0048] Checking the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information;
[0049] When the working states of all the associated working groups are in the preparation state, obtaining the shared cache address information of the target working group;
[0050] According to the shared cache address information, an action is performed by the current working group to access the shared cache data of the target working group.
[0051] The above-mentioned data processing method, device, computer equipment, storage medium and computer program product for accessing a shared cache across workgroups divide multiple workgroups that need to access shared memory with each other into a work cell, obtain target access information, and determine the target workgroup in the same work cell as the current workgroup based on the target access information, and merge adjacent workgroups into the form of work cells. Different workgroups in the same work cell can access their respective shared caches with each other, and then check the working status of the associated workgroup associated with the target access information in the work cell where the current workgroup is located based on the target access information. When the working status of all associated workgroups is in the preparation state, the shared cache address information of the target workgroup is obtained, which can ensure that the loading of data and other tasks have been completed before the shared caches between different workgroups are accessed to each other, thereby ensuring the integrity of the data. Finally, based on the shared cache address information, the action of the current workgroup accessing the shared cache data of the target workgroup is executed, thereby allowing threads belonging to different workgroups to access their respective shared caches with each other, thereby improving data utilization. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0053] Figure 1 A schematic diagram of a calculation example of a data processing method for accessing a shared cache across workgroups in an embodiment;
[0054] Figure 2 An application environment diagram of a data processing method for accessing a shared cache across workgroups in one embodiment;
[0055] Figure 3 A schematic flow chart of a data processing method for accessing a shared cache across workgroups in one embodiment;
[0056] Figure 4 This is a flow chart of step S202 of a data processing method for accessing a shared cache across workgroups in one embodiment;
[0057] Figure 5 A schematic diagram of a working grid mapping table of a GMU module in one embodiment;
[0058] Figure 6 It is a flowchart of a data processing method for accessing a shared cache across workgroups in another embodiment;
[0059] Figure 7 A schematic diagram of computing space partitioning of a data processing method for accessing a shared cache across workgroups in one embodiment;
[0060] Figure 8 A schematic diagram of a synchronization process for shared cache data in one embodiment;
[0061] Fig. 9 A structural block diagram of a data processing device for accessing a shared cache across workgroups in one embodiment;
[0062] Fig.10 is an internal structure diagram of a computer device in one embodiment;
[0063] Fig.11 FIG. 4 is a diagram showing the internal structure of a computer device in another embodiment. DETAILED DESCRIPTION
[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0065] This application takes GPU to perform general computing tasks as an example, such as Figure 1 As shown, in the related art, the shared cache used by each work group can only be accessed by all threads in the work group, but cannot be used by threads in other work groups. However, multiple work groups may have common data, which requires each work group to read and write from the external memory, increasing data transmission.
[0066] In a deep neural network, the multiplication result of two matrices (A and B) is calculated: C=A*B. Since A and B are both relatively large, they can be divided into 4 parts: A is divided into A0 and B0, and B is divided into B0 and B1. Then 4 working groups are started: grp0~grp3, and the matrix sizes calculated are:
[0067] grp0: C0 = A0*B0; grp1: C1 = A0*B1; grp2: C2 = A1*B0; grp3: C3 = A1*B1;
[0068] Because before each matrix calculation, the working group needs to read the matrix data from the external memory into its own shared cache, it can be found that A0, A1, B0, and B1 are loaded twice by the four working groups. In other words, matrices A and B are read in twice, which puts a lot of pressure on the external read and write bandwidth.
[0069] The data processing method for accessing a shared cache across workgroups provided in the embodiment of the present application can be applied to Figure 2 In the application environment shown in the figure. Among them, GPCP (Grid Processing Control Plane) is used to construct a working grid (Grid) and split (Grid) into multiple working groups. After the general computing task is sent to the GPU through the driver, the working group can be split in GPCP and then dynamically scheduled to the processing element (PE) of each slice for execution. The GPCP module can receive configuration information from the driver and store it in the register (REG). The GRID_SPLIT module is used to split the working grid, and the GRP_SPLIT module is used to split the working group and synchronize the results of the above split to the GMU module. Multiple working groups are connected to the GMU module for communication and can send information such as data loading completion to the GMU module. The GMU module can be used to record the information of multiple working groups in the working grid, and to synchronize the information of the working group and the shared address conversion. GMT represents the configuration mapping information recorded in the GMU module, and SMAT represents the shared cache address converter used to recalculate the shared cache address in other working groups.
[0070] In an exemplary embodiment, Figure 3 As shown, a data processing method for accessing a shared cache across workgroups is provided, and the method is applied to Figure 1 The system consisting of multiple working groups and GMU modules is described as an example, including the following steps S202 to S208. Among them:
[0071] Step S202: acquiring target access information, and determining a target work group according to the target access information.
[0072] Among them, the target work group and the current work group are in the same work grid, and the work grid includes multiple work groups. General computing models such as OpenCL and CS of GPU usually adopt a three-level hierarchical relationship: global space (Global Space) work group (Work-Group) work item (Work-Item). In this embodiment, a layer of logical relationship is added above the work group (work-group): work grid (work-grid), and the above hierarchical relationship becomes GlobalSpace Work-Grid Work-Group Work-Item, so that multiple work groups in the same work grid can access each other's shared cache, thereby improving data utilization.
[0073] For example, the number of workgroups in a work grid GridG (GridGx, GridGy, GridGz): represents the number of workgroups in the three-dimensional space contained in the work grid; the work grid Grid is not a standard interface setting parameter, and the existing interface can be extended to allow users to specify the number of workgroups in each work grid. These workgroups can be scheduled by the hardware GPU to the processing units in different slices for execution separately, but different workgroups in the same work grid are allowed to access each other's shared cache.
[0074] For example, in order to simplify the hardware implementation, there may be a constraint on the range of working grid values: the product size of the three dimensions of the working grid must be greater than or equal to 1 to prevent it from degenerating into a normal mode, and the product of the three must be less than or equal to a preset maximum number of working groups, which is a fixed value and can be set to 32 or 64, etc., depending on different GPU products. For example, if the maximum number of working groups of the current GPU product is set to 32, then GridG (GridGx, GridGy, GridGz) selects (1,8,2), (8,4,1), and (2,4,3) are all legal, but (10,4,1) and (8,4,2) are both over 32 and are illegal.
[0075] The target access information may include a current work group index, a target work group index, a target data address, an access type, etc. The access type may include a read type, a write type, an atomic operation type, etc.
[0076] Exemplarily, the GMU module can determine the target workgroup according to the target workgroup index in the target access information. The shared cache read (SMemRead), write (SMemWrite) and atomic operation (SMemAtomic) commands are defined as follows:
[0077] (1) data = SMemRead(grp_idx, sm_addr): Reads the data at address sm_addr from the shared cache of the workgroup numbered grp_idx and returns it.
[0078] (2) SMemWrite(grp_idx, sm_addr, data): Writes data data to the shared cache of the workgroup numbered grp_idx, at address sm_addr.
[0079] (3) ret = SMemAtomic(grp_idx, sm_addr, data, opcode): Perform atomic operations on the shared cache of the workgroup numbered grp_idx, with the address sm_addr and the operator opcode, including operations such as Add / Min / Max / And / Or / Xor / UMin / UMaxExchange / Cmpare_cxch, and return the result ret.
[0080] Exemplarily, before step S202, the steps may also include obtaining the data to be processed; loading the data to be processed into the shared caches of multiple work groups in the current work cell respectively; when the work group is loaded, setting the working state of the work group to the ready state; and stopping the data change action on the shared cache of the work group.
[0081] Exemplarily, the work group may include only one thread. When the work group includes only one thread, the thread is the main thread of the work group. When the work group is loaded, setting the working state of the work group to the ready state may include: responding to a ready command issued by the main thread of the work group, setting the working state of the work group to the ready state; and the step of stopping the data change action on the shared cache of the work group may include: stopping the data change action on the shared cache in the main thread.
[0082] Furthermore, the work group may also include multiple threads. When the work group includes multiple threads, it includes a main thread and at least one child thread. When the work group is loaded, setting the working state of the work group to the ready state may include: responding to a ready command issued by the main thread of the work group, setting the working state of the work group to the ready state; accordingly, the step of stopping the data change action on the shared cache of the work group may include: stopping the data change action on the shared cache in the main thread; stopping the data change action on the shared cache in each child thread through a thread synchronization command.
[0083] Step S204: Check the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information.
[0084] The associated workgroups associated with the target access information may be all workgroups in the current work grid.
[0085] For example, since each shared cache is used by each work group separately, it needs to be synchronized before it can be accessed by other work groups so that other work groups can obtain correct data. Therefore, before accessing the shared cache, each work group in the current work grid can synchronize a command (SMemReady) to the GMU module indicating that its data is ready. After receiving the signal, the GMU module will determine that the work group that sent the signal is in a ready state, that is, the working state of the work group is in a ready state, for example, the SM_Ready of the work group is set to 1.
[0086] Furthermore, since a work group contains multiple executing threads, it is not necessary for each thread to send the command. Only the main thread needs to send the command once, and other threads can ensure consistent behavior through the existing thread synchronization commands within the work group.
[0087] For example, when the current workgroup thread needs to access the shared cache of other workgroups, the current workgroup can send a check readiness command (SMemCheckReady) to the GMU module, and the GMU module can check the work grid where the workgroup is located to determine whether all workgroups are in a ready state, for example, whether the SM_Ready of each workgroup in the work grid is 1.
[0088] Step S206: when the working states of all associated working groups are in the preparation state, obtain the shared cache address information of the target working group.
[0089] Exemplarily, when the working status of all associated work groups is in the ready state, the GMU module allows the current work group thread to access the shared cache of other work groups. The GMU module can remap the addresses of the read and write commands of the shared cache to find the address and processing unit corresponding to the shared cache of the target work group so that the current work group can read the shared cache data of the target work group.
[0090] Step S208: According to the shared cache address information, the current work group accesses the shared cache data of the target work group.
[0091] Exemplarily, if the SM_Ready of all workgroups in the work grid is 1, the GMU module can notify the workgroup thread that it can access the shared cache of other workgroups, so that according to the shared cache address information, the action of the current workgroup accessing the shared cache data of the target workgroup is executed. Otherwise, the thread of the workgroup that needs to access the shared cache of other workgroups will be suspended until the SM_Ready of all workgroups in the work grid to which the workgroup belongs is 1.
[0092] Exemplarily, before executing the action of the current work group accessing the shared cache data of the target work group, the working status of the current work group and the associated work groups can also be set to the end access status; when the working status of all associated work groups in the current work grid is the end access status, the working status of the current work group and the associated work groups is set to the data change status, wherein the data change status is the status in which the work group can perform shared cache data change operations.
[0093] For example, after the current work group has accessed the shared cache, a shared cache release command (SMemRelease) can be sent to the GMU module, and the GMU module can set the SM_Release of the corresponding work group to 1. Since a work group contains multiple executing threads, it is not necessary for each thread to send the command, and only the main thread needs to send the command once. Other threads can ensure consistent behavior through the existing thread synchronization commands within the work group.
[0094] Furthermore, when the SM_Release of all working groups in the work grid is 1, it means that all working groups have finished accessing the shared cache. The GMU module can set the SM_Ready and SM_Release of all working groups to 0, indicating that the mutual access to the shared cache is completed, and at the same time send a signal to notify all working group threads that they can write or modify the shared cache data of their respective working groups.
[0095] In the above-mentioned data processing method for accessing a shared cache across workgroups, multiple workgroups that need to access shared memory with each other are divided into a work cell. By obtaining target access information and determining the target workgroup in the same work cell as the current workgroup based on the target access information, adjacent workgroups are merged into the form of work cells, and different workgroups in the same work cell can access each other's shared caches. Then, based on the target access information, the working status of the associated workgroup associated with the target access information in the work cell where the current workgroup is located is checked. When the working status of all associated workgroups is in the preparation state, the shared cache address information of the target workgroup is obtained, which can ensure that the loading of data and other work has been completed before the shared caches between different workgroups are accessed to each other, thereby ensuring the integrity of the data. Finally, based on the shared cache address information, the action of the current workgroup accessing the shared cache data of the target workgroup is executed, thereby allowing threads belonging to different workgroups to access each other's shared caches, thereby improving data utilization.
[0096] In an exemplary embodiment, Figure 4 As shown, step S202 includes steps S302 to S306. Among them:
[0097] Step S302: Obtain configuration mapping information corresponding to the work grid where the current work group is located.
[0098] The configuration mapping information is used to characterize the mapping configuration relationship between each work group, the target access information at least includes the target work group index and the target data address, and the configuration mapping information can be recorded in the work grid mapping table in the GMU module (such as Figure 5 As shown in the figure), the execution location and address information of each work group contained in each work grid is recorded in a table. The meanings of the symbols are:
[0099] (1) 1bit Valid indicates whether the worksheet is valid; 1 indicates a valid worksheet, and 0 indicates an invalid blank row.
[0100] (2) WG_Idx: 0~n-1, indicating the work-grid numbers of all the work-grids currently running in the GPU. There are n groups in total, which can be pre-allocated according to actual conditions.
[0101] (3) Grid_Id: The three-dimensional index number of the current working grid. Each working grid may contain multiple working groups.
[0102] (4) Grp_Idx: The number of the current work-group, which belongs to the current work grid; the number is 0 ~ m-1, m is the number of work-groups allowed in the current work grid.
[0103] (5) Slice_Id: The slice number of the workgroup to which this workgroup belongs, 0 to P-1, where P is the number of slices owned by the GPU;
[0104] (6) PE_Id: The PE number to which the workgroup belongs, 0 to Q-1, where Q is the number of processing units owned by each slice;
[0105] (7) SM_Addr: The starting address of the workgroup in the shared cache.
[0106] (8) SM_Ready: 1 bit indicates whether the shared cache data of the workgroup is ready. When this bit of all workgroups is 1, the workgroup can access the shared cache of other workgroups;
[0107] (9) SM_Release: 1 bit indicates that the work group ends accessing the shared cache of other work groups. When this bit of all work groups is 1, the work groups can modify their own shared cache data.
[0108] For example, the work grid with position number (0,1,1) generates 4 work groups, numbered grp0, grp1, grp2, grp3, respectively, which are allocated to Slice0_PE0, Slice0_PE1, Slice1_PE1 and Slice2_PE0. The starting address of their shared cache is recorded in the SM_Addr position, and SM_Ready and SM_Release are initialized to 0 for subsequent data synchronization access.
[0109] Step S304: searching the shared cache start address of the target work group in the configuration mapping information according to the target work group index.
[0110] Exemplarily, when the working status of all associated work groups is in the ready state, the GMU module allows the current work group thread to access the shared cache of other work groups. The GMU module can remap the addresses of the read and write commands of the shared cache. A shared cache address converter can be added to the GMU to recalculate the shared cache addresses in other work groups.
[0111] Step S306, calculating the shared cache address information of the target work group according to the shared cache start address and the target data address.
[0112] Exemplarily, after receiving the read, write, and atomic operations from the workgroup thread, the shared cache address converter can look up the mapping table GMT for address conversion, access the workgroup number grp_idx as needed, find the corresponding row information, obtain the slice_id and PE_id of the workgroup, and the shared cache starting address SM_Addr and other information, calculate the actual shared cache address as: SM_Addr[grp_idx] + sm_addr, and then enter the corresponding shared cache through the data connection, perform data read, write, and atomic operations, and then return the result along the original route.
[0113] by Figure 5 Take the mapping table in as an example, assuming that a thread (wave0) of working group grp0 sends the read and write command: data = SMemRead(grp2, 0x10). The command is first sent to the shared cache address converter of the GMU module. The shared cache address converter will look up the mapping table and get the SM_Addr of grp2 as 0x200, and calculate the actual address as 0x200+0x10 = 0x210. At the same time, it is known that grp2 is executed in PE1 of slice1, and the corresponding SMem read data command will be sent to PE1 of slice1, and its access address is 0x210. After PE1 reads the data, it will first return to the shared cache address converter along the data line, and then return to the original thread of grp0 along the original route.
[0114] In an exemplary embodiment, Figure 6 As shown, the above method may further include steps S402 to S406. Among them:
[0115] Step S402, obtaining a data processing task, and dividing the global space into a plurality of working grids according to the data processing task.
[0116] like Figure 7As shown, the GMU module can use a three-dimensional representation method to divide the parallel computing space, dividing the global space into work grids, requiring three groups of arrays, each of which uses three 32-bit integer arrays to represent the computing space: the three-dimensional size of the global space (Global Dimension): (GlobalDx, GlobalDy, GlobalDz); the offset of the global space (global offset): (GlobalFx, GlobalFy, GlobalFz); the three-dimensional size of the work-group: (GroupDx, GroupDy, GroupDz); the work-group index of each work-group is recorded as wid, using the triple (widx, widy, widz), which can be split into multiple work-items, and the index of each work-item is represented as lid (lidx, lidy, lidz). The GPU hardware needs to calculate the number of work-groups in three directions, GroupN (Number of work-groups), based on the global size of the computing space and the size of the work-group, so as to facilitate the loop creation of work-groups. The formula used is:
[0117] (GroupNx, GroupNy, GroupNz) = (Ceil(GlobalDx / GroupDx), Ceil(GblobalDy / GroupDy), Ceil(GlobalDz / GroupDz) )
[0118] The function Ceil indicates rounding up the remainder, and the global size must be able to divide the workgroup size. For example, if the three-dimensional size of the workgroup is (8, 4, 2), and the total global space size is (24, 20, 6), then the number of workgroups to be generated is (3, 5, 3), totaling: 3*5*3 = 45 workgroups. The GPCP unit of the GPU will be responsible for splitting the global size into multiple workgroups and sending them to each PE of the GPU according to the resource situation, including workgroup number, workgroup size and other information. Each workgroup needs to be executed in a PE on the GPU. When the PE receives the workgroup information, it allocates the shared cache SMem as needed and splits the workgroup into multiple GPU threads (waves). Each thread can be in simd32 or simd64 mode, allowing 32 or 64 work-items to perform calculations simultaneously.
[0119] Step S404: split the working grid into a plurality of working groups according to the three-dimensional structure data of each working grid.
[0120] Step S406: Determine the execution position information and data address information corresponding to each work group according to the three-dimensional structure data of the work group.
[0121] The configuration mapping information includes the execution location information and data address information of the work group. GPCP can construct a work grid and split the grid into multiple work groups, and record the split work groups into the mapping table in the GMU.
[0122] Exemplarily, the GPCP module may first receive configuration information from the driver, and then split the working grid and the working group according to the three-dimensional structure, split the global size in the Z direction into multiple layers, and the step size in the Z direction is GridGz; first split the XY two-dimensional direction of each layer, and split the next layer after the layer is completed. The two-dimensional space of each layer is split in a linear order, first in the X direction, then in the Y direction, and the step sizes in the X and Y directions are GridDx and GridDy respectively, generating the current working grid number Grid_Id, which is a triple (Grid_Idx, Grid_Idy, Grid_Idz). Next, GPCP may notify GMU to allocate an idle row (Valid is 0) from the mapping table and record the Grid_Id, and set Valid to 1. GMU returns the recorded WG_Idx to GPCP. After receiving Grid_Id, the GPCP module will start to split the working group Group in the working grid Grid, split the working group according to the three-dimensional structure, and split the GridGz in the Z direction into multiple layers; then split the XY of each layer in the two-dimensional direction, and split the next layer after the layer is completed. Subsequently, GPCP can allocate a free row from the mapping table and record the workgroup Group to obtain the workgroup number Grp_Idx. Finally, GPCP can schedule the current workgroup to execute in a processing unit in a slice based on the resource usage of each Slice, and record Slice_Id and PE_Id in the corresponding row of the corresponding mapping table. Each workgroup is allocated a shared cache in the corresponding processing unit, and its starting address is recorded in SM_Addr. The workgroup continues to be split into multiple hardware threads Wave in the processing unit and executed in the current processing unit.
[0123] In another exemplary embodiment, Figure 8 As shown, taking the above matrix calculation in the server as an example, a working grid is defined to contain 4 working groups, and the calculation content is: grp0: C0 = A0*B0; grp1: C1 = A0*B1; grp2: C2 = A1*B0; grp3: C3 = A1*B1.
[0124] In order to reduce the multiple loading of input matrix data, matrices A and B can be divided into 4 parts: A is divided into A0 and B0, B is divided into B0 and B1, and each work group loads a part into its own shared cache. The index of each work group can be used to indicate which piece of matrix data is loaded: bit[1]: 0 means loading matrix A, 1 means loading matrix B; bit[0]: 0 means loading sub-matrix 0, 1 means loading sub-matrix 1. The matrix data loaded by each work group is: grp0: Load A0; grp1: Load A1; grp2: Load B0; grp3: Load B1.
[0125] The server will read the sub-matrix into the shared cache of the work group according to grp_idx. The four work groups will load A0, A1, B0, and B1 into their respective shared caches. After each work group has completed loading, it will notify the GMU that the data is ready, and then suspend each work group until all work groups send SMemReady to the GMU. At this time, the SM_Ready of the four work groups in the GMT is 1, and then all threads of each work group can continue to execute.
[0126] Next, the server loads submatrix A into each workgroup thread, obtains the corresponding workgroup position through the workgroup index grp_idx, and obtains the workgroup index number where the submatrix is located by taking bit1 of the workgroup. Among them, grp0 and grp1 need to use A0 and store them in the shared cache of grp0; grp2 and grp3 need to use A1 and store them in the shared cache of grp1. The workgroup then notifies the GMU to perform the shared cache read data command SMemRead, along with the workgroup index and the offset address of the submatrix in the shared cache (here assumed to be 0x100). The GMU will perform address conversion and read data from the relevant shared cache.
[0127] Next, the server loads submatrix B for each workgroup thread, and obtains its corresponding workgroup position through the workgroup index grp_idx. It can take bit0 of the workgroup and add 0x2 to obtain the workgroup index number where the submatrix is located. Among them, grp0 and grp2 need to use B0 and store it in the shared cache of grp2; grp1 and grp3 need to use B1 and store it in the shared cache of grp3. The workgroup then notifies the GMU to perform the shared cache read data command SMemRead, along with the workgroup index and the offset address of the submatrix in the shared cache (here assumed to be 0x100). The GMU will perform address conversion and read data from the relevant shared cache.
[0128] After each thread in the work group obtains the sub-matrix, it will use SMemRelease to notify the GMU that the shared cache access is over. Then each work group thread can use the matrix multiply-accumulate (MMA) function to calculate the sub-matrix multiplication operation: SubC = SubA * SubB, and finally output the calculation result.
[0129] It can be seen from this embodiment that after the matrices A and B are divided into four parts, they are loaded into the shared cache by four working groups respectively, and finally access each other to complete the final matrix calculation. Compared with the previous method, matrices A and B are not read in repeatedly, reducing the pressure of external read and write bandwidth.
[0130] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0131] Based on the same inventive concept, the embodiment of the present application also provides a data processing device for accessing a shared cache across workgroups for implementing the data processing method for accessing a shared cache across workgroups involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more data processing device embodiments for accessing a shared cache across workgroups provided below can be found in the above limitations on the data processing method for accessing a shared cache across workgroups, and will not be repeated here.
[0132] In an exemplary embodiment, Fig. 9 As shown, a data processing device for accessing a shared cache across workgroups is provided, comprising: a data acquisition module 902, a status checking module 904, an address acquisition module 906 and a data access module 908, wherein:
[0133] The data acquisition module 902 is used to acquire target access information and determine a target workgroup according to the target access information, wherein the target workgroup and the current workgroup are in the same work grid, and the work grid includes multiple workgroups;
[0134] A status checking module 904 is used to check the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information;
[0135] An address acquisition module 906, configured to acquire the shared cache address information of the target work group when the working states of all associated work groups are in the preparation state;
[0136] The data access module 908 is used to execute the action of the current work group accessing the shared cache data of the target work group according to the shared cache address information.
[0137] In one embodiment, the target access information includes at least a target workgroup index and a target data address; the data acquisition module 902 includes:
[0138] The information acquisition submodule is used to obtain the configuration mapping information corresponding to the work grid where the current work group is located, wherein the configuration mapping information is used to characterize the mapping configuration relationship between each work group;
[0139] An address determination submodule, used to find the shared cache start address of the target workgroup in the configuration mapping information according to the target workgroup index;
[0140] The address calculation submodule is used to calculate the shared cache address information of the target work group according to the shared cache start address and the target data address.
[0141] In one embodiment, the device further comprises:
[0142] A data receiving module, used for acquiring data to be processed;
[0143] A data loading module is used to load the data to be processed into the shared cache of multiple working groups in the current working grid respectively;
[0144] A first state setting module is used to set the working state of the working group to a ready state when the working group is loaded;
[0145] The first process processing module is used to stop the data modification action on the shared cache of the work group.
[0146] In one embodiment, the device further comprises:
[0147] A second status setting module, used to set the working status of the current working group and the associated working group to the access termination status;
[0148] The second process processing module is used to set the working status of the current work group and the associated work groups to the data change state when the working status of all associated work groups in the current work grid is the end of access state, wherein the data change state is a state in which the work group can perform a shared cache data change operation.
[0149] In one embodiment, the work group includes a main thread and at least one child thread; the first state setting module is specifically used to: respond to a preparation command issued by the main thread of the work group, set the working state of the work group to a preparation state; accordingly, the first process processing module is specifically used to: stop the data change action on the shared cache in the main thread; stop the data change action on the shared cache in each child thread through a thread synchronization command.
[0150] In one embodiment, the configuration mapping information includes execution location information and data address information of the work group; the apparatus further includes:
[0151] The space division module is used to obtain data processing tasks and divide the global space into multiple working grids according to the data processing tasks;
[0152] A work group division module is used to divide a work cell into multiple work groups according to the three-dimensional structure data of each work cell;
[0153] The information processing module is used to determine the execution position information and data address information corresponding to each work group according to the three-dimensional structure data of the work group.
[0154] Each module in the above-mentioned data processing device for accessing a shared cache across workgroups can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.
[0155] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Fig.10As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as configuration mapping information. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data processing method for accessing a shared cache across workgroups is implemented.
[0156] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.11 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data processing method for accessing a shared cache across workgroups is implemented.
[0157] Those skilled in the art will understand that Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0158] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: obtaining target access information, and determining a target workgroup based on the target access information, wherein the target workgroup and the current workgroup are in the same workcell, and the workcell includes multiple workgroups; checking the working status of an associated workgroup associated with the target access information in the workcell where the current workgroup is located based on the target access information; when the working status of all associated workgroups is in a ready state, obtaining shared cache address information of the target workgroup; and executing an action of the current workgroup accessing the shared cache data of the target workgroup based on the shared cache address information.
[0159] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining configuration mapping information corresponding to the work cell where the current work group is located, wherein the configuration mapping information is used to characterize the mapping configuration relationship between each work group; searching for the shared cache starting address of the target work group in the configuration mapping information according to the target work group index; and calculating the shared cache address information of the target work group according to the shared cache starting address and the target data address.
[0160] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining the data to be processed; loading the data to be processed into the shared caches of multiple work groups in the current work cell respectively; when the work group is loaded, setting the working state of the work group to the ready state; and stopping the data change action on the shared cache of the work group.
[0161] In one embodiment, when the processor executes the computer program, the following steps are also implemented: setting the working status of the current work group and the associated work group to the end access status; when the working status of all the associated work groups in the current work cell is the end access status, setting the working status of the current work group and the associated work group to the data change status, wherein the data change status is a state in which the work group can perform a shared cache data change operation.
[0162] In one embodiment, when the processor executes the computer program, the following steps are also implemented: in response to a preparation command issued by the main thread of the work group, the working state of the work group is set to a preparation state; the data change action on the shared cache in the main thread is stopped; and the data change action on the shared cache in each child thread is stopped through a thread synchronization command.
[0163] In one embodiment, when the processor executes the computer program, the following steps are also implemented: obtaining data processing tasks, and dividing the global space into multiple working grids according to the data processing tasks; splitting the working grid into multiple working groups according to the three-dimensional structure data of each working grid; and determining the execution position information and data address information corresponding to each working group according to the three-dimensional structure data of the working group.
[0164] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: obtaining target access information, and determining a target workgroup based on the target access information, wherein the target workgroup and the current workgroup are in the same workcell, and the workcell includes multiple workgroups; checking the working status of an associated workgroup associated with the target access information in the workcell where the current workgroup is located based on the target access information; when the working status of all associated workgroups is in a ready state, obtaining shared cache address information of the target workgroup; and executing an action of the current workgroup accessing the shared cache data of the target workgroup based on the shared cache address information.
[0165] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining configuration mapping information corresponding to the work cell where the current work group is located, wherein the configuration mapping information is used to characterize the mapping configuration relationship between each work group; searching for the shared cache starting address of the target work group in the configuration mapping information according to the target work group index; and calculating the shared cache address information of the target work group according to the shared cache starting address and the target data address.
[0166] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the data to be processed; loading the data to be processed into the shared caches of multiple work groups in the current work cell respectively; when the work group is loaded, setting the working state of the work group to the ready state; and stopping the data change action on the shared cache of the work group.
[0167] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: setting the working status of the current work group and the associated work group to the end access status; when the working status of all the associated work groups in the current work cell is the end access status, setting the working status of the current work group and the associated work group to the data change status, wherein the data change status is a state in which the work group can perform shared cache data change operations.
[0168] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to a preparation command issued by the main thread of the work group, the working state of the work group is set to a preparation state; the data change action on the shared cache in the main thread is stopped; and the data change action on the shared cache in each child thread is stopped through a thread synchronization command.
[0169] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining data processing tasks, and dividing the global space into multiple working grids according to the data processing tasks; splitting the working grid into multiple working groups according to the three-dimensional structural data of each working grid; and determining the execution position information and data address information corresponding to each working group according to the three-dimensional structural data of the working group.
[0170] In one embodiment, a computer program product is provided, including a computer program, which implements the following steps when executed by a processor: obtaining target access information, and determining a target workgroup based on the target access information, wherein the target workgroup and the current workgroup are in the same workcell, and the workcell includes multiple workgroups; checking the working status of an associated workgroup associated with the target access information in the workcell where the current workgroup is located based on the target access information; when the working status of all associated workgroups is in a ready state, obtaining shared cache address information of the target workgroup; and executing an action of the current workgroup accessing the shared cache data of the target workgroup based on the shared cache address information.
[0171] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining configuration mapping information corresponding to the work cell where the current work group is located, wherein the configuration mapping information is used to characterize the mapping configuration relationship between each work group; searching for the shared cache starting address of the target work group in the configuration mapping information according to the target work group index; and calculating the shared cache address information of the target work group according to the shared cache starting address and the target data address.
[0172] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining the data to be processed; loading the data to be processed into the shared caches of multiple work groups in the current work cell respectively; when the work group is loaded, setting the working state of the work group to the ready state; and stopping the data change action on the shared cache of the work group.
[0173] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: setting the working status of the current work group and the associated work group to the end access status; when the working status of all the associated work groups in the current work cell is the end access status, setting the working status of the current work group and the associated work group to the data change status, wherein the data change status is a state in which the work group can perform shared cache data change operations.
[0174] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: in response to a preparation command issued by the main thread of the work group, the working state of the work group is set to a preparation state; the data change action on the shared cache in the main thread is stopped; and the data change action on the shared cache in each child thread is stopped through a thread synchronization command.
[0175] In one embodiment, when the computer program is executed by the processor, the following steps are also implemented: obtaining data processing tasks, and dividing the global space into multiple working grids according to the data processing tasks; splitting the working grid into multiple working groups according to the three-dimensional structural data of each working grid; and determining the execution position information and data address information corresponding to each working group according to the three-dimensional structural data of the working group.
[0176] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.
[0177] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0178] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0179] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A data processing method for accessing a shared cache across workgroups, characterized in that: The method comprises: Acquire target access information, and determine a target workgroup according to the target access information, wherein the target workgroup and the current workgroup are in the same work grid, and the work grid includes a plurality of the workgroups; Checking the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information; When the working states of all the associated working groups are in the preparation state, obtaining the shared cache address information of the target working group; According to the shared cache address information, an action is performed by the current working group to access the shared cache data of the target working group.
2. The method according to claim 1, characterized in that The target access information includes at least a target workgroup index and a target data address; the step of obtaining the shared cache address information of the target workgroup includes: Acquire configuration mapping information corresponding to the work grid where the current work group is located, wherein the configuration mapping information is used to characterize the mapping configuration relationship between various work groups; According to the target workgroup index, searching the configuration mapping information for a shared cache start address of the target workgroup; The shared cache address information of the target working group is calculated according to the shared cache start address and the target data address.
3. The method according to claim 1, characterized in that Before obtaining the target access information, the method further includes: Get the data to be processed; Loading the data to be processed into the shared caches of the plurality of working groups in the current working grid respectively; When the working group is loaded, setting the working state of the working group to a standby state; Stop the data modification action on the shared cache of the work group.
4. The method according to claim 3, characterized in that After accessing the shared cache data of the target workgroup according to the shared cache address information, the method further includes: Setting the working status of the current working group and the associated working group to the access-end status; When the working status of all the associated workgroups in the current work grid is the end-of-access status, the working status of the current workgroup and the associated workgroup is set to a data change status, wherein the data change status is a status in which the workgroup can perform a shared cache data change operation.
5. The method according to claim 3, characterized in that: The working group includes a main thread and at least one sub-thread; when the working group is loaded, setting the working state of the working group to a ready state includes: In response to a preparation command issued by the main thread of the work group, setting the working state of the work group to a preparation state; Accordingly, the action of stopping the data change action on the shared cache of the work group includes: Stop the data modification action on the shared cache in the main thread; The data modification action on the shared cache in each of the sub-threads is stopped through a thread synchronization command.
6. The method according to any one of claims 2 to 5, characterized in that: The configuration mapping information includes execution location information and data address information of the work group; the method further includes: Acquire a data processing task, and divide the global space into a plurality of working grids according to the data processing task; According to the three-dimensional structural data of each working cell, the working cell is divided into a plurality of working groups; The execution position information and the data address information corresponding to each of the working groups are determined according to the three-dimensional structure data of the working group.
7. A data processing device for accessing a shared cache across workgroups, characterized in that: The device comprises: A data acquisition module, used for acquiring target access information, and determining a target workgroup according to the target access information, wherein the target workgroup and the current workgroup are in the same work grid, and the work grid includes a plurality of the workgroups; A status checking module, used for checking the working status of the associated working group associated with the target access information in the working grid where the current working group is located according to the target access information; An address acquisition module, used for acquiring the shared cache address information of the target work group when the working states of all the associated work groups are in the preparation state; A data access module is used to execute the action of the current working group accessing the shared cache data of the target working group according to the shared cache address information.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.
