A method and system for improving the speed of writing shared main memory critical resources in parallel from cores based on a new generation sunway many-core processor
By creating a copy on the private local data memory of the slave core and performing a reduction operation, the problem of slow parallel write speed of slave cores to shared main memory critical resources in the Shenwei many-core processor is solved, realizing more efficient data transmission and aggregation function operation, and improving processor performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANDONG COMP SCI CENTNAT SUPERCOMP CENT IN JINAN
- Filing Date
- 2023-07-18
- Publication Date
- 2026-07-07
AI Technical Summary
In the new generation of Shenwei many-core processors, parallel write conflicts are prone to occur when the slave cores write to shared critical main memory resources, resulting in a decrease in speed.
A copy is created on the kernel's private local data storage, and locking operations on critical shared main memory resources are avoided through reduction operations. Data transfer and aggregate function operations are performed using direct memory access and remote memory access channels.
It improves the speed of parallel writes to shared main memory critical resources from the kernel, reduces the difficulty of program writing, and improves the efficiency of kernel reduction.
Smart Images
Figure CN116909741B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of electronic information technology, specifically relating to a method and system for improving the speed of parallel writes of shared main memory critical resources by the new generation Shenwei many-core processor. Background Technology
[0002] Supercomputing is a core competitive advantage in technological innovation and a vital support for national security, economic and social development. Developed countries worldwide are vying for dominance in supercomputing. Faced with this challenge, my country has placed great emphasis on the independent research and development of domestically produced supercomputer chips, and has already achieved remarkable results. The Sunway series of supercomputers and the Shenwei series of many-core processors are representative works of my country's independent innovation, demonstrating my country's strength and potential in the field of supercomputing.
[0003] The SW26010pro is a new generation of high-performance heterogeneous many-core processor independently developed in my country. The SW26010pro processor has 6 core groups (CGs). Each processor has a total of 96GB of memory. This memory can be evenly distributed among core groups 1, 2, 3, and 6. Each core group contains one Managed Processing Element (MPE) and 64 Computing Processing Elements (CPEs). An SW26010pro processor has 6 MPEs and 384 CPEs. Each slave core has its own instruction cache and Local Data Memory (LDM), also known as Local Data Cache (LD Cache). Data transfer between main memory and LDM is called Direct Memory Access (DMA). Data transfer between main memory and LD cache is called load / store instructions. Data transfer between the LDMs of the slave cores is called Remote Memory Access (RMA). Both MPEs and CPEs use the SW64 instruction set. The hardware architecture of the SW26010pro processor is as follows: Figure 1 As shown.
[0004] The new generation of Shenwei many-core processors allows slave cores to directly access main memory data via load / store instructions, enabling parallel task execution. However, with 64 slave cores executing different tasks, and the main memory accesses by these tasks being completely random, multiple slave cores may access the same main memory address simultaneously, resulting in multiple slave cores updating the same main memory data at the same time. A common method for handling parallel write conflicts in the slave core group is to lock the critical resources accessed by each slave core. Before accessing the critical resource, the array unit is locked. After the slave core finishes writing data, the lock is released. However, locking and unlocking operations require additional runtime, significantly reducing the speed of parallel writes to shared main memory critical resources by slave cores. Summary of the Invention
[0005] To address the shortcomings of existing technologies, this invention proposes a method to improve the speed of parallel writes to shared main memory critical resources from the slave core.
[0006] This invention addresses the problem of parallel write conflicts that occur when slave cores need to write to shared critical main memory resources during parallel task execution in a single-core group. This can lead to multiple slave cores simultaneously writing to the same main memory address. The main solution involves creating a copy on the slave core's private local data memory and performing a reduction operation to avoid locking operations on shared critical main memory resources during parallel writes, thereby improving the speed of parallel writes to shared critical main memory resources by slave cores.
[0007] This invention also proposes a system for improving the speed of parallel writes to shared main memory critical resources from the kernel.
[0008] The technical solution of this invention is as follows:
[0009] A method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on the new generation Shenwei many-core processor includes:
[0010] Each slave kernel allocates a data space on its private local data memory that is the same size as the critical resource data in main memory. This space is used to store the critical resource data in main memory. Critical resource data refers to data that multiple slave kernels need to access and modify simultaneously.
[0011] From the kernel, critical resource data in main memory is copied to their respective private local data memory via direct memory access (DMA) channels;
[0012] Each slave core performs read and write operations on critical resource data in its private local data memory according to the assigned subtasks; a subtask refers to breaking down a large-scale computing task into multiple smaller computing tasks and assigning them to each slave core for execution.
[0013] After all the subtasks of the slave cores have been executed, each slave core initiates a reduction operation through the remote memory access (RMA) channel. The reduction operation refers to performing a certain aggregation function operation on the critical resource data in the private local data memory of multiple slave cores to obtain a final result.
[0014] The critical resource data after reduction operations in the private local data memory is written back to main memory through the direct memory access (DMA) channel.
[0015] According to a preferred embodiment of the present invention, the protocol operation refers to: transmitting critical resource data in each private local data memory based on the tree-structured RMA communication protocol of the kernel cluster.
[0016] According to a preferred embodiment of the present invention, the critical resource data within each private local data memory is transmitted based on a tree-structured RMA communication protocol from the kernel cluster, including:
[0017] The tree-structured communication model of the kernel clusters is constructed as follows: The tree-structured communication model of the kernel clusters consists of four layers; the first layer includes 64 kernels; the second layer includes 16 kernels designated as data collection kernels within each kernel cluster of the first layer; the third layer includes 4 kernels designated as data collection kernels after the kernels of the second layer logically form a new kernel cluster; the fourth layer is 1 kernel designated as data collection kernel after the kernels of the third layer logically form a new kernel cluster.
[0018] In the tree-structured communication model for constructing a cluster of slave cores, each slave core has a unique logical number i, i = 0, 1, 2, 3, ..., 63, which is used to identify its position in the entire slave core group;
[0019] The arrows indicate that one slave core is sending data to another slave core, and the numbers on the arrows indicate which round of communication this is; each round of communication is carried out between one or more adjacent slave core clusters, and each slave core cluster consists of 4 adjacent slave cores arranged in a 2*2 matrix to execute tasks in parallel;
[0020] In the first round of communication, within a single slave core cluster, each slave core with logical number j receives data from slave cores with logical numbers j+1, j+8, and j+9, where j∈i, j=0, 2, 4, 6, 16, 18, 20, 22, 32, 34, 36, 38, 48, 50, 52, 54; after these data are transmitted to the slave core with logical number j, a reduction operation is performed.
[0021] In the second round of communication, the four slave cores that received data from the slave core cluster in the first round of communication are logically combined into a new slave core cluster. The second round of communication involves four new slave core clusters in a logical sense. Each new slave core cluster has a representative node, namely the slave core with logical numbers k = 0, 4, 32, and 36. The representative nodes of the new slave core clusters all receive data from slave cores with logical numbers k+2, k+16, and k+18.
[0022] In the third round of communication, the slave cores with logical numbers i = 4, 32, and 36 send data to the slave core with logical number i = 0.
[0023] According to a preferred embodiment of the present invention, the reduction operations include: finding the maximum value OP_MAX; finding the minimum value OP_MIN; finding the sum OP_SUM; finding the product OP_PROD; performing a logical AND OP_LAND; performing a bitwise AND OP_BAND; performing a logical OR OP_LOR; performing a bitwise OR OP_BOR; performing a logical XOR OP_LXOR; and performing a bitwise XOR OP_BXOR.
[0024] According to a preferred embodiment of the present invention, when critical resource data in the main memory is copied to their respective private local data memory, it is ensured that this critical resource data is completely consistent among the slave cores.
[0025] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program implementing steps of a method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on a new generation Shenwei many-core processor.
[0026] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the steps of a method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on a new generation of Shenwei many-core processors.
[0027] A system for improving the speed of parallel writes to shared main memory critical resources by slave cores based on the new generation Shenwei many-core processor includes:
[0028] The data space allocation unit is configured to: request data space from the kernel on its private local data memory that is the same size as the critical resource data in the main memory;
[0029] The main memory data copy unit is configured such that: the slave cores copy critical resource data in the main memory to their respective private local data memory through the direct memory access (DMA) channel, and ensure that the critical resource data is completely consistent among the slave cores;
[0030] The read / write operation unit is configured such that each slave core performs read / write operations on critical resource data in its private local data memory according to the assigned subtask.
[0031] Reduce operation unit: After all the subtasks of the slave cores have been executed, each slave core initiates a reduction operation through the remote memory access (RMA) channel. The reduction operation refers to performing a certain aggregation function operation on the critical resource data in the private local data memory of multiple slave cores to obtain a final result.
[0032] The main memory write-back unit is configured to write critical resource data after reduction operations in the private local data memory back to main memory via a direct memory access (DMA) channel.
[0033] The beneficial effects of this invention are as follows:
[0034] 1. This invention improves the speed of parallel writing to critical resources of shared main memory by creating a copy on the private local data memory of the slave kernel and performing reduction, thereby avoiding locking operations on critical resources of shared main memory during parallel writing by the slave kernel.
[0035] 2. This invention provides a kernel reduction function based on a kernel cluster, which improves the efficiency of kernel reduction while reducing the difficulty of program writing.
[0036] 3. The method and function proposed in this invention are applicable to the Sunway supercomputer based on the new generation of Shenwei many-core processor and have a certain degree of universality. Attached Figure Description
[0037] Figure 1 This is a schematic diagram of the hardware architecture of the SW26010pro processor.
[0038] Figure 2 This is a flowchart illustrating the method for improving the speed of parallel writing to shared main memory critical resources by slave cores based on the new generation Shenwei many-core processor of the present invention.
[0039] Figure 3 This is a schematic diagram of the tree-structured communication protocol implemented by multiple slave core clusters in the Shenwei many-core processor architecture of the present invention;
[0040] Figure 4 This is a schematic diagram of the tree-like communication model of the kernel cluster in this invention. Detailed Implementation
[0041] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto.
[0042] Example 1
[0043] A method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on the new generation Shenwei many-core processor includes:
[0044] Each slave kernel allocates a data space on its private local data memory that is the same size as the critical resource data in main memory to store the critical resource data in main memory. For example, if the critical resource size in main memory is 1KB, then a data space of the same size as 1KB will be allocated on the slave kernel's private local data memory. Critical resource data refers to data that multiple slave kernels need to access and modify simultaneously; without locking, this will lead to data inconsistency problems.
[0045] Slave kernels copy critical resource data from main memory to their respective private local data memory via direct memory access (DMA) channels. This allows slave kernels to read and write critical resource data on their private local data memory without locking the main memory. Locking reduces the speed at which slave kernels read and write shared critical main memory resources because it incurs synchronization waits and communication overhead between slave kernels.
[0046] Each slave core performs read and write operations on critical resource data in its private local data memory according to the assigned subtasks; a subtask refers to breaking down a large-scale computing task into multiple smaller computing tasks and assigning them to each slave core for execution.
[0047] After all the subtasks of the slave cores have been executed, each slave core initiates a reduction operation through the remote memory access (RMA) channel. The reduction operation refers to performing a certain aggregation function operation on the critical resource data in the private local data memory of multiple slave cores to obtain a final result.
[0048] Slave core 0 writes the reduced critical resource data from its private local data memory back to main memory via a direct memory access (DMA) channel. This completes the read / write operation on the shared main memory critical resource without requiring locking.
[0049] The method of this invention can effectively improve the speed of reading and writing critical resources of shared main memory from cores in the Shenwei many-core processor, thereby improving the performance and efficiency of supercomputers. Figure 2 The flowchart of the above method is shown.
[0050] Example 2
[0051] The method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on the new generation Shenwei many-core processor, as described in Example 1, differs in that:
[0052] Protocol operation refers to the communication of critical resource data in each private local data storage based on the tree-structured RMA communication protocol of the kernel cluster.
[0053] Critical resource data within their respective private local data memories are communicated using a tree-structured RMA communication protocol from the kernel cluster, including:
[0054] like Figure 3 The diagram illustrates a tree-structured communication protocol implemented using multiple slave core clusters within the Shenwei many-core processor's hardware architecture. Each slave core cluster is a 2×2 array of slave cores. This protocol fully utilizes the Remote Memory Access (RMA) data transfer method between multiple slave core LDMs, effectively reducing the time required for data transfer and synchronization.
[0055] Constructing a tree-like communication model from the kernel cluster, such as Figure 4 As shown, the tree-structured communication model of the kernel cluster consists of four layers: the first layer includes 64 kernels; the second layer includes 16 kernels designated for data collection within each kernel cluster of the first layer; the third layer includes 4 kernels designated for data collection after the kernels of the second layer logically form a new kernel cluster; and the fourth layer is 1 kernel designated for data collection after the kernels of the third layer logically form a new kernel cluster.
[0056] In the tree-structured communication model for constructing a cluster of slave cores, each slave core has a unique logical number i, i = 0, 1, 2, 3, ..., 63, which is used to identify its position in the entire slave core group;
[0057] The arrows indicate that one slave core is sending data to another slave core, and the numbers on the arrows indicate which round of communication this is; each round of communication is carried out between one or more adjacent slave core clusters, and each slave core cluster consists of 4 adjacent slave cores arranged in a 2*2 matrix to execute tasks in parallel;
[0058] In the first round of communication, within a single slave core cluster, each slave core with logical number j receives data from slave cores with logical numbers j+1, j+8, and j+9, where j∈i, j=0, 2, 4, 6, 16, 18, 20, 22, 32, 34, 36, 38, 48, 50, 52, 54. After these data are transmitted to the slave core with logical number j, a reduction operation is performed. This round of communication involves 16 adjacent and structurally identical slave core clusters, and each slave core cluster has one slave core receiving data from the other three slave cores.
[0059] In the second round of communication, the four slave cores that received data from the slave core cluster in the first round of communication are logically combined into a new slave core cluster. The second round of communication involves four new slave core clusters in a logical sense. Each new slave core cluster has a representative node, namely the slave core with logical numbers k = 0, 4, 32, and 36. The representative nodes of the new slave core clusters all receive data from slave cores with logical numbers k+2, k+16, and k+18.
[0060] In the third round of communication, the slave cores with logical numbers i = 4, 32, and 36 send data to the slave core with logical number i = 0.
[0061] This invention implements the reduction function CPEs_Reduce() within a single slave core cluster in the Shenwei many-core processor architecture. Programmers can call this function to perform reduction operations on single or multiple slave core clusters within a slave core group. The design and implementation of this function reduces the difficulty of slave core cluster reduction operations for the Shenwei many-core processor.
[0062] The implementation of the CPEs_Reduce() function is shown below:
[0063]
[0064] The source code for the parallel write operation to shared main memory critical resources and locking is as follows:
[0065]
[0066] The code below shows how the slave kernel creates a copy and performs a reduction on its private local data storage to avoid locking critical resources in shared main memory during parallel writes. The CPEs_Reduce() function is based on the RMA communication protocol of the slave kernel cluster.
[0067]
[0068]
[0069] The reduction operations include: finding the maximum value (OP_MAX); finding the minimum value (OP_MIN); finding the sum (OP_SUM); finding the product (OP_PROD); logical AND (OP_LAND); bitwise AND (OP_BAND); logical OR (OP_LOR); bitwise OR (OP_BOR); logical XOR (OP_LXOR); and bitwise XOR (OP_BXOR).
[0070] When copying critical resource data from main memory to their respective private local data memory, it is ensured that this critical resource data is completely consistent across each slave core.
[0071] The random ray method (TRRM) is a novel approach to solving partial differential equations (PDEs) based on the Method of Characteristics (MOC). TRRM programs can be used for numerical simulations of nuclear reactors, employing random rather than deterministic characteristic trajectories to simulate and solve problems. When the TRRM program runs in parallel on a Shenwei many-core processor, different slave cores simulate different rays according to the parallel approach. Because the trajectories of these rays are completely random, different slave cores may access the same data in main memory. That is, two different rays may traverse the same region, and the slave cores simulating these two rays may simultaneously perform calculations on this region. Data related to this region then becomes critical resources. To ensure the correctness of the calculation results, locking operations are required for these critical resources. While locking ensures the correctness of the calculation results, its additional overhead reduces the parallel efficiency of the slave cores.
[0072] This invention performs parallel optimization of the random ray tracing method. The computation time (seconds) and speedup of the random ray tracing method program optimized using the method of this invention are shown in Table 1.
[0073] Table 1
[0074]
[0075]
[0076] As shown in Table 1, the method of the present invention has a very good acceleration effect.
[0077] Example 3
[0078] A computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method for improving the speed of parallel writes to shared main memory critical resources based on the new generation Shenwei many-core processor described in Embodiment 1 or 2.
[0079] Example 4
[0080] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method for improving the speed of parallel writes to shared main memory critical resources based on a new generation Shenwei many-core processor as described in Embodiment 1 or 2.
[0081] Example 5
[0082] A system for improving the speed of parallel writes to shared main memory critical resources by slave cores based on the new generation Shenwei many-core processor includes:
[0083] The data space allocation unit is configured to: request data space from the kernel on its private local data memory that is the same size as the critical resource data in the main memory;
[0084] The main memory data copy unit is configured such that: the slave cores copy critical resource data in the main memory to their respective private local data memory through the direct memory access (DMA) channel, and ensure that the critical resource data is completely consistent among the slave cores;
[0085] The read / write operation unit is configured such that each slave core performs read / write operations on critical resource data in its private local data memory according to the assigned subtask.
[0086] Reduce operation unit: After all the subtasks of the slave cores have been executed, each slave core initiates a reduction operation through the remote memory access (RMA) channel. The reduction operation refers to performing a certain aggregation function operation on the critical resource data in the private local data memory of multiple slave cores to obtain a final result.
[0087] The main memory write-back unit is configured to write critical resource data after reduction operations in the private local data memory back to main memory via a direct memory access (DMA) channel.
Claims
1. A method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on a new generation of Shenwei many-core processors, characterized in that, include: The kernel requests a data space on its private local data memory that is the same size as the critical resource data in the main memory, and uses it to store the critical resource data in the main memory. From the kernel, critical resource data in main memory is copied to their respective private local data memory via direct memory access channels; Each slave core performs read and write operations on critical resource data in its private local data memory according to the assigned subtasks; a subtask refers to breaking down a large-scale computing task into multiple smaller computing tasks and assigning them to each slave core for execution. After all the subtasks of the slave cores have been executed, each slave core initiates a reduction operation through the remote memory access channel. The reduction operation refers to performing a certain aggregation function operation on the critical resource data in the private local data memory of multiple slave cores to obtain a final result. The critical resource data after reduction operations in the private local data memory is written back to the main memory through the direct memory access channel. Protocol operation refers to the communication of critical resource data in each private local data storage based on the tree-structured RMA communication protocol of the kernel cluster.
2. The method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on the new generation Shenwei many-core processor according to claim 1, characterized in that, Critical resource data within their respective private local data memories are communicated using a tree-structured RMA communication protocol from the kernel cluster, including: The tree-structured communication model of the kernel clusters is constructed as follows: The tree-structured communication model of the kernel clusters consists of four layers; the first layer includes 64 kernels; the second layer includes 16 kernels designated as data collection kernels within each kernel cluster of the first layer; the third layer includes 4 kernels designated as data collection kernels after the kernels of the second layer logically form a new kernel cluster; the fourth layer is 1 kernel designated as data collection kernel after the kernels of the third layer logically form a new kernel cluster. In the tree-structured communication model for constructing a cluster of slave cores, each slave core has a unique logical number i, i=0, 1, 2, 3, ..., 63, which is used to identify its position in the entire slave core group; The arrows indicate that one slave core is sending data to another slave core, and the numbers on the arrows indicate which round of communication this is; each round of communication is carried out between one or more adjacent slave core clusters, and each slave core cluster consists of 4 adjacent slave cores arranged in a 2*2 matrix to execute tasks in parallel; In the first round of communication, within a single slave core cluster, each slave core with logical number j receives data from slave cores with logical numbers j+1, j+8, and j+9, where j∈i, j=0, 2, 4, 6, 16, 18, 20, 22, 32, 34, 36, 38, 48, 50, 52, 54; after these data are transmitted to the slave core with logical number j, a reduction operation is performed. In the second round of communication, the four slave cores that received data from the slave core cluster in the first round of communication are logically combined into a new slave core cluster. The second round of communication involves four new slave core clusters in a logical sense. Each new slave core cluster has a representative node, namely the slave core with logical numbers k=0, 4, 32, and 36. The representative nodes of the new slave core clusters all receive data from slave cores with logical numbers k+2, k+16, and k+18. In the third round of communication, the slave cores with logical numbers i=4, 32, and 36 send data to the slave core with logical number i=0.
3. The method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on the new generation Shenwei many-core processor according to claim 1, characterized in that, The reduction operations include: finding the maximum value (OP_MAX); finding the minimum value (OP_MIN); finding the sum (OP_SUM); finding the product (OP_PROD); logical AND (OP_LAND); bitwise AND (OP_BAND); logical OR (OP_LOR); bitwise OR (OP_BOR); logical XOR (OP_LXOR); and bitwise XOR (OP_BXOR).
4. A method for improving the speed of parallel writes to shared main memory critical resources by slave cores based on a new generation Shenwei many-core processor, as described in any one of claims 1-3, characterized in that, When copying critical resource data from main memory to their respective private local data memory, it is ensured that this critical resource data is completely consistent across each slave core.
5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method for improving the speed of parallel writes to shared main memory critical resources based on the new generation Shenwei many-core processor as described in any one of claims 1-4.
6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of any of claims 1-4, which are based on the new generation Shenwei many-core processor, to improve the speed of parallel writes to shared main memory critical resources by slave cores.
7. A system for improving the speed of parallel writes to shared main memory critical resources by slave cores based on a new generation Shenwei many-core processor, characterized in that, include: The data space allocation unit is configured to: request data space from the kernel on its private local data memory that is the same size as the critical resource data in the main memory; The main memory data copy unit is configured such that: the slave cores copy critical resource data in the main memory to their respective private local data memory through the direct memory access channel, and ensure that the critical resource data is completely consistent among the slave cores; The read / write operation unit is configured such that each slave core performs read / write operations on critical resource data in its private local data memory according to the assigned subtask. Reduction Operation Unit: After all the subtasks of the slave cores have been executed, each slave core initiates a reduction operation through the remote memory access channel. The reduction operation refers to performing a certain aggregation function operation on the critical resource data in the private local data memory of multiple slave cores to obtain a final result. The reduction operation refers to: reducing the critical resource data in the private local data memory of each slave core based on the tree-shaped RMA communication protocol of the slave core cluster. The main memory write-back unit is configured to write critical resource data after reduction operations in the private local data memory back to main memory via the direct memory access channel.