Memory access reordering processing method and system based on simt architecture, device and medium

Abstract

The application provides a memory access reordering processing method, system, device and medium based on a SIMT architecture, and belongs to the technical field of memory access optimization. The method comprises the following steps: configuring a memory access reordering unit in a parallel processing unit pipeline based on a SIMT architecture, and constructing a cache reordering unit between cache unit levels; incrementally modifying a decoding and execution module in the parallel processing unit and extending a memory access optimization instruction and decoding logic; setting a cooperative work flow of the two units; constructing a functional architecture of the cache reordering unit, realizing data prefetching, classified storage and on-demand feedback through functional configuration; dividing software instruction kernels into memory access rule type kernels and memory access irregular type kernels, and configuring a software and hardware processing mechanism for the kernels; by configuring software side parameters and hardware side instructions for the memory access irregular type kernels, the parallel processing unit is controlled to dynamically reorder memory access requests, and logically adjacent address indexes are concentratedly allocated to the same processing unit for execution.

Description

Technical Field

[0001] This invention belongs to the field of memory access optimization technology, and more specifically relates to a memory access reordering processing method, system, device and medium based on SIMT architecture. Background Technology

[0002] With the continuous increase in computing power, parallel processing units based on the SIMT architecture have become increasingly important products on the computing edge due to their highly parallel computing capabilities and excellent computing performance. They are widely used in rule-based computationally intensive tasks such as machine learning, artificial intelligence, and image recognition. However, with the continuous expansion of application fields, especially when facing irregular memory access computing, the parallel computing capabilities of the traditional SIMT architecture are becoming increasingly difficult to achieve, and the utilization rate of computing units is declining sharply, gradually becoming a barrier to the development of SIMT architecture parallel processing units.

[0003] To mitigate the impact of irregular memory access computations on the performance of parallel processing units in SIMT architectures, the industry primarily employs software optimization methods. For example, data compression techniques are used to reorganize the data to be accessed, reducing memory footprint and improving data locality; or load balancing algorithms are used to evenly distribute irregular workloads across processing units, preventing overload in some units. However, these software optimization methods have several limitations. Firstly, techniques like data compression and load balancing require significant programming work. Programmers need a deep understanding of the parallel processing unit structure and algorithm execution flow to manually optimize the code, increasing development difficulty and cost, and reducing code maintainability and portability. Secondly, software optimization methods often rely on specific hardware architectures and data characteristics, making them difficult to apply universally in the field of irregular memory access computations. Summary of the Invention

[0004] To address the above problems, the present invention aims to provide a memory access reordering processing method, system, device, and medium based on SIMT architecture. By optimizing irregular memory access computing applications at the hardware level, and relatively decoupling irregular memory access optimization from software programming execution, it can improve the computing performance of parallel processing units in SIMT architecture while ensuring the portability of the project.

[0005] To achieve the above objectives, the present invention employs the following technical solution:

[0006] In a first aspect, embodiments of this application provide a memory access reordering method based on a SIMT architecture, including:

[0007] Configure a memory access reordering unit in the parallel processing unit pipeline based on the SIMT architecture, and build a cache reordering unit between cache unit levels; incrementally modify the decoding and execution modules in the parallel processing unit and extend the memory access optimization instructions and decoding logic; set the collaborative workflow of the memory access reordering unit and the cache reordering unit.

[0008] Construct the functional architecture of the cache reordering unit, configure data prefetching, classification, storage, ring interconnection and data reply functions to realize data prefetching, classified storage and on-demand feedback;

[0009] The software instruction kernel is divided into memory access regular kernel and memory access irregular kernel, and software and hardware processing mechanisms are configured for the two types of kernels. By configuring software-side parameters and hardware-side instructions for memory access irregular kernel, the parallel processing unit is controlled to dynamically reorder memory access requests at runtime, and logically adjacent address indices are concentrated and allocated to the same processing unit for execution.

[0010] In an optional implementation, configuring memory access reordering units in the parallel processing unit pipeline based on the SIMT architecture and constructing cache reordering units between cache unit levels includes:

[0011] The SIMT architecture embeds a memory reordering unit in the parallel processing unit pipeline, and configures an independent memory reordering unit for each parallel processing unit.

[0012] Within each cache unit, a cache reordering unit is constructed between the L1 and L2 caches. An independent cache reordering unit is configured for each memory partition. Cross-partition data collaboration is achieved by interconnecting the cache reordering units in a ring.

[0013] In an optional implementation, the incremental modification of the decoding and execution module and the expansion of memory access optimization instructions and decoding logic in the parallel processing unit include:

[0014] In the parallel processing unit, the decode module and the execution module are incrementally modified based on the built-in instruction fetch module, decode module, issue module, execution module and write-back module;

[0015] Extend the lru.config and lru.load directives in the decoding module and add corresponding decoding logic;

[0016] Add a memory access reordering unit to the execution module to execute extended lru.config and lru.load directives.

[0017] In an optional implementation, the collaborative workflow of setting the memory access reordering unit and the cache reordering unit includes:

[0018] When the execution module receives the lru.config instruction, it starts the memory access reordering unit and temporarily terminates the execution of the memory access loading module in the execution module. The memory access loading module sends the instruction operands to the cache reordering unit in the corresponding cache unit, and the cache reordering unit completes the data prefetching and reordering work.

[0019] When the execution module receives the lru.load instruction, the memory access reordering unit sends a data request to the cache unit, reads the reordered data from the cache reordering unit and stores it in the emit module of the parallel processing unit. After execution is completed, the execution of the memory access loading module is resumed.

[0020] In an optional implementation, the functional architecture of the cache reordering unit, configured with data prefetching, classification, storage, ring interconnection, and data response functions, realizes data prefetching, classified storage, and on-demand feedback, including:

[0021] The cache reordering unit includes a data prefetch module, a data classification module, a data storage module, a ring interconnect module, and a data recovery module.

[0022] When the cache reordering unit receives the lru.config instruction and its operands, it performs data prefetching according to the corresponding configuration information; based on the number of parallel processing units, the number of computational elements and the bit width, the memory space is segmented and the data of the corresponding memory segment is read in parallel by the data prefetching module.

[0023] The data classification module receives local prefetched data and interacts with other cache reordering units through the ring interconnection module, classifying the data according to the range of the calculated element values. Based on the classification results, it receives data suitable for local storage and forwards data unsuitable for local storage to other cache reordering units.

[0024] The data storage module determines the overflow signal count value of the computed elements according to the lru.config instruction; when the number of stored data elements reaches the set value, the overflow signal is triggered, the maximum value of the computed elements in the storage space is removed, and the maximum value is updated to the second highest value.

[0025] The ring interconnect module determines the temporary ring interconnect network based on the number and location of parallel processing units in the lru.config command, determines the route selection based on the size of the evicted computational elements, and sends them to the corresponding superior or inferior parallel processing units.

[0026] The data response module monitors the status of relevant modules. When the preset conditions are met, it enters the ready state. After receiving the lru.load instruction, it reads data from the data storage module and feeds it back to the parallel processing unit.

[0027] In an optional implementation, the process of dividing the software instruction kernel into memory-access-regular kernels and memory-access-irregular kernels, and configuring hardware and software processing mechanisms for the two types of kernels, includes:

[0028] The software instruction kernel is divided into memory access regular kernels and memory access irregular kernels;

[0029] For memory access rule-based kernels, the execution process and data flow graph of the original SIMT architecture's parallel processing unit hardware and software remain unchanged;

[0030] For kernels with irregular memory access, corresponding APIs are added in software, and corresponding execution units are added in hardware.

[0031] In an optional implementation, the step of configuring software-side parameters and hardware-side instructions for a memory-irregular kernel to control the parallel processing unit to dynamically reorder memory access requests at runtime, allocating logically adjacent address indices to the same processing unit for execution, includes:

[0032] For kernels with irregular memory access, configure lru.config information on the host side and lru.load instructions on the parallel processing unit side.

[0033] After configuration, the parallel processing unit is controlled to reorder memory access during execution, assigning similar address indices to parallel processing units under the same SIMT architecture for execution.

[0034] Secondly, embodiments of this application also provide a memory access reordering processing system based on a SIMT architecture, comprising:

[0035] The hardware architecture construction and collaborative workflow configuration module is used to configure memory access reordering units in the parallel processing unit pipeline based on SIMT architecture, and to build cache reordering units between cache unit levels; to incrementally modify the decoding and execution modules in the parallel processing unit and extend memory access optimization instructions and decoding logic; and to set the collaborative workflow of memory access reordering units and cache reordering units.

[0036] The cache reordering unit configuration module is used to build the functional architecture of the cache reordering unit, configure data prefetching, classification, storage, ring interconnection and data reply functions, and realize data prefetching, classified storage and on-demand feedback;

[0037] The software instruction configuration and interaction module is used to divide the software instruction kernel into memory access regular kernels and memory access irregular kernels, and to configure the software and hardware processing mechanisms for the two types of kernels. By configuring software-side parameters and hardware-side instructions for memory access irregular kernels, it controls the parallel processing unit to dynamically reorder memory access requests at runtime, and to allocate logically adjacent address indices to the same processing unit for execution.

[0038] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the memory access reordering processing method based on the SIMT architecture as described above.

[0039] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the steps of the memory access reordering processing method based on the SIMT architecture as described in any of the above embodiments.

[0040] As can be seen from the above technical solutions, the present invention has the following advantages:

[0041] The memory reordering method based on the SIMT architecture provided in this application firstly constructs a memory reordering unit (LRU) and a cache reordering unit (CRU), which are embedded in the parallel processing unit pipeline and cache unit of the SIMT architecture, respectively. These form a collaborative architecture with the parallel processing unit, L1 / L2 cache, memory controller, and on-chip network bus. Each parallel processing unit is configured with an independent memory reordering unit, and each memory partition is configured with an independent cache reordering unit. The cache reordering units achieve cross-partition data collaboration through ring interconnection, ensuring global memory reordering. Secondly, in the core unit CRU, this application designs a pipelined processing mode, including five modules: data prefetching module, data classification module, data storage module, ring interconnection module, and data recovery module. This pipelined approach realizes data memory access merging, improving execution efficiency. Finally, this application interacts with the software stack of the SIMT architecture parallel processing unit through two layers of software APIs, achieving functional extensions on the original instruction set, making memory reordering more accurate.

[0042] This application interacts with the traditional SIMT architecture parallel processing unit software stack through two layers of software APIs at the software stack level, achieving functional extensions on the original SIMT instruction set. At the hardware level, it constructs a memory reordering unit (LRU) and a cache reordering unit (CRU), which are embedded in the SIMT architecture parallel processing unit pipeline and cache unit, respectively. They form a collaborative architecture with the parallel processing unit, L1 / L2 cache, memory controller, and on-chip network bus, realizing cross-partition data collaboration and global memory reordering, which greatly improves the execution efficiency of the SIMT parallel processing unit for irregular memory access applications.

[0043] This application significantly improves the locality of memory access and effectively reduces cache miss rate by using a centralized allocation mechanism of logically adjacent address indexes, combined with data prefetching, classified storage and ring interconnection functions of the cache reordering unit. At the same time, the dynamic overflow signal mechanism and cross-partition data collaboration reduce invalid data occupation and cross-memory partition access latency, fundamentally optimizing the memory access efficiency and cache resource utilization of the parallel processing unit under the SIMT architecture, and greatly improving the overall throughput.

[0044] This application extends the lru.config and lru.load instructions and dedicated decoding logic, enabling the hardware to directly perceive and optimize irregular memory access patterns, reducing software intervention overhead. The pipeline dynamic adaptation mechanism (such as temporarily terminating the memory access loading module and reordering requests at runtime) avoids pipeline stalls caused by memory irregularities in traditional SIMT architectures. The ring interconnect network and on-demand feedback module further optimize on-chip bandwidth utilization, making the hardware execution flow highly matched with the memory access pattern.

[0045] This application targets kernels with irregular memory access. A software and hardware collaboration mechanism (such as the combination of API configuration and hardware instructions) enables runtime dynamic address reordering, solving the adaptation problem of traditional SIMT architecture for scenarios such as sparse computing and graph traversal. The routing selection and computation element classification strategy of the ring interconnect module ensures data load balancing, avoids performance bottlenecks, and enables the SIMT architecture to efficiently support key areas such as scientific computing and machine learning that rely on irregular memory access. The execution efficiency of specific algorithms is improved by orders of magnitude. Attached Figure Description

[0046] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating the memory access reordering method based on the SIMT architecture provided in this application.

[0048] Figure 2 A schematic diagram of the collaborative architecture of the memory access reordering unit based on the SIMT parallel processing unit provided in this application.

[0049] Figure 3 A schematic diagram of the hardware architecture and core modules of the cache reordering unit provided in this application.

[0050] Figure 4 This is a schematic diagram illustrating the process of embedding the memory access reordering API provided in this application.

[0051] Figure 5 This is a schematic diagram of the memory access reordering execution process provided in this application.

[0052] Figure 6 This is a schematic diagram of the memory access reordering processing system based on the SIMT architecture provided in this application.

[0053] Figure 7 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0054] The various embodiments of this disclosure will be described more fully in the detailed steps of the memory access reordering method based on the SIMT architecture described below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0055] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Please see Figure 1 The diagram shows a flowchart of a memory access reordering method based on a SIMT architecture in a specific embodiment. The method includes:

[0058] S1: Configure a memory access reordering unit in the parallel processing unit pipeline based on the SIMT architecture, and build a cache reordering unit between cache unit levels; incrementally modify the decoding and execution modules in the parallel processing unit and extend the memory access optimization instructions and decoding logic; set the collaborative workflow of the memory access reordering unit and the cache reordering unit.

[0059] In a specific implementation, an independent memory reordering unit (LRU) for each parallel processing unit is embedded in the parallel processing unit pipeline based on the SIMT architecture, and an independent cache reordering unit (CRU) is built between the L1 / L2 caches of the cache unit to achieve cross-partition data collaboration for each memory partition through a ring interconnect.

[0060] The existing five-stage pipeline of the parallel processing unit (including the instruction fetch module, decode module, issue module, execution module, and write-back module) is incrementally modified in its decode and execution modules. The decode module is expanded with two instructions, "lru.config" and "lru.load", and their corresponding decoding logic. In the execution module, a Memory Reordering Unit (LRU) is added to the existing integer computation module (ALU), floating-point computation module (FPU), special function computation module (SFU), and memory access loading module (LSU). When the execution module receives the "lru.config" instruction, it starts the LRU and temporarily terminates LSU execution. The LRU sends the instruction operands to the corresponding CRU to complete data prefetching and reordering. When it receives the "lru.load" instruction, the LRU module sends a data request to the cache unit, reads the reordered data from the CRU, stores it in the issue module, and resumes LSU execution after execution.

[0061] S2: Construct the functional architecture of the cache reordering unit, configure data prefetching, classification, storage, ring interconnection and data reply functions to realize data prefetching, classified storage and on-demand feedback.

[0062] In a specific implementation, the Cache Reordering Unit (CRU) is configured with five modules: data prefetching, data classification, data storage, ring interconnection, and data recovery.

[0063] Upon receiving the "lru.config" command and its operands, the memory space is segmented according to the configuration information. Each data prefetching module reads data from its corresponding memory segment in parallel. The data classification module receives the locally prefetched data and interacts with other CRUs through the ring interconnect module, classifying it according to the numerical range of the computed elements. Items suitable for local storage are received, while those unsuitable are forwarded. The data storage module adopts a direct-mapped multi-body structure, determining the overflow signal count value of the computed elements according to the command. When the set value is reached, the overflow signal is triggered, the maximum value is evicted, and the value is updated to the second highest value. The ring interconnect module determines the temporary ring interconnect network based on the number and location of parallel processing units in the command, determines the route selection according to the size of the evicted elements, and sends the data to the corresponding parallel processing unit. The data response module monitors the module status, enters the ready state when the conditions are met, and reads data from the data storage module after receiving the "lru.load" command, feeding it back to the parallel processing unit.

[0064] S3: Divide the software instruction kernel into memory access regular kernels and memory access irregular kernels, and configure software and hardware processing mechanisms for the two types of kernels; by configuring software-side parameters and hardware-side instructions for memory access irregular kernels, control the parallel processing unit to dynamically reorder memory access requests at runtime, and allocate logically adjacent address indices to the same processing unit for execution.

[0065] In specific implementations, the software instruction kernel is divided into regular and irregular memory access types. The regular memory access type maintains the original SIMT architecture parallel processing unit software and hardware execution process and data flow graph unchanged, while the irregular memory access type adds corresponding APIs in software and corresponding execution units in hardware.

[0066] When upper-level software developers write code, they distinguish between two types of kernels. For kernels with no memory access rules, they configure "lru.config" information on the kernel side and "lru.load" instructions on the parallel processing unit side. This enables the SIMT parallel processing unit configured to execute LRU to reorder memory accesses during execution, so that indexes of nearby addresses are executed by the same parallel processing unit, thereby improving memory locality utilization.

[0067] In this embodiment, a two-level hardware-level reordering architecture is formed by embedding a memory reordering unit within the parallel processing unit pipeline and constructing a ring-connected cache reordering unit between cache levels. The lru.config and lru.load instructions are extended to achieve hardware-software co-configuration, enabling the cache reordering unit to prefetch memory segments, classify and store data, and interact across partitions based on instruction parameters, dynamically adjusting the data layout in conjunction with an overflow eviction mechanism. Simultaneously, a differentiated processing mechanism is designed for regular and irregular memory access kernels. For irregular kernels, hardware-software collaboration at runtime centrally allocates logically adjacent address indices to the same processing unit, achieving dynamic reordering of memory access requests. This design effectively enhances the locality of data access, improves cache utilization, and reduces memory access latency. While maintaining the original execution efficiency of regular memory access kernels, it significantly optimizes the parallel processing capabilities in irregular memory access scenarios, resulting in a synergistic improvement in overall system memory access efficiency and resource utilization.

[0068] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0069] The memory reordering unit collaborative architecture based on SIMT parallel processing units is as follows: Figure 2 As shown, in this step, a Memory Reordering Unit (LRU) and a Cache Reordering Unit (CRU) are constructed, embedded in the parallel processing unit pipeline and cache unit of the SIMT architecture, respectively. They form a collaborative architecture with the parallel processing unit, L1 / L2 cache, memory controller, and on-chip network bus. Each parallel processing unit is configured with an independent Memory Reordering Unit, and each memory partition is configured with an independent Cache Reordering Unit. The Cache Reordering Units achieve cross-partition data collaboration through a ring interconnect, ensuring global memory reordering. The workflow of each unit module is shown below:

[0070] First, within the parallel processing unit, the decoding and execution modules were incrementally modified based on the original five-stage pipeline of "fetch-decode-issue-execute-write-back". In the decoding module, the instructions "lru.config" and "lru.load" were extended, and corresponding decoding logic was added to ensure correct instruction execution. In the execution module, a Memory Reordering Unit (LRU) was added to the original integer computation module (ALU), floating-point computation module (FPU), special function computation module (SFU), and memory access load module (LSU), corresponding to the extended "lru.config" and "lru.load" instructions in the decoding module. When the execution module receives the "lru.config" instruction from the decoding module, it starts the LRU and temporarily terminates the LSU execution. The LRU sends the operands of the "lru.config" instruction to the cache reordering unit (CRU) in the corresponding cache unit, where the CRU performs data prefetching and reordering based on the operands. When the execution module receives the "lru.load" instruction from the decoding module, the LRU sends a data request to the cache unit, reads the reordered data from the CRU, and stores it in the issue module in the parallel processing unit. After the corresponding "lru.load" instruction is executed, the execution module starts the LSU and terminates the LRU execution. In this step, the "lru.config" and "lru.load" instructions appear in pairs and are executed sequentially.

[0071] Secondly, within the cache unit, this step incrementally constructs a cache reordering unit (CRU) between the existing L1 / L2 caches. The cache reordering unit is responsible for receiving instructions from the LRUs of the parallel processing unit, reading data from the L2 cache, and communicating with the CRUs in all cache units to achieve data reordering across the entire cache unit. Subsequently, when an LRU in the parallel processing unit sends a data request, the corresponding CRU in the cache unit accepts the request and feeds back the reordered data to the parallel processing module. Within the cache unit, the L1 cache and the cache reordering unit are parallel and independent, corresponding to the LSU memory access unit and LRU memory access unit in the parallel processing module, respectively. Only one module is allowed to work at a time, and the data path to the parallel processing unit and L2 cache is time-division multiplexed. Furthermore, to ensure accurate correspondence of the reordered data, the cache partitions formed by the L1 cache and the cache reordering unit are mapped one-to-one with the parallel processing units.

[0072] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0073] The specific architecture of the cache reordering unit in the parallel processing unit cache module of the SIMT architecture is as follows: Figure 3 As shown, it is divided into five main modules: data prefetching module, data classification module, data storage module, ring interconnection module, and data recovery module. It uses a pipelined approach to process data access and merging. The specific working process is as follows:

[0074] First, the cache reordering unit is initiated by the LRU in the corresponding parallel processing unit sending the "lru.config" configuration. The operands accompanying the "lru.config" start instruction include the starting address of the memory access, the number of elements to be computed in the processing unit, the bit width of the computed elements, and the number of parallel processing units to execute. After receiving the "lru.config" instruction and its information, the cache reordering unit processes the configuration instruction and prefetches data according to the processing result. The specific instruction configuration process is as follows:

[0075] Suppose that during this startup, 16 parallel processing units need to be invoked to execute computation tasks, and address 0x00008000 is set as the starting address for memory access operations. Each parallel processing unit needs to process 32 computational elements, and each computational element has a bit width of 8 bits. Under this configuration, the instruction configuration processing mechanism establishes a specific mapping and operation. On one hand, the 16 parallel processing units will form a one-to-one correspondence with 16 cache reordering units. Each cache reordering unit undertakes an independent data prefetching task, and the length of the prefetched data is 32 computational elements multiplied by the 8-bit bit width of each element, i.e., 256 bits. On the other hand, in terms of memory space usage, starting from address 0x00008000, the memory space will be segmented according to the length of the prefetched data (32x8) of each cache reordering unit, dividing the entire memory space into 16 segments, and these 16 memory segments correspond one-to-one with the 16 cache reordering units. During the data prefetching phase, the data prefetching modules in the 16 cache reordering units work in parallel, each responsible for reading the data in its corresponding memory space.

[0076] After the data prefetch module finishes reading data from the L2 cache, the read data is sent to the data classification module. The data classification module receives prefetched data from the data prefetch module within its own cache reordering unit. Simultaneously, it interacts with other cache reordering modules via the ring interconnect module, sending out prefetched data unsuitable for local storage through the ring interconnect module and receiving prefetched data suitable for local storage from the ring interconnect module. The specific data classification process is as follows:

[0077] Using the example of a startup requiring 16 parallel processing units to execute computation tasks, each processing 32 computational elements, and each computational element having an 8-bit width, the data classification process is illustrated. After startup, the data classification module first determines the representable data range based on the computational element's bit width. In this example, an 8-bit wide computational element can represent decimal data ranging from 0 to 255. Subsequently, the data classification module initially groups the computational elements according to the number of participating parallel processing units and sets corresponding thresholds. With 16 parallel processing units participating in the computation, the numerical range of the computational elements is evenly divided into 16 groups, specifically 0-15, 16-31, etc., with each group having the same numerical span. Then, the initial numerical ranges of these 16 computational elements are respectively configured into 16 cache reordering units, establishing a clear mapping relationship. After completing the initial configuration, the data classification module begins receiving data from the data prefetching module in the local cache reordering unit. For each received computational element, the data classification module checks whether its value is within the currently configured numerical range of the computational element. If the range requirement is met, the computed element is sent to the data storage module of the local cache reordering unit for storage. If not, the data is forwarded to another cache reordering unit via the ring interconnect module to ensure that the data is allocated to a suitable storage location. Finally, when the data storage module of the local cache reordering unit is full of 32 computed elements to be computed according to the pre-set initial value range, if there is still data that meets the value range that needs to be written, the stored data needs to be adjusted. To ensure the validity and rationality of the stored data, the data storage module will evict the computed element with the largest value and update the maximum value of the computed element's value range to the second highest value among the currently maintained computed elements, and continue to make subsequent adjustments until the data storage modules in all 16 memory access reordering units have stored 32 computed elements.

[0078] The data storage module is a storage unit within the cache reordering unit. It adopts a direct-mapped multi-body structure and features configurable storage depth, a computational element overflow signal, and a data write eviction mechanism. Specifically, the data storage module determines the computational element overflow signal count value based on the number of computational elements in the processing unit as specified in the "lru.config" command. When the number of data elements stored in this data storage module reaches the number of computational elements in the processing unit, it triggers the computational element overflow signal, evictions the maximum value of the computational elements in the storage space, and reads the second-highest value of the computational elements in the storage space to feed back to the data classification module.

[0079] The ring interconnect module is the interaction module within the cache reordering unit. It features configurable mappings and routing options, connecting various cache reordering units. Specifically, the ring interconnect module is responsible for two aspects: firstly, determining the temporary ring interconnect network based on the number and location of parallel processing units specified in the "lru.config" command, ensuring that only the parallel processing units configured for computation are within this network, while other processing units remain outside. Secondly, it is responsible for determining routing based on the size of evicted computational elements. If the value of an evicted computational element is greater than the range of local computational elements, it is sent to the next-level parallel processing unit; otherwise, it is sent to the next-level parallel processing unit.

[0080] The data response module, located within the cache reordering unit, is responsible for receiving the "lru.load" instruction from the LRU module in the parallel processing unit, reading data from the data storage module, and feeding it back to the parallel processing unit. Specifically, the data response module constantly monitors the status of the data prefetch module, the ring interconnect module, and the data storage module. When the data storage module is overflowing, the data prefetch module is empty, and the ring interconnect module has no data interaction, the data response module enters a ready state and can read data from the data storage module and feed it back to the corresponding parallel processing unit.

[0081] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0082] To achieve more accurate memory access reordering, this step interacts with the traditional SIMT architecture's parallel processing unit software stack through two layers of software APIs, extending the functionality of the original SIMT instruction set. The specific architecture is as follows: Figure 4 and Figure 5 As shown.

[0083] First, the software instruction kernel is divided into two types: one is a memory-access-regular kernel (such as kernel(2)) and the other is a memory-access-irregular kernel (such as kernel(1)). For memory-access-regular kernels, the existing SIMT architecture parallel processing unit software and hardware are sufficient to solve the problem with high efficiency, so the data flow graph and software and hardware execution process are not changed; for memory-access-irregular kernels, the existing SIMT architecture parallel processing unit software and hardware are insufficient to solve the problem with high efficiency, so the software adds corresponding APIs and the hardware adds corresponding execution units.

[0084] Specifically, in this step, such as Figure 4As shown, when writing code, upper-level software developers only need to distinguish between memory-access-regular and memory-access-irregular kernels, and configure the lru.config information of the memory-access-irregular kernel on the host side and the lru.load on the parallel processing unit side. They do not need a detailed understanding of the SIMT architecture's parallel processing unit hardware. For a SIMT parallel processing unit without LRU configuration, its execution process is as follows: Figure 5 As shown, each parallel processing unit reads data from the high-level cache according to the address index in its memory access instruction, and does not reorder the data relationships based on the address indexes. However, for SIMT parallel processing units configured for LRU execution, there is a reordering process during execution. The address indexes in their memory access instructions are reordered, and then nearby address indexes are executed by the same SIMT architecture parallel processing unit, improving the utilization of memory locality.

[0085] like Figure 6 As shown, the following are embodiments of the memory access reordering processing system based on the SIMT architecture provided in this disclosure. This system and the memory access reordering processing method based on the SIMT architecture in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the memory access reordering processing system based on the SIMT architecture, please refer to the embodiments of the memory access reordering processing method based on the SIMT architecture described above.

[0086] A memory access reordering processing system based on SIMT architecture includes: a hardware architecture construction and collaborative workflow configuration module, a cache reordering unit configuration module, and a software instruction configuration and interaction module.

[0087] The hardware architecture construction and collaborative workflow configuration module is used to configure memory access reordering units in the parallel processing unit pipeline based on SIMT architecture, and to build cache reordering units between cache unit levels; to incrementally modify the decoding and execution modules in the parallel processing unit and extend memory access optimization instructions and decoding logic; and to set the collaborative workflow of memory access reordering units and cache reordering units.

[0088] The cache reordering unit configuration module is used to build the functional architecture of the cache reordering unit, configure data prefetching, classification, storage, ring interconnection and data reply functions, and realize data prefetching, classified storage and on-demand feedback.

[0089] The software instruction configuration and interaction module is used to divide the software instruction kernel into memory access regular kernels and memory access irregular kernels, and to configure the software and hardware processing mechanisms for the two types of kernels. By configuring software-side parameters and hardware-side instructions for memory access irregular kernels, it controls the parallel processing unit to dynamically reorder memory access requests at runtime, and to allocate logically adjacent address indices to the same processing unit for execution.

[0090] The memory access reordering processing system based on the SIMT architecture provided in this embodiment, by configuring a memory access reordering unit and a cache reordering unit and setting a collaborative workflow, combined with incremental modification decoding and execution modules, extended memory access optimization instructions, and classification of software instruction kernels and configuration of corresponding hardware and software processing mechanisms, can effectively improve memory access efficiency, optimize data locality, enhance the ability to handle irregular memory access requests, realize runtime dynamic reordering of memory access requests, and centrally allocate logically adjacent address indices to the same processing unit for execution, thereby improving the resource utilization and data processing efficiency of parallel processing units, reducing memory access latency, and improving the overall system performance.

[0091] Figure 7 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0092] The memory reordering method based on the SIMT architecture provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the electronic device includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0093] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0094] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0095] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0096] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0097] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0098] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0099] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0100] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0101] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0102] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0103] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0104] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0105] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0106] The aforementioned electronic device implements the memory access reordering processing method based on the SIMT architecture of this application. By configuring memory access reordering units in the parallel processing unit pipeline of the SIMT architecture, constructing a ring-connected cache reordering unit between cache levels, and extending the lru.config and lru.load instructions to achieve hardware and software collaboration, it combines segmented prefetching, classified storage, and cross-partition data interaction mechanisms for memory space, and designs differentiated processing strategies for regular and irregular memory access kernels. This achieves the beneficial effects of enhancing data access locality, improving cache utilization, reducing memory access latency, and significantly optimizing parallel processing capabilities in irregular memory access scenarios.

[0107] The storage medium provided in this application stores a program product capable of implementing a memory access reordering processing method based on the SIMT architecture.

[0108] The memory access reordering methods based on the SIMT architecture include:

[0109] Configure a memory access reordering unit in the parallel processing unit pipeline based on the SIMT architecture, and build a cache reordering unit between cache unit levels; incrementally modify the decoding and execution modules in the parallel processing unit and extend the memory access optimization instructions and decoding logic; set the collaborative workflow of the memory access reordering unit and the cache reordering unit.

[0110] Construct the functional architecture of the cache reordering unit, configure data prefetching, classification, storage, ring interconnection and data reply functions to realize data prefetching, classified storage and on-demand feedback;

[0111] The software instruction kernel is divided into memory access regular kernel and memory access irregular kernel, and software and hardware processing mechanisms are configured for the two types of kernels. By configuring software-side parameters and hardware-side instructions for memory access irregular kernel, the parallel processing unit is controlled to dynamically reorder memory access requests at runtime, and logically adjacent address indices are concentrated and allocated to the same processing unit for execution.

[0112] In some possible implementations, the memory reordering processing method based on the SIMT architecture of this disclosure can be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0113] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0114] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A memory access reordering method based on SIMT architecture, characterized in that, include: Configure a memory access reordering unit in the parallel processing unit pipeline based on the SIMT architecture, and build a cache reordering unit between cache unit levels; incrementally modify the decoding and execution modules in the parallel processing unit and extend the memory access optimization instructions and decoding logic; set the collaborative workflow of the memory access reordering unit and the cache reordering unit. Construct the functional architecture of the cache reordering unit, configure data prefetching, classification, storage, ring interconnection and data reply functions to realize data prefetching, classified storage and on-demand feedback; The software instruction kernel is divided into memory access regular kernel and memory access irregular kernel, and software and hardware processing mechanisms are configured for the two types of kernels. By configuring software-side parameters and hardware-side instructions for memory access irregular kernel, the parallel processing unit is controlled to dynamically reorder memory access requests at runtime, and logically adjacent address indices are concentrated and allocated to the same processing unit for execution. The incremental modification of the decoding and execution module in the parallel processing unit and the extension of memory access optimization instructions and decoding logic include: In the parallel processing unit, the decode module and the execution module are incrementally modified based on the built-in instruction fetch module, decode module, issue module, execution module and write-back module; Extend the lru.config and lru.load directives in the decoding module and add corresponding decoding logic; Add a memory access reordering unit to the execution module to execute extended lru.config and lru.load instructions; The collaborative workflow of the memory access reordering unit and the cache reordering unit includes: When the execution module receives the lru.config instruction, it starts the memory access reordering unit and temporarily terminates the execution of the memory access loading module in the execution module. The memory access loading module sends the instruction operands to the cache reordering unit in the corresponding cache unit, and the cache reordering unit completes the data prefetching and reordering work. When the execution module receives the lru.load instruction, the memory access reordering unit sends a data request to the cache unit, reads the reordered data from the cache reordering unit and stores it in the emit module of the parallel processing unit. After execution is completed, the execution of the memory access loading module is resumed.

2. The memory access reordering method based on SIMT architecture according to claim 1, characterized in that, The configuration of memory access reordering units in the parallel processing unit pipeline based on the SIMT architecture, and the construction of cache reordering units between cache unit levels, include: The SIMT architecture embeds a memory reordering unit in the parallel processing unit pipeline, and configures an independent memory reordering unit for each parallel processing unit. Within each cache unit, a cache reordering unit is constructed between the L1 and L2 caches. An independent cache reordering unit is configured for each memory partition. Cross-partition data collaboration is achieved by interconnecting the cache reordering units in a ring.

3. The memory access reordering method based on SIMT architecture according to claim 1, characterized in that, The functional architecture for constructing the cache reordering unit is configured with data prefetching, classification, storage, ring interconnection, and data retrieval functions to achieve data prefetching, classified storage, and on-demand feedback, including: The cache reordering unit includes a data prefetch module, a data classification module, a data storage module, a ring interconnect module, and a data recovery module. When the cache reordering unit receives the lru.config instruction and its operands, it performs data prefetching according to the corresponding configuration information; based on the number of parallel processing units, the number of computational elements and the bit width, the memory space is segmented and the data of the corresponding memory segment is read in parallel by the data prefetching module. The data classification module receives local prefetched data and interacts with other cache reordering units through the ring interconnection module, classifying the data according to the range of the calculated element values. Based on the classification results, it receives data suitable for local storage and forwards data unsuitable for local storage to other cache reordering units. The data storage module determines the overflow signal count value of the computed elements according to the lru.config instruction; when the number of stored data elements reaches the set value, the overflow signal is triggered, the maximum value of the computed elements in the storage space is removed, and the maximum value is updated to the second highest value. The ring interconnect module determines the temporary ring interconnect network based on the number and location of parallel processing units in the lru.config command, determines the route selection based on the size of the evicted computational elements, and sends them to the corresponding superior or inferior parallel processing units. The data response module monitors the status of relevant modules. When the preset conditions are met, it enters the ready state. After receiving the lru.load instruction, it reads data from the data storage module and feeds it back to the parallel processing unit.

4. The memory access reordering method based on SIMT architecture according to claim 3, characterized in that, The software instruction kernel is divided into memory-access-regular kernels and memory-access-irregular kernels, and hardware and software processing mechanisms are configured for the two types of kernels, including: The software instruction kernel is divided into memory access regular kernels and memory access irregular kernels; For memory access rule-based kernels, the execution process and data flow graph of the original SIMT architecture's parallel processing unit hardware and software remain unchanged; For kernels with irregular memory access, corresponding APIs are added in software, and corresponding execution units are added in hardware.

5. The memory access reordering method based on SIMT architecture according to claim 4, characterized in that, The method of configuring software-side parameters and hardware-side instructions for memory-irregular kernels to control parallel processing units to dynamically reorder memory access requests at runtime, and allocating logically adjacent address indices to the same processing unit for execution, includes: For kernels with irregular memory access, configure lru.config information on the host side and lru.load instructions on the parallel processing unit side; After configuration, the parallel processing unit is controlled to reorder memory access during execution, assigning similar address indices to parallel processing units under the same SIMT architecture for execution.

6. A memory access reordering processing system based on SIMT architecture, characterized in that, The system employs the memory access reordering method based on the SIMT architecture as described in any one of claims 1 to 5; The system includes: The hardware architecture construction and collaborative workflow configuration module is used to configure memory access reordering units in the parallel processing unit pipeline based on SIMT architecture, and to build cache reordering units between cache unit levels; to incrementally modify the decoding and execution modules in the parallel processing unit and extend memory access optimization instructions and decoding logic; and to set the collaborative workflow of memory access reordering units and cache reordering units. The cache reordering unit configuration module is used to build the functional architecture of the cache reordering unit, configure data prefetching, classification, storage, ring interconnection and data reply functions, and realize data prefetching, classified storage and on-demand feedback; The software instruction configuration and interaction module is used to divide the software instruction kernel into memory access regular kernels and memory access irregular kernels, and configure the software and hardware processing mechanisms for the two types of kernels. By configuring software-side parameters and hardware-side instructions for memory access irregular kernels, it controls the parallel processing unit to dynamically reorder memory access requests at runtime, and allocates logically adjacent address indices to the same processing unit for execution. The incremental modification of the decoding and execution module in the parallel processing unit and the extension of memory access optimization instructions and decoding logic include: In the parallel processing unit, the decode module and the execution module are incrementally modified based on the built-in instruction fetch module, decode module, issue module, execution module and write-back module; Extend the lru.config and lru.load directives in the decoding module and add corresponding decoding logic; Add a memory access reordering unit to the execution module to execute extended lru.config and lru.load instructions; The collaborative workflow of the memory access reordering unit and the cache reordering unit includes: When the execution module receives the lru.config instruction, it starts the memory access reordering unit and temporarily terminates the execution of the memory access loading module in the execution module. The memory access loading module sends the instruction operands to the cache reordering unit in the corresponding cache unit, and the cache reordering unit completes the data prefetching and reordering work. When the execution module receives the lru.load instruction, the memory access reordering unit sends a data request to the cache unit, reads the reordered data from the cache reordering unit and stores it in the emit module of the parallel processing unit. After execution is completed, the execution of the memory access loading module is resumed.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the memory access reordering processing method based on the SIMT architecture as described in any one of claims 1 to 5.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the memory access reordering processing method based on the SIMT architecture as described in any one of claims 1 to 5.

Images