Method of loading data to a vector register, processor, device, medium

Abstract

A method, a processor, an electronic device, and a non-transitory storage medium for loading data into vector registers using a gather instruction are provided. The method includes loading data in a memory into a first vector register via a first set of load operations of a plurality of load operations of the gather instruction; loading data in the memory into a second vector register via a second set of load operations of the plurality of load operations of the gather instruction, wherein the second vector register is different from the first vector register, wherein a last load operation of the first set of load operations and a first load operation of the second set of load operations are adjacent in the plurality of load operations of the gather instruction; and merging data in the first vector register and data in the second vector register into one vector register.

Description

Technical Field

[0001] This application relates to the field of integrated circuits, and more specifically to a method for loading data into a vector register using the Gather instruction, a processor, an electronic device, and a non-temporary storage medium. Background Technology

[0002] An instruction set is a collection of instructions used by the CPU to calculate and control the computer system. Instruction sets are generally divided into Reduced Instruction Set Computer (RISC) and Complex Instruction Set Computer (CISC).

[0003] Modern processors include vector processing units, which are crucial components for data-parallel computation. Single Instruction Multiple Data (SIMD) refers to a single-instruction multiple-data technique that allows a single set of instructions to perform parallel operations on multiple sets of data. The core of the processor's vector unit is its supported SIMD instruction set (also known as floating-point instruction set or vector instruction set) and vector registers. SIMD instructions can control multiple parallel processing elements simultaneously on a single controller, executing multiple data streams in a single instruction operation, thus improving program speed. SIMD instructions can simultaneously perform the same operation on a set of data (also called a "data vector") on the controller, achieving spatial parallelism. SIMD helps the Central Processing Unit (CPU) achieve data-level parallelism (DLP).

[0004] Vector registers, also known as floating-point registers, can store multiple elements compared to ordinary general-purpose registers and are the core storage component for running SIMD instruction sets. Unlike scalar registers, which can only store one set of data, vector registers can store multiple sets of data (integers or floating-point numbers), and their width may be 128 / 256 / 512 bits depending on the specific implementation. A single SIMD instruction can operate on multiple elements stored in a vector register simultaneously. For example, the AVX2 instruction set is used in modern processors. This instruction set provides load instructions to load multiple consecutive values ​​stored in memory into a vector register at once, and permute instructions to dynamically reconstruct the elements in the vector register.

[0005] In the field of Digital Signal Processing (DSP) design, Gather / Scatter instructions can be used by programs. The Gather instruction can load a set of data from different locations in memory into a register, while the Scatter instruction can write data from a register to different locations in memory.

[0006] In AVX2 memory access support, each SIMD data vector element can come from non-contiguous memory addresses. The Gather instruction in AVX2 has a base address and a vector register as operands. The vector register stores the offset (displacement) of each element in the SIMD data relative to the base address. The CPU can "aggregate" several non-contiguous data items into a single SIMD register. The Scatter instruction can "scatter" the data in the register to different locations in memory. Both Gather and Scatter instructions are complex instructions and are broken down into many micro-operations to execute separate load and write-back operations.

[0007] To shorten the execution time of the Gather instruction, it is necessary to increase the parallelism between the micro-operations of the Gather instruction. Summary of the Invention

[0008] According to one aspect of this application, a method for loading data into a vector register using the Gather instruction is provided, comprising: loading data in memory into a first vector register through a first set of load operations in a plurality of load operations of the Gather instruction; loading data in memory into a second vector register through a second set of load operations in a plurality of load operations of the Gather instruction, wherein the second vector register is different from the first vector register, wherein the last load operation of the first set of load operations and the first load operation of the second set of load operations are adjacent in the plurality of load operations of the Gather instruction; and merging the data in the first vector register and the data in the second vector register into a single vector register.

[0009] According to another aspect of this application, a processor is provided that loads data into a vector register using the Gather instruction, comprising: a loader configured to load data in memory into a first vector register via a first set of load operations in a plurality of load operations of the Gather instruction; and to load data in memory into a second vector register via a second set of load operations in a plurality of load operations of the Gather instruction, wherein the second vector register is different from the first vector register, and wherein the last load operation of the first set of load operations and the first load operation of the second set of load operations are adjacent in the plurality of load operations of the Gather instruction; and a merger configured to merge the data in the first vector register and the data in the second vector register into a single vector register.

[0010] According to another aspect of this application, a method for loading data into a register is provided, comprising: loading data in memory into a first register through a first group of load operations in a plurality of load operations; loading data in memory into a second register through a second group of load operations in a plurality of load operations, wherein the second register is different from the first register, wherein the last load operation of the first group of load operations and the first load operation of the second group of load operations are adjacent in the plurality of load operations; and merging the data in the first register and the data in the second register into a single register.

[0011] According to another aspect of this application, a processor for loading data into registers is provided, comprising: a loader configured to load data in memory into a first register via a first set of load operations in a plurality of load operations; and to load data in memory into a second register via a second set of load operations in a plurality of load operations, wherein the second register is different from the first register, and wherein the last load operation of the first set of load operations and the first load operation of the second set of load operations are adjacent in the plurality of load operations; and a merger configured to merge the data in the first register and the data in the second register into a single register.

[0012] According to another aspect of this application, an electronic device is provided, comprising: a memory for storing instructions; and a processor for reading the instructions from the memory and performing methods according to various embodiments of the application.

[0013] According to another aspect of this application, a non-temporary storage medium is provided, on which instructions are stored, wherein, when read by a processor, the instructions cause the processor to perform methods according to various embodiments of the application. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This illustrates the operation of the traditional Gather and Scatter instructions in registers.

[0016] Figure 2 The execution process of a traditional SIMD addition instruction is shown.

[0017] Figure 3 The flowchart for the execution of the traditional Gather command is shown.

[0018] Figure 4 This diagram illustrates a flowchart of how the traditional Gather command loads data multiple times.

[0019] Figure 5 A flowchart illustrating a method for loading data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0020] Figure 6 A schematic diagram illustrating an example of a method for loading data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0021] Figure 7 A schematic diagram illustrating another example of a method for loading data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0022] Figure 8 A flowchart of a method 800 for loading data into a register according to an embodiment of this application is shown.

[0023] Figure 9 A block diagram of a processor that loads data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0024] Figure 10 A block diagram of a processor that loads data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0025] Figure 11 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present application is shown.

[0026] Figure 12A schematic diagram of a non-transitory readable storage medium according to an embodiment of the present disclosure is shown. Detailed Implementation

[0027] Specific embodiments of this application will now be described in detail, with examples of the application illustrated in the accompanying drawings. Although this application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit this application to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of this application as defined by the appended claims. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.

[0028] Figure 1 This illustrates the operation of the traditional Gather and Scatter instructions in registers.

[0029] like Figure 1 As shown, the Gather instruction can gather data scattered in memory, such as addresses 0, 2, 4, and 6, into consecutive addresses in a register, such as 0, 1, 2, and 3. The Scatter instruction can scatter data in a register, such as addresses 0, 1, 2, and 3, into memory, such as addresses 0, 2, 4, and 6.

[0030] When executing SIMD instructions, the CPU stores multiple sets of data in a vector register and operates on multiple sets of data simultaneously using the vector register. Figure 2 This illustrates the execution process of a traditional SIMD addition instruction. For example... Figure 2 As shown, x and y are both vector registers, and each vector register contains 4 sets of data (A, B, C, D). The addition instruction of SIMD can be used to obtain the result of adding these 4 sets of data respectively (Ax+Ay, Bx+By, Cx+Cy, Dx+Dy).

[0031] The earliest proposed Scale, Index, Base (SIB) address indexing mode (which consists of several inputs for calculating the virtual address accessed by memory) used scalar registers (64-bit wide registers that store fixed-point data (i.e., integers), also known as fixed-point registers) to store index values. Later, the Advanced Vector Extensions 2 (AVX2) instruction set introduced a new Scale, Vector Index, Base (VSIB) address indexing mode (the difference from SIB is that the index is stored in a vector register), using a vector register to store multiple index values. Compared to the original instruction, which could only provide one index value and calculate one virtual address, the VSIB address indexing mode can calculate multiple virtual addresses using multiple index values, and VSIB address indexing instructions support accessing memory using multiple virtual addresses.

[0032] Currently, only the Gather and scatter instructions use the VSIB address indexing mode. The Gather instruction reads data from multiple SIB addresses and places the read results into a vector register in sequence, while the scatter instruction writes the values ​​in the vector register into multiple SIB addresses in sequence.

[0033] Figure 3 The flowchart illustrating the execution of a traditional Gather command is shown. Figure 3 As shown, the Gather instruction retrieves each index value from the vector register, calculates the address corresponding to each index, and loads the data corresponding to each address into the vector register.

[0034] However, in x86 vector instructions, only the vector portion of the data (e.g., for...) is processed. Figure 2 Instructions that modify only a portion of Ax, Bx, Cx, and Dx in the vector register are not frequently used. Furthermore, considering the complexity of physical circuit design, in practical circuit design, there are no operations that only write to a part of the vector register. The Gather instruction, however, loads data multiple times and saves it to different locations in the same vector register. Therefore, each load requires waiting for the previous load to write its result back to the vector register, reading the value from the vector register, merging it with the currently loaded data, and then writing it back to the vector register. This creates dependencies between each load in the Gather instruction, impacting its execution speed.

[0035] Figure 4 This diagram illustrates a flowchart of how the traditional Gather command loads data multiple times.

[0036] The Gather instruction loads data multiple times and saves it to different locations in the same vector register. Therefore, each load operation needs to wait for the previous load to write its result back to the vector register, then read the value of the vector register, merge it with the data being loaded, and then write it back to the vector register. Thus, there is a dependency between any two adjacent load operations.

[0037] This application proposes an optimization scheme that groups multiple load operations. Load operations within the same group can depend on each other, while load operations in different groups are independent. Ultimately, the results stored in these temporary vector registers are merged and written back to the instruction-level defined vector register. The grouping method can be two load operations per group, four load operations per group, or other granularities. For example, in the example of two load operations per group, even-numbered loads can depend on the results of odd-numbered loads, meaning that every two loads are written back to the same temporary vector register, creating mutual dependence. After all dependency chains have been executed, a micro-operation (UOP, where the CPU determines which operation to execute based on the micro-operation) is used to write the results from multiple temporary vector registers back to a single temporary vector register.

[0038] Because each dependency chain requires a temporary vector register to store the loaded data (using the same register would cause two consecutive load instructions to write to the same register, resulting in the later load instruction directly overwriting the value of the previous load instruction), splitting into n dependency chains would use n temporary vector registers (n is a positive integer greater than 1). Therefore, when the running program has a tight demand for vector registers, it is necessary to adjust the optimization scheme for Gather instructions. For example, instead of the original two adjacent load instructions forming a dependency chain, every four load instructions can form a dependency chain, sacrificing some parallelism in the execution of Gather instructions. However, by reducing the use of vector registers, the blocking caused by insufficient vector registers for non-Gather instructions can be reduced, thereby improving the overall performance of the program.

[0039] Splitting dependency chains into more chains leads to the use of more temporary registers, more micro-operations, and merging of the final result. The performance improvement brought by splitting dependency chains is not a function that increases linearly with the number of split dependency chains. Therefore, when implementing it, it is necessary to comprehensively consider the cost of increasing the number of temporary registers used, the number of micro-operations for merging data, and the performance improvement brought by increasing parallelism in order to determine how many different dependency chains to split into for parallel execution.

[0040] One scenario where compilers heavily utilize the Gather instruction is for regular array access, where the difference between the address loaded by Gather and the address loaded previously is a fixed value or the difference changes in a predictable pattern. Therefore, after a period of training, the processor's prefetcher can easily predict which address will be loaded next when it encounters a Gather instruction again. The prefetcher can issue a prefetch in advance during the current loading phase, fetching the data to be loaded next into the L1 Data Cache. When the Gather instruction executes the next load, the required data is already in the L1 Data Cache, significantly reducing the time spent loading data into the data return register each time. Thus, in this case, the bottleneck within the Gather instruction is mainly the dependency between multiple load operations rather than the latency of the load operation itself, thereby proving the effectiveness of the optimization in this patent.

[0041] The various embodiments of this application will now be described in detail.

[0042] Figure 5 A flowchart of a method 500 for loading data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0043] like Figure 5 As shown, a method 500 for loading data into a vector register using the Gather instruction includes: step 510, loading data from memory into a first vector register using a first group of load operations in a plurality of load operations of the Gather instruction; step 520, loading data from memory into a second vector register using a second group of load operations in a plurality of load operations of the Gather instruction, wherein the second vector register is different from the first vector register, and the last load operation of the first group of load operations and the first load operation of the second group of load operations are adjacent in the plurality of load operations of the Gather instruction; step 530, merging the data in the first vector register and the data in the second vector register into a single vector register.

[0044] Therefore, to address the dependencies within the micro-operations of the Gather instruction, a micro-operation implementation scheme that splits the dependency chain is proposed. By loading the first group of load operations (which are mutually dependent) from the multiple load operations of the Gather instruction into the first vector register, and by loading the second group of load operations (which are mutually dependent) into a second vector register different from the first vector register, the dependency between the first and second groups of load operations (at least the last load operation of the first group and the first load operation of the second group) is broken. This objectively enables the first and second groups of load operations (at least the last load operation of the first group and the first load operation of the second group) to be executed in parallel, improving the parallelism of the micro-operations within the Gather instruction, thereby greatly shortening the execution time of the Gather instruction and ultimately improving processor performance.

[0045] Figure 6 A schematic diagram illustrating an example of a method for loading data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0046] like Figure 6 As shown on the left, suppose the Gather instruction originally intended to load data multiple times and save it to different locations in the same vector register. Therefore, each load required waiting for the previous load to write its result back to the vector register, then reading the value of the vector register, merging it with the data being loaded, and then writing it back to the vector register. Thus, traditionally, there is a dependency between any two adjacent load operations.

[0047] And such Figure 6 As shown on the right, the eight load operations in the Gather instruction that were originally interdependent and to be loaded into the same vector register are divided into, for example, four groups. The first group consists of two consecutive / adjacent load operations, the second group consists of two consecutive / adjacent load operations, the third group consists of two consecutive / adjacent load operations, and the fourth group consists of two consecutive / adjacent load operations. It can be seen that the load operations in the first group are adjacent to those in the second group, but the load operations in the first group are not adjacent to those in the third or fourth group, and so on.

[0048] This application not only divides the eight load operations into four groups, but also loads the data in memory into four different vector registers during the execution of each group of load operations. Specifically, the first group of load operations within the multiple load operations of the Gather instruction loads the data in memory into the first vector register; the second group of load operations within the multiple load operations of the Gather instruction loads the data in memory into a second vector register different from the first vector register; and one or more third group of load operations within the multiple load operations of the Gather instruction load the data in memory into one or more third vector registers different from the first or second vector register.

[0049] Since the first vector register, the second vector register, one or more third vector registers, and the fourth vector register are registers with different destinations, for example, data loaded into the second vector register does not need to wait for data loaded into the first vector register to be merged; for example, data loaded into one or more third vector registers does not need to wait for data loaded into the first or second vector registers to be merged; for example, data loaded into other vector registers does not need to wait for data loaded into the first, second, or one or more third vector registers to be merged. Therefore, data loaded into one vector register does not need to wait for data loaded into another vector register. Thus, the execution of load operations loaded into different vector registers is independent of each other and can be executed in parallel, thereby improving the parallelism of load operations in the Gather instruction and speeding up the execution of the Gather instruction.

[0050] In other words, dividing the load operation in the Gather instruction into several groups requires several vector registers. Of course, these vector registers are temporary, and finally the data loaded into these temporary vector registers needs to be merged into a single vector register.

[0051] like Figure 6 As shown on the right, only two adjacent load operations are allowed to depend on each other; that is, two adjacent load operations are written back to the same vector register, while load operations in different groups are written back to different vector registers. Finally, the results stored in these temporarily used vector registers need to be merged and written back to the vector register defined at the instruction level. For example... Figure 6In the example shown on the right, there are two adjacent load operations in a group: the odd-numbered load and the even-numbered load. For example, the even-numbered load can depend on the result of the odd-numbered load, meaning that every two loads are written back to the same temporary vector register, making them interdependent. Loads from different groups are written back to different temporary vector registers, making them independent and allowing for parallel execution. After all the dependency chains have finished executing, a merging micro-operation is needed to write the results from multiple temporary vector registers back to a single temporary vector register.

[0052] Of course, the above example divides the 8 load operations into multiple groups containing the same number of load operations. However, the number of load operations in each group can also be different, achieving the same goal of increasing the parallelism of the load operations in the Gather instruction and speeding up its execution. For example, the 8 load operations can be divided into 3 groups: the first group contains 2 load operations, the second group contains 4 load operations, and the third group contains 2 load operations. In this case, parallelism is also improved, but it may be necessary to wait for the 4 load operations in the second group to complete before merging can be completed.

[0053] The current approach can determine the best average performance of a certain splitting scheme by investigating existing test programs and evaluating the performance of different splitting schemes across multiple test programs. This allows for a static determination of the splitting scheme for each Gather instruction encountered, such as splitting into groups of a fixed number of loading operations each time.

[0054] However, it is also possible to consider implementing multiple Gather instruction splitting schemes. By dynamically detecting the current state of the processor (e.g., detecting the number of remaining vector registers), the processor can decide whether to split the Gather instruction into an aggressive scheme with more dependency chains or a conservative scheme with fewer dependency chains in order to reserve vector registers for other instructions.

[0055] Specifically, the fewer load operations included in each group, the more groups can execute independently, requiring more vector registers and thus increasing the parallelism of the load operations in the Gather instruction. For example, dividing 8 load operations into groups of 2 results in 4 groups (dependency chains), all of which can execute in parallel, resulting in high parallelism. However, dividing 8 load operations into groups of 4 results in 2 groups (dependency chains), which execute in parallel, but with lower parallelism than 4 groups. Therefore, to increase the execution parallelism of the Gather instruction, the number of load operations in each group should be reduced. If the CPU determines that the Gather instruction is a critical instruction in the instruction dependency chain, meaning many instructions depend on it, then the execution parallelism (or execution time) of the Gather instruction becomes even more critical. This is because even if vector register resources are allocated to other instructions, they may still fail to execute due to their dependency on the Gather instruction. (This assumes, of course, that the CPU can determine that the Gather instruction is on the critical path.)

[0056] Furthermore, the number of load operations in each group can also be based on the number of available vector registers. Since each split dependency chain requires a temporary vector register to store the loaded data (using the same register would lead to confusion about which load returned the data due to out-of-order execution of load instructions), splitting into n dependency chains will use n temporary vector registers (n is a positive integer greater than 1). Therefore, if there are more than a predetermined threshold of available vector registers, the number of load operations in each group can be reduced, increasing the number of dependency chains and thus increasing the number of temporary vector registers required. However, when the running program has a tight demand for vector registers (e.g., the number of available vector registers is less than or equal to the predetermined threshold), the optimization scheme for the Gather instruction needs to be adjusted. This can increase the number of load operations in each group, reducing the number of dependency chains and thus reducing the number of temporary vector registers required.

[0057] For example, the original two adjacent load instructions forming a dependency chain can be changed to every four load instructions forming a dependency chain, that is, a total of 2 groups (2 dependency chains). Figure 7 A schematic diagram illustrating another example of a method for loading data into a vector register using the Gather instruction according to an embodiment of this application is shown. Figure 7As shown, every four loads form a dependency chain, resulting in a total of two groups (two dependency chains). Since only two dependency chains can run in parallel, and each dependency chain contains four load operations that depend on each other, the parallelism of the Gather instruction execution is sacrificed to some extent. However, by reducing the use of vector registers, the blocking caused by insufficient vector registers for non-Gather instructions can be reduced, thereby improving the overall program performance.

[0058] Therefore, the number of load operations in each group can be comprehensively considered based on the parallelism requirements of the Gather instruction and the number of available vector registers. Dividing into more dependency chains leads to the use of more temporary registers and more micro-operations and merging of the final result. The performance improvement brought by splitting dependency chains is not a linear function of the number of split dependency chains. Therefore, during implementation, it is necessary to comprehensively consider the cost of increasing the number of temporary registers used, the number of micro-operations for merging data, and the performance improvement brought by increasing parallelism to determine the specific number of different dependency chains to be split into for parallel execution.

[0059] In summary, in one embodiment, the execution of each adjacent loading operation in the first group of loading operations is interdependent, the execution of each adjacent loading operation in the second group of loading operations is interdependent, and the execution of the first group of loading operations and the execution of the second group of loading operations are independent of each other.

[0060] In one embodiment, the loading operations in the first group (excluding the last loading operation) are either adjacent or non-adjacent, and the loading operations in the second group (excluding the first loading operation) are either adjacent or non-adjacent. That is, there can also be two independent loading operations in either the first or second group, as long as at least the last loading operation in the first group and the first loading operation in the second group are adjacent among the multiple loading operations of the Gather instruction. In other words, according to the embodiments of this application, two originally dependent adjacent loading operations can be transformed into two independent loading operations that can be executed in parallel. Of course, in most cases, the loading operations in the first group (excluding the last loading operation) are also adjacent, and the loading operations in the second group (excluding the first loading operation) are also adjacent.

[0061] The first group of load operations includes a first number of load operations, and the second group of load operations includes a second number of load operations. The first number and the second number may be the same or different, and both are greater than one. The more vector registers used, the smaller the first number and / or the second number, and the greater the parallelism of the Gather instruction execution.

[0062] Then, because data is loaded from memory into various temporary vector registers in order to improve parallelism, the loaded data needs to be merged into a single vector register after completion.

[0063] In one embodiment, merging the data in the first vector register and the data in the second vector register into one vector register includes one of the following steps: merging the data in the first vector register and the data in the second vector register into either the first vector register or the second vector register; or merging the data in the first vector register and the data in the second vector register into a vector register different from either the first or the second vector register. That is, the data in two vector registers can be merged into one of the two vector registers, or the data in two vector registers can be merged into another separate vector register. In short, the scattered data is merged into one vector register, thus completing the operation that the Gather instruction would normally perform, i.e., retrieving data from memory and writing it into the same vector register.

[0064] One scenario where compilers heavily utilize the Gather instruction is the regular access to arrays, where the difference between the address loaded by Gather and the address loaded in each instance is a fixed value or the difference changes in a certain pattern. Therefore, the prefetcher in the processor, after a period of training, can easily predict which address will be loaded next when encountering the Gather instruction again. The prefetcher can issue a prefetch in advance during the current loading phase, retrieving the data to be loaded next into the L1 data cache (which is classified into L1, L2, and L3 caches based on access speed, from fastest to slowest. In the memory architecture, the L1 data cache has the second fastest access speed after registers). When the Gather instruction executes for the next load, the required data is already in the L1 data cache, and retrieving the data directly from the L1 data cache significantly shortens the time from issuing the load instruction to the data returning from memory to the register. Therefore, it can be concluded that the bottleneck within the Gather instruction is mainly the dependency between multiple load operations rather than the latency of the load operation itself, thus proving the effectiveness of the optimization scheme proposed in this application.

[0065] Therefore, in one embodiment, the method 500 may further include: using a prediction model to predict which address in memory the next load operation will access, wherein the prediction model is trained based on the accesses to various addresses in memory by various load operations; prefetching data at the predicted address from memory into an L1 data cache in the current load operation; and loading the prefetched data from the L1 data cache into the first, second, or one or more third vector registers in response to executing the next load operation.

[0066] Therefore, it is necessary to train the memory access instructions to obtain the memory address access patterns. During each load, the address to be loaded next can be inferred based on the access patterns. Data that is not in the L1 data cache can be retrieved back to the L1 data cache in advance to reduce the latency caused by L1 data cache miss in the next load.

[0067] Furthermore, the grouping of load operations in the Gather instruction can be considered in conjunction with a prefetcher. If the prefetcher performs well for a particular Gather instruction, with a high hit rate for the next load address, the bottleneck of the Gather instruction can be considered to be its internal dependencies, and the optimization scheme provided in the embodiments of this application is the best approach. However, if the prefetcher performs poorly for a Gather instruction with irregular access, the bottleneck of the Gather instruction can be considered to be the latency of loading data retrieved from memory, rather than its internal dependencies. In this case, the optimization scheme of this patent is unnecessary; that is, it is not necessary to use more temporary vector registers and more micro-operations to improve the internal parallelism of the Gather instruction.

[0068] Therefore, in one embodiment, the method 500 further includes: if the hit rate of the address predicted by the prediction model hitting the actual access address of the next load operation is greater than a predetermined threshold, then the method of loading data into the vector register using the Gather instruction is performed; if the hit rate of the address predicted by the prediction model hitting the actual access address of the next load operation is less than or equal to the predetermined threshold, then the method of loading data into the vector register using the Gather instruction is not performed.

[0069] In summary, this application proposes a micro-operation implementation scheme that can decompose dependency chains to address the dependencies within the Gather instruction. This improves the parallelism of the micro-operations within the Gather instruction, thereby significantly shortening the execution time of the Gather instruction and ultimately improving processor performance.

[0070] In addition, for other complex instructions with severe internal dependencies, the approach provided by this application can be used to break the dependency relationship by using a temporary register (not necessarily a vector register) to save intermediate results.

[0071] Figure 8 A flowchart of a method 800 for loading data into a register according to an embodiment of this application is shown.

[0072] like Figure 8 As shown, the method 800 for loading data into a register includes: step 810, loading data from memory into a first register through a first group of load operations in a plurality of load operations; step 820, loading data from memory into a second register through a second group of load operations in a plurality of load operations, wherein the second register is different from the first register, wherein the last load operation of the first group of load operations and the first load operation of the second group of load operations are adjacent in the plurality of load operations; and step 830, merging the data in the first register and the data in the second register into a single register.

[0073] In this way, the dependencies between complex instructions that are heavily dependent on each other can be broken by using temporary registers (not necessarily vector registers) to store intermediate results and then merging them at the end. This improves the parallelism of instruction execution, shortens the execution time of instructions, and ultimately improves the performance of the processor.

[0074] Figure 9 A block diagram of a processor 900 that loads data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0075] like Figure 9 As shown, the processor 900 that loads data into a vector register using the Gather instruction includes: a loader 910 configured to load data in memory into a first vector register via a first set of load operations in a plurality of load operations of the Gather instruction; and to load data in memory into a second vector register via a second set of load operations in a plurality of load operations of the Gather instruction, wherein the second vector register is different from the first vector register, and wherein the last load operation of the first set of load operations and the first load operation of the second set of load operations are adjacent in the plurality of load operations of the Gather instruction; and a merger 920 configured to merge the data in the first vector register and the data in the second vector register into a single vector register.

[0076] In one embodiment, the execution of each adjacent loading operation in the first group of loading operations is interdependent, the execution of each adjacent loading operation in the second group of loading operations is interdependent, and the execution of the first group of loading operations is independent of the execution of the second group of loading operations.

[0077] In one embodiment, the load operations in the first group of load operations, except for the last load operation, are either adjacent or non-adjacent; the load operations in the second group of load operations, except for the first load operation, are either adjacent or non-adjacent; the first group of load operations includes a first number of load operations; the second group of load operations includes a second number of load operations; and the first number and the second number are the same or different, and both are greater than one. The more vector registers used, the smaller the first number and / or the second number, and the greater the parallelism of the Gather instruction execution.

[0078] In one embodiment, the merger 920 is configured to perform one of the following steps: merging data in the first vector register and data in the second vector register into the first vector register or the second vector register; merging data in the first vector register and data in the second vector register into a vector register different from the first vector register or the second vector register.

[0079] In one embodiment, the loader 910 is further configured to: load data in memory into one or more third vector registers via one or more third group load operations of a plurality of load operations of the Gather instruction, wherein the one or more third vector registers are different from both the second vector register and the first vector register; and merge the data in the one or more third vector registers into the one vector register.

[0080] In one embodiment, the processor 900 further includes: a prefetcher (not shown) configured to: use a prediction model to predict which address in memory the next load operation will access, wherein the prediction model is trained based on accesses to addresses in memory by various load operations; and prefetch data at the predicted address from memory into an L1 data cache in advance during the current load operation; wherein the loader is configured to load the prefetched data from the L1 data cache into the first, second, or one or more third vector registers in response to performing the next load operation.

[0081] In one embodiment, the prefetcher is configured to: perform the method of loading data into the vector register using the Gather instruction if the hit rate of the address predicted by the prediction model hitting the actual access address of the next load operation is greater than a predetermined threshold; and not perform the method of loading data into the vector register using the Gather instruction if the hit rate of the address predicted by the prediction model hitting the actual access address of the next load operation is less than or equal to the predetermined threshold.

[0082] Therefore, to address the dependencies in the micro-operations within the Gather instruction, a micro-operation implementation scheme that splits the dependency chain is proposed. By loading the first group of load operations (which are mutually dependent) from the multiple load operations of the Gather instruction into the first vector register, and by loading the second group of load operations (which are mutually dependent) into a second vector register (different from the first vector register), the dependency relationship between the first group of load operations is broken. This objectively enables the first and second groups of load operations to be executed in parallel, improving the parallelism of the micro-operations within the Gather instruction, thereby significantly shortening the execution time of the Gather instruction and ultimately improving processor performance.

[0083] Figure 10 A block diagram of a processor 1000 that loads data into a vector register using the Gather instruction according to an embodiment of this application is shown.

[0084] like Figure 10 As shown, the processor 1000 for loading data into registers includes: a loader 1010 configured to load data from memory into a first register via a first set of load operations in a plurality of load operations; and to load data from memory into a second register via a second set of load operations in a plurality of load operations, wherein the second register is different from the first register, and wherein the last load operation of the first set of load operations and the first load operation of the second set of load operations are adjacent in the plurality of load operations; and a merger 1020 configured to merge the data in the first register and the data in the second register into a single register.

[0085] In this way, the dependencies between complex instructions that are heavily dependent on each other can be broken by using temporary registers (not necessarily vector registers) to store intermediate results and then merging them at the end. This improves the parallelism of instruction execution, shortens the execution time of instructions, and ultimately improves the performance of the processor.

[0086] Figure 11 A block diagram of an exemplary electronic device suitable for implementing embodiments of the present application is shown.

[0087] An electronic device may include a processor (H1); and a storage medium (H2) coupled to the processor (H1) and storing processor-executable instructions therein for performing the steps of various methods of embodiments of the present application when executed by the processor.

[0088] The processor (H1) may include, but is not limited to, one or more processors or microprocessors.

[0089] Storage media (H2) may include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (e.g., hard disks, floppy disks, solid-state drives, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, etc.).

[0090] In addition, the electronic device may also include a data bus (H3), an input / output (I / O) bus (H4), a display (H5), and input / output devices (H6) (e.g., keyboard, mouse, speaker, etc.).

[0091] The processor (H1) can communicate with external devices (H5, H6, etc.) via the I / O bus (H4) through a wired or wireless network (not shown).

[0092] The storage medium (H2) may also store at least one processor-executable instruction for performing the steps of various functions and / or methods in the embodiments described in this technology when executed by the processor (H1).

[0093] In one embodiment, the at least one processor-executable instruction may also be compiled into or comprise a software product, wherein one or more processor-executable instructions are executed by the processor at runtime to perform the steps of the various functions and / or methods in the embodiments described herein.

[0094] Figure 12 A schematic diagram of a non-transitory readable storage medium according to an embodiment of the present disclosure is shown.

[0095] like Figure 12As shown, the readable storage medium 1220 stores instructions, such as readable instruction 1210. When the readable instruction 1210 is executed by the processor, the various methods described above can be performed. The readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the readable storage medium 1220 can be connected to a computing device such as a computer, and then, when the computing device executes the readable instruction 1210 stored on the readable storage medium 1220, the various methods described above can be performed.

[0096] In summary, 1. To address the dependencies within the micro-operations of the Gather instruction, a micro-operation implementation scheme that can decompose the dependency chain is proposed, improving the parallelism of the micro-operations within the Gather instruction and thus significantly shortening the execution time of the Gather instruction. 2. Considering that the additional resources required by the decomposition scheme may affect the execution efficiency of other non-Gather instructions, a specific decomposition scheme is considered based on the overall program situation.

[0097] Of course, the specific embodiments described above are merely examples and not limitations. Those skilled in the art can combine and integrate some steps and devices from the various embodiments described separately above to achieve the effects of this application based on the concept of this application. Such combined and integrated embodiments are also included in this application, but will not be described one by one here.

[0098] Note that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of the various embodiments of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations; these details do not restrict this application from being implemented using the aforementioned specific details.

[0099] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0100] The flowcharts and method descriptions in this disclosure are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the given order. As those skilled in the art will recognize, the steps in the above embodiments can be performed in any order. Words such as "then," "next," etc., are not intended to limit the order of the steps; these words are only used to guide the reader through the description of these methods. Furthermore, any reference to a singular element, such as the use of the articles "a," "one," or "the," is not to be construed as limiting that element to the singular.

[0101] Furthermore, the steps and apparatus in the various embodiments herein are not limited to any one embodiment. In fact, new embodiments can be conceived by combining relevant steps and apparatus in the various embodiments herein based on the concepts of this application, and these new embodiments are also included within the scope of this application.

[0102] The various operations of the methods described above can be performed by any suitable means capable of performing the corresponding functions. Such means may include various hardware and / or software components and / or modules, including but not limited to hardware circuits, application-specific integrated circuits (ASICs), or processors.

[0103] The various exemplified logic blocks, modules, and circuits described herein can be implemented or performed using a general-purpose processor, digital signal processor (DSP), ASIC, field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor, but alternatively, it can be any commercially available processor, controller, microcontroller, or state machine. The processor can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, a microprocessor cooperating with a DSP core, or any other such configuration.

[0104] The steps of the methods or algorithms described in this disclosure can be directly embedded in hardware, in a software module executed by a processor, or a combination of both. The software module can reside in any form of tangible storage medium. Some examples of storage media that can be used include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disks, removable disks, CD-ROMs, etc. The storage medium can be coupled to the processor so that the processor can read information from and write information to the storage medium. Alternatively, the storage medium can be integral with the processor. The software module can be a single instruction or many instructions, and can be distributed across several different code segments, different programs, and across multiple storage media.

[0105] The methods disclosed herein include actions for implementing the described methods. The methods and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions may be modified without departing from the scope of the claims.

[0106] The above functions can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored as instructions on a tangible readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example, and not limitation, such tangible media can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital universal disc (DVD), floppy disk, and Blu-ray disc, wherein a disc typically magnetically reproduces data, while a disc optically reproduces data using lasers.

[0107] Therefore, a computer program product can perform the operations given herein. For example, such a computer program product can be a readable tangible medium having instructions tangibly stored (and / or encoded) thereon, which can be executed by a processor to perform the operations described herein. The computer program product may include packaging materials.

[0108] Software or instructions can also be transmitted via a transmission medium. For example, software can be transmitted from a website, server, or other remote source using transmission media such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, or microwave.

[0109] Furthermore, modules and / or other suitable means for carrying out the methods and techniques described herein can be downloaded and / or obtained by user terminals and / or base stations as appropriate. For example, such a device can be coupled to a server to facilitate the transmission of means for carrying out the methods described herein. Alternatively, the various methods described herein can be provided via storage components (e.g., RAM, ROM, physical storage media such as CDs or floppy disks) so that user terminals and / or base stations can obtain the various methods when coupled to the device or when storage components are provided to the device. Furthermore, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0110] Other examples and implementations are within the scope and spirit of this disclosure and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Features implementing the functions can also be physically located in various places, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the described examples are preferred or better than other examples.

[0111] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0112] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0113] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for loading data into a vector register using the Gather instruction, comprising: The data in memory is loaded into the first vector register through the first group of load operations in the multiple load operations of the Gather instruction; Data in memory is loaded into a second vector register through the second group of load operations in the multiple load operations of the Gather instruction. The second vector register is different from the first vector register. The last load operation of the first group of load operations and the first load operation of the second group of load operations are adjacent in the multiple load operations of the Gather instruction. The data in the first vector register and the data in the second vector register are merged into a single vector register. In this process, the execution of each adjacent loading operation in the first group of loading operations is interdependent, the execution of each adjacent loading operation in the second group of loading operations is interdependent, and the execution of the first group of loading operations and the execution of the second group of loading operations are independent of each other.

2. The method of claim 1, wherein, In the first group of load operations, the load operations except the last load operation are either adjacent or not adjacent. In the second group of load operations, the load operations except the first load operation are either adjacent or not adjacent. The first group of load operations includes a first number of load operations, and the second group of load operations includes a second number of load operations. The first number and the second number are the same or different, and both are greater than one. The more vector registers used, the smaller the first number and / or the second number, and the greater the parallelism of the Gather instruction execution.

3. The method of claim 1, wherein, The step of merging the data in the first vector register and the data in the second vector register into a single vector register includes one of the following steps: Merge the data in the first vector register and the data in the second vector register into the first vector register or the second vector register; The data in the first vector register and the data in the second vector register are merged into a vector register that is different from either the first vector register or the second vector register.

4. The method according to claim 1, further comprising: Data in memory is loaded into one or more third vector registers through one or more third group load operations in a plurality of load operations of the Gather instruction, wherein the one or more third vector registers are different from both the second vector register and the first vector register; The data in the one or more third vector registers are merged into the one vector register.

5. The method according to claim 4, further comprising: This is used to use a predictive model to predict which address in memory the next load operation will access, wherein the predictive model is trained based on the accesses to various addresses in memory by each load operation. In the current loading operation, data at the predicted address is prefetched from memory into the L1 data cache; In response to performing the next load operation, the prefetched data is loaded from the L1 data cache into the first vector register, the second vector register, or one or more third vector registers.

6. The method according to claim 5, further comprising: If the hit rate of the predicted address in the next load operation is greater than a predetermined threshold, then the method of loading data into the vector register using the Gather instruction is performed. If the hit rate of the predicted address in the next load operation is less than or equal to a predetermined threshold, then the method of loading data into the vector register using the Gather instruction will not be performed.

7. A processor that loads data into a vector register using the Gather instruction, comprising: A loader is configured to load data in memory into a first vector register via a first set of load operations in a plurality of load operations of the Gather instruction; and to load data in memory into a second vector register via a second set of load operations in a plurality of load operations of the Gather instruction, wherein the second vector register is different from the first vector register, and wherein the last load operation of the first set of load operations and the first load operation of the second set of load operations are adjacent in the plurality of load operations of the Gather instruction; The merger is configured to merge the data in the first vector register and the data in the second vector register into a single vector register. In this process, the execution of each adjacent loading operation in the first group of loading operations is interdependent, the execution of each adjacent loading operation in the second group of loading operations is interdependent, and the execution of the first group of loading operations and the execution of the second group of loading operations are independent of each other.

8. The processor according to claim 7, wherein, In the first group of load operations, the load operations except the last load operation are either adjacent or not adjacent. In the second group of load operations, the load operations except the first load operation are either adjacent or not adjacent. The first group of load operations includes a first number of load operations, and the second group of load operations includes a second number of load operations. The first number and the second number are the same or different, and both are greater than one. The more vector registers used, the smaller the first number and / or the second number, and the greater the parallelism of the Gather instruction execution.

9. The processor according to claim 7, wherein, The merger is configured to perform one of the following steps: Merge the data in the first vector register and the data in the second vector register into the first vector register or the second vector register; The data in the first vector register and the data in the second vector register are merged into a vector register that is different from either the first vector register or the second vector register.

10. The processor according to claim 7, wherein, The loader is also configured to: Data in memory is loaded into one or more third vector registers through one or more third group load operations in a plurality of load operations of the Gather instruction, wherein the one or more third vector registers are different from both the second vector register and the first vector register; The data in the one or more third vector registers are merged into the one vector register.

11. The processor of claim 10, further comprising: The prefetcher is configured as follows: This is used to use a predictive model to predict which address in memory the next load operation will access, wherein the predictive model is trained based on the accesses to various addresses in memory by each load operation. as well as In the current loading operation, data at the predicted address is prefetched from memory into the L1 data cache; The loader is configured to load the prefetched data from the L1 data cache into the first vector register, the second vector register, or one or more third vector registers in response to performing the next load operation.

12. The processor of claim 11, wherein the prefetcher is configured to: If the hit rate of the predicted address in the next load operation is greater than a predetermined threshold, then the method of loading data into the vector register using the Gather instruction is performed. If the hit rate of the predicted address in the next load operation is less than or equal to a predetermined threshold, then the method of loading data into the vector register using the Gather instruction will not be performed.

13. An electronic device, comprising: Memory, used to store instructions; A processor for reading instructions from the memory and executing the method as described in any one of claims 1-6.

14. A non-temporary storage medium having instructions stored thereon, in, When the instruction is read by the processor, it causes the processor to execute the method as described in any one of claims 1-6.

Images