A method for arranging SIMD instructions based on a RISC-V architecture

Abstract

The application discloses a SIMD instruction arrangement method based on a RISC-V architecture and belongs to the field of processor instruction arrangement.The method comprises the following steps: identifying and analyzing a SIMD applicable part in a calculation process as a calculation task for SIMD optimization; designing a basic code template according to the calculation task, and abstractly representing the calculation task; mapping operations in the basic code to a SIMD calculation structure; designing a SIMD instruction according to the vector extension characteristics of the RISC-V architecture; and rearranging the order of the SIMD instruction so that the instructions in a loop body are unfolded and executed in the same instruction set to reduce the idle period between instructions, thereby improving the parallel degree and throughput of instruction execution.The application shortens the execution period of all types of instructions, greatly improves the overall calculation performance, reduces energy consumption, improves resource utilization, and makes the GPU more efficient when processing large-scale parallel calculation tasks.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of instruction scheduling of processors, and in particular to a SIMD instruction scheduling method based on RISC-V architecture. BACKGROUND

[0002] In the field of artificial intelligence (AI) today, the graphics processing unit (GPU) has become the key hardware for processing computationally intensive tasks. The core of the GPU is composed of general-purpose CUDA-core and specialized Tensor-core, both of which provide powerful processing capabilities for AI computing. CUDA-core, as the basic processing unit of GPU, can handle a variety of general-purpose computing tasks in parallel, while Tensor-core is optimized for deep learning and matrix operations to accelerate these computationally intensive operations.

[0003] In CUDA-core, single instruction multiple data stream (SIMD) technology is the key to achieving efficient data processing. SIMD technology allows a single instruction to operate on multiple data elements simultaneously, with these data existing in the form of vectors, significantly improving the parallelism and efficiency of data processing, especially in image processing and machine learning fields.

[0004] RISC-V architecture has gained widespread application in processor design for its simplicity and scalability. In the RISC-V architecture, instruction execution adopts a five-stage pipeline model, including instruction fetch (Instruction Fetch), instruction decode (Instruction Decode), execution (Execute), memory access (Memory), and write back (Write Back) five stages. This design aims to improve the throughput of instruction processing, but in processing loop instructions, the traditional execution mode has not fully utilized the parallel processing potential of the pipeline, and there are still the following problems:

[0005] 1. Inadequate utilization of instruction-level parallelism

[0006] Traditional compilers use zero-time unrolling (i.e., no unrolling) for SIMD instruction scheduling of loop bodies, resulting in more idle cycles in the pipeline. For example, in a five-stage pipeline, if there is a dependency between instructions and the loop is not unrolled, a stall cycle needs to be inserted.

[0007] 2. Insufficient adaptation of hardware characteristics

[0008] Existing methods fail to fully incorporate the characteristics of processor vector components (such as vector register grouping strategies, data alignment constraints, and mixed-precision computing capabilities), resulting in insufficient register utilization (typically below 70%) and wasted memory bandwidth. For example, if the target platform requires FP16 data to be aligned on 32-byte boundaries in memory, failing to unroll loops can result in unaligned accesses (e.g., simd_loadu), leading to performance degradation.

[0009] 3. Multi-operator fusion optimization is limited

[0010] Existing solutions mostly optimize a single operator (such as ReLU) and lack a unified optimization framework for operators such as Norm, Softmax, Pooling and their fusion operations, which limits the possibility of cross-operator instruction reuse.

[0011] 4. Interface abstraction and implementation are disconnected

[0012] SIMD programming interfaces in high-level languages ​​(such as simd_load, simd_store, simd_vfcmpleh, and simd_vfselleh) are highly coupled with underlying hardware features, making manual optimization difficult and difficult to adapt to different vector bit widths (such as 128-bit / 256-bit) and data types (FP16 / INT4).

[0013] In zero-unroll code, each iteration consists of three instructions (Load→Compute→Store), with no parallelism between iterations. Assuming a pipeline depth of 5 stages, completing N iterations requires approximately 5N clock cycles, resulting in a pipeline utilization of only 20%.

[0014] In many computing scenarios, there are no data dependencies between instructions in different iterations of a loop. This means that once an instruction completes a stage in the pipeline, the next instruction can be executed after the previous instruction has already passed all or part of the pipeline, thus avoiding execution blockages caused by instruction dependencies. Therefore, by reordering instructions, the parallel processing capabilities of the five-stage pipeline can be fully utilized, further improving the GPU's execution efficiency.

[0015] Specifically, when executing load instructions within a loop, since load instructions in different loop iterations have no data dependencies, they can be executed continuously in the pipeline. For example, while the previous load instruction is in the execution phase, the next load instruction can begin the decode phase, and the one after that can begin the instruction fetch phase. This approach effectively reduces the waiting time for each instruction in the pipeline, allowing five load instructions that originally required 25 clock cycles to be completed in just 9 clock cycles, significantly improving the GPU's instruction execution throughput and overall performance. Summary of the Invention

[0016] In view of the problems of low pipeline utilization, waste of register resources and insufficient optimization of multi-operator fusion caused by SIMD instruction arrangement in the prior art, the purpose of the present invention is to provide a SIMD instruction arrangement method based on RISC-V architecture.

[0017] In order to achieve the above object, the technical solution adopted by the present invention is as follows: a SIMD instruction arrangement method based on RISC-V architecture, comprising the following steps:

[0018] Identify and analyze SIMD-applicable parts of a computational process as computational tasks; the computational process is a matrix operation or activation function processing process in a deep learning model, and the applicable parts include vector normalization, ReLU activation function, Softmax, Pooling operations, and their fusion operators;

[0019] Construct a parameterized code template SIMDTempl according to the computing task<T,VW,MP,CP> The template includes a data type parameter T (such as float16v16, intv16), a vector width parameter VW (16 or 32), a memory access mode parameter MP (continuous access STRIDED or aggregate hash GATHER), and a calculation mode parameter CP (element-by-element ELEM or fused multiply-add DOTP).

[0020] Mapping operations in the base code to SIMD computation structures (e.g., FP16 comparison → simd_vfcmpleh, conditional selection → simd_vfselleh), including instruction set adaptation, dynamic register grouping, and memory access mode optimization; the SIMD computation structure includes vector multiplication and addition, vector comparison, and vector type conversion;

[0021] Designing SIMD instructions based on the vector extension feature of the RISC-V architecture, wherein the SIMD instructions include vector width configuration, data layout optimization, and five loop unrolling;

[0022] The SIMD instruction execution process is optimized through an instruction reordering strategy. The strategy adopts a scheduling algorithm based on a dependency distance matrix to centrally execute instructions of the same type and eliminate cross-iteration register conflicts.

[0023] Furthermore, mapping the operations in the basic code to the SIMD computing structure includes:

[0024] Create a mapping table from mathematical operations to target platform SIMD instructions, including:

[0025] FP16 comparison operations are mapped to simd_vfcmpleh instructions;

[0026] Conditional selection operations are mapped to simd_vfselleh instructions;

[0027] Type conversion operations are mapped to simd_vfcvtsh or simd_vfcvths instructions;

[0028] Dynamically divide 32 vector registers into input buffer, calculation workspace and output buffer.

[0029] Furthermore, the dynamic grouping of registers includes dividing the 32 vector registers into:

[0030] Input buffer, 4 groups × 4 registers;

[0031] Computation workspace, 2 groups × 8 registers;

[0032] Output buffer, 2 groups × 4 registers;

[0033] The register overflow problem is solved by the Chordal graph coloring algorithm, and the vector length is dynamically adjusted using the vsetvli instruction.

[0034] Furthermore, the SIMD instruction includes a multi-operator fusion implementation, and the multi-operator fusion implementation includes:

[0035] LayerNorm fusion implementation includes variance calculation and normalization operations, and uses simd_reduc_plush for reduction and summation;

[0036] Softmax implementation includes maximum reduction, exponential calculation and normalization operations, using simd_smaxh and simd_vfmah instructions;

[0037] The ReLU and Pooling fusion operations are implemented through the simd_vfselleh and simd_vfaddh instructions.

[0038] Furthermore, the five-cycle expansion includes:

[0039] Divide the loop body into five parallel execution units, each unit processes VW data elements;

[0040] Use the #pragma unroll 5 compiler directive to control the number of unrolls;

[0041] Perform a sequence of Load-Compute-Store operations in each iteration of the unroll;

[0042] Eliminate cross-iteration register conflicts through register rotation technology.

[0043] Furthermore, the values ​​of the VW data elements are dynamically determined according to the SIMD bit width of the processor. When the processing unit width is 128 bits, VW=16; when the processing unit width is 256 bits, VW=32.

[0044] Furthermore, the instruction reordering strategy includes: constructing an instruction dependency graph, calculating the RAW / WAW dependency distances between instructions; and applying an improved list scheduling algorithm to generate an optimal execution sequence.

[0045] The present invention also provides a SIMD instruction arrangement device based on the RISC-V architecture, comprising:

[0046] A processor, configured to execute the SIMD instruction arrangement method based on the RISC-V architecture;

[0047] A memory for storing program data, wherein the program data includes a SIMD instruction template library, a loop unrolling optimization module, and an instruction reordering scheduling module; wherein the SIMD instruction template library supports vectorized operations of data types such as FP16 and INT4, the loop unrolling optimization module supports configurations of five or more unrolling times, and the instruction reordering scheduling module implements an instruction reordering algorithm based on a dependency distance matrix.

[0048] The present invention further provides an electronic device, comprising a memory and a processor, wherein the memory is coupled to the processor;

[0049] The memory is used to store program data, and the processor is used to execute the program data to implement the SIMD instruction arrangement method based on the RISC-V architecture;

[0050] The program data includes a SIMD interface adaptation layer, which supports simd_load, simd_store, and simd_vfselleh interface calls.

[0051] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the SIMD instruction arrangement method based on the RISC-V architecture.

[0052] The beneficial effects of the present invention are: the present invention significantly improves the SIMD instruction execution efficiency of CUDA-core in the GPU, and greatly improves the overall computing performance by shortening the execution cycle of all types of instructions; the present invention significantly improves the SIMD instruction execution efficiency of CUDA-core in the GPU, and experimental verification shows that it can effectively improve computing performance.

[0053] By optimizing the arrangement of instructions in the pipeline, this invention reduces the idle time of the pipeline, lowers energy consumption, and improves resource utilization, making the GPU more efficient when processing large-scale parallel computing tasks; parameterized template design reduces the threshold for manual optimization; and the instruction reordering algorithm can adapt to different pipeline depths (such as five levels / seven levels).

[0054] The present invention has good compatibility and scalability, can be seamlessly integrated into the existing RISC-V architecture, supports multiple data types such as FP16 / BF16 / INT4 and multi-operator fusion (such as ReLU+Pooling, Norm+Softmax), and adapts to different vector bit widths (128bit / 256bit) and memory access modes (continuous / hash).

[0055] The implementation of this invention not only optimizes the efficiency issues in the existing technology, but also provides strong technical support for the in-depth application of GPU in the fields of artificial intelligence, big data processing, etc., and has important practical application value and broad market prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0058] Figure 2 It is a parametric template framework diagram;

[0059] Figure 3 Mapping flow chart for SIMD instructions;

[0060] Figure 4 This is a diagram of the multi-operator fusion architecture;

[0061] Figure 5 This is a timing comparison chart of the RISC-V five-stage pipeline. DETAILED DESCRIPTION

[0062] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0063] The terms used in this invention are for the purpose of describing specific embodiments only and are not intended to limit the invention. The singular forms "a," "the," and "the" used in this invention and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0064] It should be understood that although the terms "first," "second," "third," etc. may be used in the present invention to describe various information, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, without departing from the scope of the present invention. Depending on the context, the term "if" as used herein may be interpreted as "when," "when," or "in response to determining."

[0065] The present invention will be described in detail below with reference to the accompanying drawings. Unless there is any conflict, the features of the following embodiments and implementations may be combined with each other.

[0066] like Figure 1 As shown, an embodiment of the present invention provides a SIMD instruction reordering method based on the RISC-V architecture, comprising the following steps:

[0067] Step 1: Identify and analyze SIMD-friendly parts of the computational process

[0068] In this step, the present invention first analyzes the computational flow to identify computing tasks suitable for SIMD processing. These tasks are typically continuous and data-intensive operations, such as vector normalization (norm) and the ReLU activation function. By analyzing the characteristics of these tasks, the present invention determines which parts can be optimized using SIMD instructions to improve data processing speed and efficiency.

[0069] Specifically, this embodiment uses a multi-dimensional analysis method to identify computing tasks, including the following sub-steps:

[0070] 1.1 Data Flow Graph Analysis: Build a control flow graph using the compiler intermediate representation (IR) and use data dependency analysis algorithms (such as the GCD test) to identify vectorizable regions. For example, in a matrix multiplication kernel function, it is possible to identify the inner loop within a loop as fully parallelizable.

[0071] 1.2. Instruction-level parallelism evaluation: The LLVM-based LoopVectorize tool counts the SIMD instruction coverage in basic blocks. When the floating-point operation density exceeds 60% and the memory access pattern is regular, it is marked as a high-priority optimization area.

[0072] 1.3. Hardware feature matching: Dynamically adjust the recognition threshold based on the VLEN (vector register length) parameter of the target processor. For example, in a VLEN=512-bit architecture, the minimum data parallelism must be ≥16 to trigger SIMD conversion.

[0073] For example, for the following ReLU function:

[0074] / / Original scalar implementation

[0075] for(int i=0; i <len; i++)

[0076] {

[0077] y[i] = x[i]>0 ? x[i] : 0;

[0078] }

[0079] The following characteristics were found through analysis:

[0080] Data independence: The calculation of each element does not depend on other elements.

[0081] Data width: Both input and output are in FP16 format.

[0082] Computational density: A single computation requires only one comparison and one selection operation.

[0083] Memory bandwidth: Each 64-byte cache line contains 32 FP16 elements.

[0084] Step 2: Design the basic code template for the computing task

[0085] After determining the computing tasks for which SIMD is applicable, the present invention will write basic code templates for these tasks. These templates will contain basic operation flows but will not involve specific SIMD instructions. The design of the basic code will take into account the mapping and optimization of subsequent SIMD instructions to ensure the scalability and maintainability of the code.

[0086] like Figure 2 As shown in the figure, the parameterized template framework includes the data type parameter T, the vector width parameter VW, the memory access mode parameter MP, and the calculation mode parameter CP; and includes three core components: the loading component, the calculation component, and the storage component; the details are as follows:

[0087] / / Template definition (pseudocode)

[0088] template<typename T, int VW, MemPattern MP, ComputePattern CP>

[0089] class SIMDTempl{

[0090] public:

[0091] using VecReg = VectorRegister<T,VW> ;

[0092] void load(VecReg* v, const T* ptr);

[0093] void compute(VecReg* v);

[0094] void store(const VecReg* v, T* ptr);

[0095] };

[0096] For example, the template instantiation for ReLU:

[0097] template<>

[0098] class SIMDTempl<__Float16,16,STRIDED,ELEMWISE>{

[0099] public:

[0100] using VecReg = float16v16;

[0101] static void load(VecReg&v, const __Float16* ptr){

[0102] v = simd_load <float16v16>(ptr);

[0103] }

[0104] static void compute(VecReg&v){

[0105] / / Application-specific instruction simd_vfselles

[0106] v = simd_vfselles(v, 0, v); / / Conditional selection

[0107] }

[0108] };

[0109] Step 3: Mapping the basic code to SIMD computation structures

[0110] Consult the SIMD interface manual and related documents to find the calculation structure corresponding to the basic code, such as vector multiplication and addition (FMA), vector reciprocal, etc. Figure 3 As shown, the operations in the basic code are mapped to the computing structure provided by the SIMD instruction set, which specifically includes the following sub-steps:

[0111] 3.1. Establish an operator feature database to store 16 basic SIMD operation templates, including vector multiplication and addition (FMA), vector comparison, and vector type conversion.

[0112] 3.2. Perform AST (Abstract Syntax Tree) analysis on the basic code to identify operator nodes that are feasible for vectorization.

[0113] 3.3. Use dynamic programming algorithm to perform optimal template matching, so that the matching error rate is less than 0.5%.

[0114] 3.4. Generate an intermediate representation IR containing vector register allocation constraints.

[0115] / / Access optimization

[0116] #pragma unroll 5

[0117] for(int i=0; i<5; i++){

[0118] Simd_load(v[i], x + i*vec_len);

[0119] }

[0120] Step 4: Design and implement SIMD instructions

[0121] Based on the characteristics of the processor's vector unit, specific SIMD instructions are designed. This involves determining the vector width, arranging the data layout within the vector registers, and utilizing SIMD functions in high-level languages. A specific SIMD instruction example involves declaring SIMD registers, specifying their addresses, and designing a computational loop. The loop contains instructions such as "Load" to load data into the registers, performing multiply-add operations, and "Store" to save the register contents. These SIMD instructions maximize the processor's vector processing capabilities, improving data processing parallelism and throughput.

[0122] The loop body is specifically:

[0123] / / Five loop expansion implementation

[0124] void relu_forward(__Float16* x, __Float16* y, int len){

[0125] int i=0;

[0126] float16v16 hvec[5]; / / 5 groups of registers

[0127] floatv16 fvec[5];

[0128] / / Main loop: five expansions

[0129] for(; i+80<=len; i+=80){ / / 16 elements×5=80

[0130] / / Parallel loading

[0131] #pragma unroll 5

[0132] for(int j=0; j<5; j++){

[0133] simd_load(hvec[j], x + i + j*16);

[0134] }

[0135] / / parallel computing

[0136] #pragma unroll 5

[0137] for(int j=0; j<5; j++){

[0138] hvec[j] = simd_vfselles(hvec[j], 0, hvec[j]); / / Conditional selection

[0139] }

[0140] / / Parallel storage

[0141] #pragma unroll 5

[0142] for(int j=0; j<5; j++){

[0143] simd_store(hvec[j], y + i + j*16);

[0144] }

[0145] }

[0146] / / Residue processing

[0147] for(; i+16<=len; i+=16){

[0148] float16v16 hv = simd_load(x+i);

[0149] hv = simd_vfselles(hv, 0, hv);

[0150] simd_store(hv, y+i);

[0151] }

[0152] }

[0153] Step 5: Optimize SIMD instruction execution process

[0154] In this step, an instruction reordering strategy is applied, which aims to reduce the execution latency and improve the processing efficiency of the CUDA-core in the GPU by reorganizing the execution order of SIMD instructions.

[0155] In the RISC-V architecture, the five-stage pipeline significantly improves efficiency. For example, in a loop, executing five instructions would typically take 5*5=25 cycles. After the five-stage pipeline, this can be completed in just 4+5=9 cycles, significantly reducing instruction execution time. This calculation requires that subsequent instructions cannot depend on previous instructions; otherwise, execution will be blocked until the dependent instructions complete.

[0156] In this step, the method expands multiple instructions of the same type that were originally executed separately in different loops, such as load instructions. Specifically, the method extracts five consecutive instruction loops from independent execution sequences, and concentrates the same instructions, and executes them continuously in sequence, thereby achieving instruction reordering. Through this instruction reordering technology, the load instructions that were originally scattered in multiple clock cycles are executed in a concentrated manner, and it is ensured that when subsequent instructions are executed, their dependent instructions have been executed, thereby reducing idle cycles between instructions. This not only improves the data processing throughput, but also optimizes the computing efficiency of the GPU core, bringing significant performance improvements to processors based on the RISC-V architecture.

[0157] The instruction reordering technology is specifically as follows:

[0158] a. Build an instruction dependency graph (IGD) to identify instruction pairs without data dependencies.

[0159] b. Apply a greedy algorithm to cluster instructions: group instructions of the same functional unit into one category; sort them according to the pipeline stage conflict matrix, including:

[0160] vector <instruction>schedule(vector <instruction>&instrs,DependencyGraph&dg){

[0161] vector <instruction>result;

[0162] while(!instrs.empty()){

[0163] auto candidate = select_ready_instrs(instrs, dg);

[0164] auto best = find_optimal_sched(candidate, dg);

[0165] result.push_back(best);

[0166] update_dependencies(best, dg);

[0167] }

[0168] return result;

[0169] }

[0170] c. Implement five-stage instruction scheduling:

[0171] clock cycle Instruction fetch Decoding implement Memory Access Write Back 1 L0 2 L1 L0 3 L2 L1 L0 4 L3 L2 L1 L0 5 L4 L3 L2 L1 L0 6 L4 L3 L2 L1 ...

[0172] d. Through the above scheduling, the instruction throughput is improved by 2.8 times.

[0173] Through the implementation of the above technical solutions, the present application can significantly improve the performance of the GPU based on the RISC-V architecture when executing SIMD instructions, reduce the instruction execution time, and improve the computing efficiency, thereby exhibiting higher performance in processing large-scale parallel computing tasks.

[0174] Among them, as shown in Figure 4 , the multi-operator fusion extension is implemented, and the method is extended to other computing modes:

[0175] / / Multi-operator fusion function definition (ReLU + LayerNorm + Softmax)

[0176] void fused_relu_norm_softmax(

[0177] _Float16* input, / / Input: FP16 data, shape [N, D]

[0178] _Float16* output, / / Output: FP16 data, shape [N, D]

[0179] int N, / / batch size

[0180] int D / / Feature dimension (must be a multiple of 16)

[0181] ) {

[0182] float16v16 hvec[5]; / / FP16 input register (5 groups × 16 elements)

[0183] floatv16 fvec[5]; / / FP32 intermediate register (5 groups × 16 elements)

[0184] for (int b = 0; b <N; b++) { / / 遍历批量

[0185] _Float16* x = input + b * D;

[0186] _Float16* y = output + b * D;

[0187] for (int i = 0; i + 80<= D; i += 80) {

[0188] / / 1. Load FP16 data in parallel

[0189] #pragma unroll 5

[0190] for (int j = 0; j<5; j++) {

[0191] simd_load(hvec[j], x + i + j * 16);

[0192] }

[0193] / / 2. ReLU calculation (FP16)

[0194] #pragma unroll 5

[0195] for (int j = 0; j<5; j++) {

[0196] hvec[j] = simd_vfselleh(hvec[j], 0, hvec[j]);

[0197] }

[0198] / / 3. Convert to FP32 precision

[0199] #pragma unroll 5

[0200] for (int j = 0; j<5; j++) {

[0201] fvec[j] = simd_vfcvths(hvec[j]);

[0202] }

[0203] / / 4. LayerNorm计算(FP32)

[0204] {

[0205] floatv16 sum = {0};

[0206] floatv16 sq_sum = {0};

[0207] #pragma unroll 5

[0208] for (int j = 0; j<5; j++) {

[0209] sum += fvec[j];

[0210] sq_sum += fvec[j] * fvec[j];

[0211] }

[0212] float mean = simd_reduc_plusfs(sum) / D;

[0213] float inv_std = 1.0f / sqrtf(simd_reduc_plusfs(sq_sum) / D - mean * mean + 1e-6f);

[0214] #pragma unroll 5

[0215] for (int j = 0; j<5; j++) {

[0216] fvec[j] = (fvec[j]- mean) * inv_std;

[0217] }

[0218] }

[0219] / / 5. Softmax计算(FP32)

[0220] {

[0221] floatv16 max_val = fvec[0];

[0222] #pragma unroll 5

[0223] for (int j = 1; j<5; j++) {

[0224] max_val = simd_smaxh(max_val, fvec[j]);

[0225] }

[0226] #pragma unroll 5

[0227] for (int j = 0; j<5; j++) {

[0228] fvec[j] = fvec[j]- max_val;

[0229] }

[0230] floatv16 exp_sum = {0};

[0231] #pragma unroll 5

[0232] for (int j = 0; j<5; j++) {

[0233] fvec[j] = vexp(fvec[j]);

[0234] exp_sum += fvec[j];

[0235] }

[0236] float inv_exp_sum = 1.0f / simd_reduc_plusfs(exp_sum);

[0237] #pragma unroll 5

[0238] for (int j = 0; j<5; j++) {

[0239] fvec[j] = fvec[j]* inv_exp_sum;

[0240] }

[0241] }

[0242] / / 6. Convert back to FP16 and store

[0243] #pragma unroll 5

[0244] for (int j = 0; j<5; j++) {

[0245] hvec[j] = simd_vfcvtsh(fvec[j]);

[0246] simd_store(hvec[j], y + i + j*16);

[0247] }

[0248] }

[0249] / / 7. Residual processing (processing the part with less than 80 elements)

[0250] for (int i = 0; i + 16<= D; i += 16) {

[0251] simd_load(hvec[0], x + i);

[0252] hvec[0] = simd_vfselleh(hvec[0], 0, hvec[0]);

[0253] fvec[0] = simd_vfcvths(hvec[0]);

[0254] / / Norm / Softmax calculation is omitted here (same as the main loop logic)

[0255] hvec[0] = simd_vfcvtsh(fvec[0]);

[0256] simd_store(hvec[0], y + i);

[0257] }

[0258] }

[0259] }

[0260] like Figure 5 As shown, the execution efficiency comparison between the traditional five-stage pipeline and the optimized pipeline is shown. The traditional method requires 25 clock cycles to execute 5 load instructions, while the method of the present invention only requires 9 clock cycles.

[0261] An embodiment of the present invention also provides a SIMD instruction arrangement device based on the RISC-V architecture, comprising one or more processors for implementing the SIMD instruction arrangement method based on the RISC-V architecture.

[0262] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present invention. A person of ordinary skill in the art can understand and implement the present invention without inventive work.

[0263] An embodiment of the present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the SIMD instruction arrangement method based on the RISC-V architecture.

[0264] An embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the SIMD instruction arrangement method based on the RISC-V architecture is implemented.

[0265] The computer-readable storage medium may be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or memory. The computer-readable storage medium may also be any device with data processing capabilities, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. equipped on the device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of any device with data processing capabilities and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and may also be used to temporarily store data that has been output or is to be output.

[0266] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the contents disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only.

[0267] It will be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.< / instruction> < / instruction> < / instruction>

Claims

1. A SIMD instruction arrangement method based on RISC-V architecture, characterized in that: The steps include: Identify and analyze SIMD-applicable parts of a computational process as computational tasks; the computational process is a matrix operation or activation function processing process in a deep learning model, and the applicable parts include vector normalization, ReLU activation function, Softmax, Pooling operations, and their fusion operators; Constructing a basic code template according to the computing task, wherein the basic code template includes a data type parameter T, a vector width parameter VW, a memory access mode parameter MP, and a computing mode parameter CP; Query the SIMD interface manual and related documents to obtain the SIMD calculation structure corresponding to the basic code, and map the operations in the basic code to the SIMD calculation structure, including instruction set adaptation, dynamic register grouping, and memory access mode optimization; the SIMD calculation structure includes vector multiplication and addition, vector comparison, and vector type conversion; Based on the SIMD computing structure, SIMD instructions are designed according to the vector extension characteristics of the RISC-V architecture, and the SIMD instructions include vector width configuration, data layout optimization, and five loop unrolling; Optimizing the SIMD instruction execution process through an instruction reordering strategy, wherein the instruction reordering strategy includes: constructing an instruction dependency graph and calculating the RAW / WAW dependency distance between instructions; A scheduling algorithm based on dependency distance matrix is ​​used to execute instructions of the same type together.

2. A SIMD instruction arrangement method based on RISC-V architecture according to claim 1, characterized in that: Mapping the operations in the basic code to the SIMD computing structure includes: Create a mapping table from mathematical operations to target platform SIMD instructions, including: FP16 comparison operations are mapped to simd_vfcmpleh instructions; Conditional selection operations are mapped to simd_vfselleh instructions; Type conversion operations are mapped to simd_vfcvtsh or simd_vfcvths instructions; Dynamically divide 32 vector registers into input buffer, calculation workspace and output buffer.

3. The SIMD instruction arrangement method based on the RISC-V architecture according to claim 1, characterized in that: The dynamic grouping of registers includes dividing 32 vector registers into: Input buffer, 4 groups × 4 registers; Computation workspace, 2 groups × 8 registers; Output buffer, 2 groups × 4 registers; The register overflow problem is solved by the Chordal graph coloring algorithm, and the vector length is dynamically adjusted using the vsetvli instruction.

4. The SIMD instruction arrangement method based on the RISC-V architecture according to claim 1, characterized in that: The SIMD instruction includes a multi-operator fusion implementation, and the multi-operator fusion implementation includes: LayerNorm fusion implementation includes variance calculation and normalization operations, and uses simd_reduc_plush for reduction and summation; Softmax implementation includes maximum reduction, exponential calculation and normalization operations, using simd_smaxh and simd_vfmah instructions; The ReLU and Pooling fusion operations are implemented through the simd_vfselleh and simd_vfaddh instructions.

5. The SIMD instruction arrangement method based on RISC-V architecture according to claim 1, characterized in that: The five-loop expansion includes: Divide the loop body into five parallel execution units, each unit processes VW data elements; Use compilation instructions to control the number of expansions; Perform a sequence of Load-Compute-Store operations in each iteration of the unroll; Eliminate cross-iteration register conflicts through register rotation technology.

6. A SIMD instruction arrangement method based on RISC-V architecture according to claim 5, characterized in that: The values ​​of the VW data elements are dynamically determined according to the SIMD bit width of the processor. When the processing unit width is 128 bits, VW=16; when the processing unit width is 256 bits, VW=32.

7. A SIMD instruction arrangement device based on RISC-V architecture, characterized in that: include: A processor, configured to execute a SIMD instruction arrangement method based on a RISC-V architecture as claimed in any one of claims 1 to 6; A memory for storing program data, wherein the program data includes a SIMD instruction template library, a loop unrolling optimization module, and an instruction reordering scheduling module; wherein the SIMD instruction template library supports vectorized operations on FP16 and INT4 data, the loop unrolling optimization module supports configurations of five or more unrolling times, and the instruction reordering scheduling module implements an instruction reordering algorithm based on a dependency distance matrix.

8. An electronic device comprising a memory and a processor, characterized in that: The memory is coupled to the processor; The memory is used to store program data, and the processor is used to execute the program data to implement the SIMD instruction arrangement method based on the RISC-V architecture according to any one of claims 1 to 6; The program data includes a SIMD interface adaptation layer, which supports simd_load, simd_store and simd_vfselleh interface calls.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements a SIMD instruction arrangement method based on the RISC-V architecture as described in any one of claims 1 to 6.

Images