A processing method of a reduction operator and a vector processor

Abstract

The application provides a processing method of a reduction operation operator and a vector processor. The processing method of the reduction operation operator is applied to the vector processor and comprises the following steps: receiving a processing request of a reduction operation operator; if the reduction operation operator belongs to a target class operator, performing parallel operation processing on data in corresponding positions of each data block corresponding to the reduction operation operator to obtain a target data block; wherein the operation processing corresponds to an operation code corresponding to the reduction operation operator; and performing operation processing on each data of the target data block to obtain target data. The processing method of the reduction operation operator and the vector processor provided in the embodiment of the application improve the data processing efficiency of the vector processor.

Description

Technical Field

[0001] This invention relates to the field of chip technology, and more specifically to a method for processing reduction operators and a vector processor. Background Technology

[0002] The Argmax (Argument of Maximum) and argmin (Argument of Minimum) operators are fundamental and crucial choice operators in mathematics and computer science. The Argmax operator is used to quickly locate and return the index of the maximum value in a given set, array, or probability distribution, while the argmin operator is used to quickly locate and return the index of the minimum value in a given set, array, or probability distribution, making them suitable for rapid decision-making scenarios involving high-dimensional data.

[0003] In large models (such as Transformer and GPT series), the argmax and argmin operators, as fundamental and frequently used operations, are widely applied in key stages such as classification prediction, sequence decoding, sparse selection in attention mechanisms, and action decision-making in reinforcement learning, playing an irreplaceable role, especially in each prediction stage of text generation. In the Single Instruction Multiple Data (SIMD) hardware architecture based on RISC-V Vector Extension (RVV), the efficient implementation of the argmax and argmin operators is crucial to unleashing the potential of vector computation. In existing technologies, RVV-based processors typically rely on a combination of vector comparison and merging instructions when executing the argmax operator. A divide-and-conquer approach is often used, dividing a long data vector into multiple segments. Then, within each segment, a combination of vector comparison, masking operations, and reduction instructions is used to calculate the local maximum and index. Finally, the maximum value and its corresponding index are found from the local maximums of all data segments. Due to the reliance on multiple iterations or reduction operations across vector segments, the instruction sequence is long and the control logic is complex.

[0004] Therefore, how to reduce the instruction overhead when executing argmax / argmin operators has become an important issue that urgently needs to be addressed in this field. Summary of the Invention

[0005] To address the problems in the prior art, embodiments of the present invention provide a data processing method and vector processor based on a single vector instruction, which can at least partially solve the problems existing in the prior art.

[0006] In a first aspect, the present invention proposes a method for processing reduction operators, applied to a vector processor, comprising: Receive processing requests from reduction operation operators; If the reduction operator belongs to the target type operator, then the data at the corresponding positions of each data block corresponding to the reduction operator are processed in parallel to obtain the target data block; wherein, the processing corresponds to the opcode corresponding to the reduction operator. The target data block is traversed and processed to obtain the target data.

[0007] Secondly, the present invention proposes a vector processor for executing the processing method of the reduction operation operator described in any of the above embodiments.

[0008] Thirdly, the present invention provides an artificial intelligence chip, including the vector processor described in the above embodiments.

[0009] Fourthly, the present invention provides a board including at least one artificial intelligence chip as described in the above embodiments.

[0010] Fifthly, the present invention provides an electronic device, including at least one artificial intelligence chip as described in the above embodiments or at least one board as described in the above embodiments.

[0011] The present invention provides a method for processing reduction operators and a vector processor that receives a processing request for a reduction operator. If the reduction operator is a target type operator, the corresponding data in each data block corresponding to the reduction operator is processed in parallel to obtain a target data block. The processing corresponds to the opcode of the reduction operator. The method iterates through each data in the target data block to obtain the target data. Since parallel processing is performed on the corresponding data in each data block before processing, the instruction overhead of the vector processor is reduced, and the data processing efficiency of the vector processor is improved. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a schematic diagram of the vector register file structure in a RISC-V vector processing unit provided in an embodiment of the present invention.

[0013] Figure 2This is a schematic diagram of the execution flow of Element-wise instructions in a vector processor according to an embodiment of the present invention.

[0014] Figure 3 This is a schematic diagram of the execution flow of the Reduction instruction in a vector processor according to an embodiment of the present invention.

[0015] Figure 4 This is a flowchart illustrating a method for processing reduction operators according to an embodiment of the present invention.

[0016] Figure 5 This is a schematic diagram of the implementation process of the argmax operator in the existing technology.

[0017] Figure 6 This is a flowchart illustrating a method for processing reduction operators provided in another embodiment of the present invention.

[0018] Figure 7 This is a flowchart illustrating a method for processing reduction operators provided in another embodiment of the present invention.

[0019] Figure 8 This is a flowchart illustrating a method for processing reduction operators provided in another embodiment of the present invention.

[0020] Figure 9 This is a schematic diagram of the implementation process of the argmax operator provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Here, the illustrative embodiments and descriptions of the present invention are used to explain the present invention, but are not intended to limit the present invention. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other. The acquisition, storage, use, and processing of data in the technical solutions of this application all comply with the relevant provisions of laws and regulations. The user information in the embodiments of this application is obtained through legal and compliant means, and the acquisition, storage, use, and processing of user information have been agreed upon by the customer.

[0022] To facilitate understanding of the technical solution provided in this application, the research background of the technical solution in this application will be briefly explained below.

[0023] In tasks such as natural language generation, machine translation, speech recognition, and image classification, the argmax and argmin operators have become core foundational technologies for ensuring model generation accuracy, improving inference efficiency, and enhancing output controllability by accurately selecting optimal candidate results, simplifying decision logic, and supporting greedy decoding strategies. Whether selecting the next token with the highest probability from the vocabulary or focusing on key input information in attention mechanisms, the argmax and argmin operators provide direct and reliable computational support for model decision-making. However, in practical applications, the argmax and argmin operators suffer from high instruction overhead, low pipeline efficiency, and increased data dependencies leading to more bubbles, making it difficult to meet the demands for low latency and high throughput computation in large-scale data scenarios. Especially in applications such as deep learning inference and signal processing, frequent calls to the argmax / argmin operators can slow down model execution and become a bottleneck in system performance.

[0024] Traditional processors, when executing argmax / argmin operators, need to compare elements one by one and maintain the current extremum and its position, which is difficult to meet the throughput requirements of real-time inference. Existing RVV software libraries rely heavily on general vectorized loops or staged reduction for argmax / argmin support, lacking deep pipelining optimizations tailored to operator characteristics and hardware features, thus failing to fully leverage RVV's potential in data parallelism and memory bandwidth.

[0025] VLEN (Vector Register Length): Represents the total bit width of the vector register.

[0026] LMUL (Length Multiplier): Indicates the number of registers used in a single operation of a vector instruction. For example, LMUL=1 means using 1 register, and LMUL=2 means concatenating 2 consecutive registers into a longer logical vector.

[0027] SEW (Selected Element Width): Represents the data width of each vector element (e.g., 8 bits, 16 bits, 32 bits).

[0028] DLEN (Data Length): Represents the data bit width of the physical SIMD execution unit in the hardware architecture. It is usually smaller than VLEN. For example, an RVV processor that supports VLEN=4096 may only have a SIMD bit width of 1024 bits. This design aims to strike a balance between area, power consumption, and performance. Full-width SIMD units would significantly increase chip area and power consumption.

[0029] When the SIMD cell width is VLEN / N, processing one vector register data requires N cycles. Hardware typically implements this using a pipelined approach. Although one vector register data needs to be split into N batches for processing, the processing of different batches can be pipelined in parallel, thereby improving the throughput of SIMD.

[0030] Increasing LMUL increases the data width of the register. Since the underlying hardware still processes data in segments with a fixed SIMD width, the total number of instruction execution cycles will also increase accordingly when LMUL is increased.

[0031] For an RVV processor with VLEN=4096 and DLEN=1024, processing one vector register data takes 4 cycles when LMUL=1; when LMUL=4, processing one vector register data will take 4x4=16 cycles.

[0032] Figure 1 This diagram illustrates the structure of the vector register file in a RISC-V vector processing unit. The vector register file comprises 32 sets of architecture registers, which are aggregated under the control of a vector register length multiplication factor, enabling flexible adjustment of the effective length of the vector registers. For example: When LMUL=1, the architecture registers V0, V1, V2...V31 are independent and not aggregated; When LMUL=2, the architecture registers V0 and V1 are aggregated, V2 and V3 are aggregated...V30 and V31 are aggregated, and the vector length is doubled; When LMUL=4, the architecture registers V0-V3 are aggregated, V4-V7 are aggregated...V28-V31 are aggregated, and the vector length increases by 4 times; When LMUL=8, the aggregation of V0-V7, V8-V15, ..., V24-V31 increases the vector length by 8 times.

[0033] Hardware SIMD bit width: This refers to the bit width that a single SIMD instruction can actually be processed in parallel, represented by DLEN. A single architecture register can be divided into one or more domains, each with a bit width equal to the SIMD bit width. Through architecture registers composed of multiple domains, the hardware can support vector operations of different lengths (i.e., logic adapted to VLEN), such as scenarios where LMUL < 1. Assuming a single architecture register can be divided into four domains, the data from these four domains are executed sequentially on the hardware SIMD unit according to the actual situation, for example: When LMUL=1, all four fields in the architecture register are valid, and data in a single architecture register needs to be executed four times on the hardware SIMD. When LMUL=1 / 2, only the first two fields in the architecture register are valid, and data in a single architecture register needs to be executed twice on the hardware SIMD. When LMUL=1 / 4 or 1 / 8, only the first field of the architecture register is valid, and a single architecture register data needs to be executed once on the hardware SIMD. In the RVV instruction set, RVV instructions can be divided into element-wise instructions and fully vector-dependent instructions.

[0034] Element-wise instructions: These instructions perform the same operation independently and in parallel on each element in the vector register. There is no data dependency between the elements, so they can be executed in parallel naturally; such as addition instructions, multiplication instructions, comparison instructions, etc.

[0035] Fully vector-dependent instructions: The output of these instructions depends on all valid elements of the entire input vector. They typically "compress" a vector into a scalar (or a smaller data structure); such as reduction instructions (e.g., vredmax, vredusum, vredand), and other globally dependent operation instructions (e.g., vfirst).

[0036] The following section explains in detail the significant differences in hardware execution efficiency between element-wise instructions and fully vector-dependent instructions in an RVV-based segmented pipelining architecture.

[0037] Figure 2 This demonstrates the execution flow of element-wise instructions in a vector processor: 1. Load Phase: Loading source data At time T+0, Load unit 0 and Load unit 1 start in parallel, reading two sets of source operands from memory. Each Load unit can transfer 1024 bits of data per cycle. At time T+3, the first set of 1024 bits of data is written to a (C0) field of vector registers V0 and V1; subsequently, at times T+4 to T+6, the second to fourth sets of 1024 bits of data are written to the C1 to C3 fields of V0 and V1 respectively.

[0038] 2. Element-wise computation stage: Domain-by-domain segment operation

[0039] Once the C0 field of vector registers V0 and V1 is ready at time T+3, the element-wise operation begins reading data from the C0 field of vector registers V0 and V1 at time T+4, and then immediately releases the C0 field. The calculation result is completed after T+6 and written to the C0 field of vector register V2 at time T+8. Subsequent fields (C1 to C3) become ready sequentially at times T+4 to T+6, and the element-wise operation accordingly acquires input at times T+5 to T+7 and produces the corresponding results at times T+7 to T+9.

[0040] 3. Storage (Store) stage: Results are written back to memory.

[0041] At time T+8, the data in the C0 field of vector register V2 is ready, triggering Store unit 0 to start writing the result back to memory. The data is retrieved by Store unit 0 at time T+9, and the C0 field of vector register V2 is released; subsequent fields (C1 to C3) are ready sequentially at times T+9 to T+11, and are retrieved by Store unit 0 and written to memory at times T+10 to T+12, while the corresponding fields are released.

[0042] 4. Overlapping and continuous processing of production lines

[0043] Since the C0-C3 fields of vector registers V0 and V1 are released successively between T+4 and T+7, the Load unit can reload the next set of source data into the C0-C3 fields of vector registers V0 and V1 between T+7 and T+10. The newly loaded data is used by element-wise operations between T+8 and T+11, and the calculation results are generated between T+10 and T+13. At this time, the C0-C3 fields of vector register V2 have been completely released between T+9 and T+12, so the newly generated results can be written to the corresponding fields of vector register V2 between T+12 and T+15.

[0044] The entire element-wise operation execution process is highly overlapping between various functional units, forming a continuous pipeline processing. Data in each register field is "used and released immediately" to make room for subsequent data loading, so that no cavitation occurs in the entire pipeline, thus enabling a high-throughput element-wise operation pipeline.

[0045] Figure 3 This demonstrates the execution flow of the Reduction instruction (a typical fully vector-dependent instruction) in a vector processor: 1. Load Phase: Loading source data At time T+0, Load Unit 0 and Load Unit 1 start in parallel, loading the two sets of source operands required for the Reduction instruction from memory: Load Unit 0 loads a set of vector data and stores it in vector register V0; Load Unit 1 loads an initial scalar value and stores it in the C0 field of vector register V1. Each memory access unit can move 1024 bits of data per cycle. At time T+3, the first 1024-bit vector data is written to the C0 field of vector register V0, and at the same time, the scalar data is written to the C0 field of vector register V1. Subsequently, from time T+4 to T+6, Load Unit 0 writes the subsequent vector data sequentially to the C1 to C3 fields of vector register V0.

[0046] 2. Reduction Calculation Stage: Segment-by-segment accumulation

[0047] Once the C0 field of vector registers V0 and V1 is ready at time T+3, the Reduction operation begins reading data from vector register C0 at time T+4 and immediately releases the C0 field of vector registers V0 and V1. Then, between times T+5 and T+7, the Reduction operation sequentially retrieves the vector data from the C1 to C3 fields of vector register V0, completing the reduction calculation for the entire vector. The final result, a scalar value, is generated at time T+9.

[0048] 3. Result write-back: Multiplexed register

[0049] Since the initial scalar stored in vector register V1 was released at time T+4, the output scalar result generated at time T+9 can reuse vector register V1. This result is written to the C0 field of vector register V1 at time T+11 as the input scalar for the next reduction operation.

[0050] 4. Delayed execution of the second reduction (pipeline cavitation)

[0051] Although the vector data required for the second reduction has been fully loaded between T+7 and T+10 (C0 to C3 of vector register V0 are ready), its scalar input depends on the output of the previous reduction operation, which is not available until T+11. Therefore, the second reduction operation can only begin execution at T+12, resulting in a four-cycle pipeline cavitation between T+7 and T+11. This cavitation is caused by the chained dependency of the scalar results, limiting the parallelism of the reduction instructions.

[0052] 5. Subsequent data loading delay

[0053] The second reduction operation uses the C0-C3 fields of vector register V0 between T+12 and T+15, and these fields are occupied during this period. Therefore, Load unit 0 can only start loading the next batch of vector data after T+15. The next batch of vector data will be gradually written to the C0-C3 fields of vector register V0 between T+15 and T+18. In addition, scalar data will not be valid until T+19, thus further delaying the start time of the third reduction.

[0054] While reduction instructions can efficiently process vector data, their output depends on the previous scalar output, forming a strong dependency chain. This cross-cycle scalar result dependency significantly impacts pipeline efficiency, preventing fully pipelined continuous execution, causing pipeline bubbling, and reducing overall throughput. Bubbling becomes more severe as the ratio between the width of the vector register data and the width of the SIMD data increases. For example, in architectures like RVV, increasing the LMUL to concatenate multiple physical vector registers into a wider logical vector register allows for the processing of longer vector data, but it also means that a single reduction operation needs to traverse more domains (such as C0 to C7 or even more) to complete the reduction of the entire vector.

[0055] This application analyzes the characteristics of the SIMD vector computation unit of RISC-V vector extension and the hardware implementation characteristics of commonly used RVV instructions, and proposes a method for processing reduction operators. It can realize a bubble-free pipelined parallel scheme for argmax / argmin operators, and construct a dedicated computation pipeline with low latency and high throughput, which is especially suitable for processing reduction operators such as argmax and argmin operators with large data volumes.

[0056] Figure 4 This is a flowchart illustrating a method for processing reduction operators according to an embodiment of the present invention, as shown below. Figure 4 As shown, the method for processing reduction operators provided in this embodiment of the invention includes: S401, Receive processing request for reduction operation operator; Specifically, the vector processor can receive processing requests for reduction operators, and then determine whether the reduction operator belongs to the target class operator. If the reduction operator is an operator in the target class operator, then the reduction operator belongs to the target class operator. If the reduction operator is not an operator in the target class operator, then the reduction operator does not belong to the target class operator.

[0057] The target operators are predefined and include, but are not limited to, the argmax operator, the argmin operator, the summation reduction operator, the logical AND reduction operator, and the logical OR reduction operator. These are used to reduce the target data based on the data to be processed. The summation reduction operator performs an element-wise summation operation on the data to be processed to obtain the summed result. The logical AND reduction operator performs a logical AND operation on each element of the data to be processed to obtain the aggregated logical AND result. The logical OR reduction operator performs a logical OR operation on each element of the data to be processed to obtain the aggregated logical OR result.

[0058] The implementation process of target-type operators in the prior art divides the data to be processed into blocks. For each data block, the corresponding local reduction instruction is used to calculate the local reduction result of the data block. Since the local reduction instruction is a fully vector-dependent instruction, its execution process is a serialized reduction process. The output local reduction result can only be generated after all the data in the entire data block has participated in the operation, and it cannot overlap with other operations. Furthermore, other related processing operations can only be performed after the local reduction instruction is completed, forming a strict sequential dependency, which reduces the execution efficiency of the target-type operator.

[0059] Figure 5 This paper demonstrates the conventional implementation flow of the argmax operator in the RISC-V Vector Extension (RVV) architecture. The implementation process of the argmax operator in the prior art involves processing the data in blocks and using fully vector-dependent instructions (such as reduction and vfirst) provided by RVV to calculate local maxima and indices block by block, finally aggregating to obtain the global maxima and its index. The implementation process of the argmax operator in the prior art is as follows.

[0060] The first step is to divide the data to be processed. The data to be processed includes m data points. A single vector instruction can process n data points, meaning a single vector instruction can process n data points. Therefore, the data to be processed is divided into x = ... There are x data blocks, namely the 0th data block, the 1st data block, the 2nd data block, ..., the (x-1)th data block.

[0061] Step 2: Initialize the historical maximum value and index. Set the initial value of the historical maximum value to negative infinity (-inf) to record the currently known maximum value; set the index (q) of the historical maximum value to 0 to record the position corresponding to the historical maximum value.

[0062] The third step is to calculate the local maximum of the current data block. The i-th data block is taken as the current data block, and the local reduction instruction `vredmax` is used to calculate the local maximum of the current data block. The local reduction instruction `vredmax` is a fully vector-dependent instruction, requiring traversal of the entire data block to output the local maximum, resulting in a relatively long latency. The initial value of `i` is 0.

[0063] Step 4: Determine if the local maximum value of the current data block is greater than the historical maximum value. Compare the local maximum value of the current data block with the historical maximum value. If the local maximum value of the current data block is less than or equal to the historical maximum value, proceed to step 9; if the local maximum value of the current data block is greater than the historical maximum value, proceed to step 5.

[0064] Step 5: Update the historical maximum value. Update the historical maximum value based on the local maximum value of the current data block.

[0065] Step 6: Mark the position of the historical maximum value. Using comparison instructions, compare each data point in the current data block with the historical maximum value, generate a mask corresponding to the historical maximum value, and mark the position where the data equals the historical maximum value.

[0066] Step 7: Obtain the index position of the historical maximum value in the current data block. Using the vector mask processing instruction vfirst, find the first marker value of 1 in the mask corresponding to the historical maximum value, and return the index position of the marker value 1 as the index position c of the historical maximum value in the current data block.

[0067] Step 8: Update the index of the historical maximum value. Update the index of the historical maximum value to the starting offset of the current data block + c, that is, the index of the historical maximum value p = i × x + c.

[0068] Step 9: Determine if there are any uncompared data blocks. Compare i with x-1. If i is less than x-1, then there are still uncompared data blocks. Update i = i+1 and return to step 3. If i equals x-1, then the comparison of x data blocks is complete, and proceed to step 10.

[0069] Step 10: Output the index of the maximum value. Output the index of the maximum value of the data to be processed.

[0070] In existing implementations of the argmax operator, a local reduction instruction (vredmax) is used to calculate the local maximum value for each data block. The execution of vredmax is a serialized reduction process; its output local maximum value can only be generated after all data in the entire data block has participated in the computation, preventing overlap with other operations. Furthermore, comparison operations and vector masking can only proceed after the local reduction instruction is completed, creating a strict sequential dependency. More importantly, the execution of the local reduction instruction for the next data block must wait for the completion of the local reduction instruction for the previous data block, because the update of the global maximum value index depends on the comparison results of the previous data block.

[0071] The implementation of the argmax operator in existing technologies can cause severe pipeline blockage, especially in scenarios with high LMUL or large vector width, where the cavitation cycle increases significantly.

[0072] S402. If the reduction operation operator belongs to the target type operator, then the data at the corresponding positions of each data block corresponding to the reduction operation operator are processed in parallel to obtain the target data block; wherein, the processing corresponds to the opcode corresponding to the reduction operation operator. Specifically, if the reduction operator belongs to the target class operator, the vector processor performs parallel operations on the data at corresponding positions of each data block corresponding to the reduction operator to obtain the target data block. The vector processor can perform operations on the data at corresponding positions of two data blocks in parallel. These operations include, but are not limited to, addition, multiplication, logical AND, logical OR, maximum value calculation, and minimum value calculation, etc., and are set according to actual needs; this embodiment of the invention does not impose limitations. The operations correspond to the opcodes of the reduction operator. Each data block corresponding to the reduction operator can be obtained by dividing the data to be processed corresponding to the reduction operator into blocks.

[0073] For example, the reduction operator is the argmax operator, used to find the maximum value of data in each data block. The vector processor performs parallel comparisons of the data at corresponding positions in two data blocks, retaining the larger value at each position. After all data blocks have been compared, the target data block can be obtained. The target data block is the maximum value data block, and the data at each position of the maximum value data block is the maximum value of the data at the corresponding position in each of the other data blocks.

[0074] For example, the reduction operator is a logical AND reduction operator, used to calculate the logical AND result of the data in each data block. The vector processor performs logical AND operations on the corresponding positions of the two data blocks in parallel, retaining the logical AND operation results at each position. After all data blocks have undergone logical AND operations, the target data block can be obtained. The data at each position of the target data block is the logical AND operation result of the data at the corresponding positions of the other data blocks.

[0075] S403. Traverse each data in the target data block and perform calculations to obtain the target data.

[0076] Specifically, after obtaining the target data block, the vector processor traverses each piece of data in the target data block and performs calculations to obtain the target data.

[0077] For example, if the goal is to find the maximum value, the vector processor will iterate through each data element in the target data block, comparing them to obtain the maximum value as the target data. The target data is the maximum value of all data elements in the target data block, and also the maximum value in each data block.

[0078] For example, if the goal is to obtain the result of a logical AND operation, the vector processor will traverse each piece of data in the target data block, performing the logical AND operation to obtain a final value as the target data. The target data is the result of the logical AND operation on each piece of data in the target data block, and also the result of the logical AND operation on the data in each individual data block.

[0079] This application improves the implementation process of target-type operators. Compared with the implementation process of target-type operators in the prior art, it eliminates the need for local reduction instructions for each data block. Instead, a local reduction instruction is used only once in step S403, meaning the fully vector-dependent instruction is called only once, thus avoiding pipeline bubbling problems caused by fully vector-dependent instructions. For the computational processing of data in each data block, parallel computation is performed on the data at corresponding positions. This can be achieved using element-wise operation instructions, fully utilizing the high parallelism and low latency characteristics of element-wise instructions, significantly improving the overall throughput while ensuring the correctness of the output results.

[0080] The method for processing reduction operators provided in this embodiment of the invention receives a processing request for a reduction operator; if the reduction operator belongs to a target type operator, then the data at corresponding positions of each data block corresponding to the reduction operator are processed in parallel to obtain a target data block; wherein the processing corresponds to the opcode corresponding to the reduction operator; the processing is performed by traversing each data of the target data block to obtain target data. Since the data at corresponding positions of each data block are processed in parallel first to obtain a target data block before processing, the instruction overhead of the vector processor can be reduced and the data processing efficiency of the vector processor can be improved.

[0081] Figure 6 This is a flowchart illustrating a method for processing reduction operators according to another embodiment of the present invention, as shown below. Figure 6 As shown, based on the above embodiments, further, the step of performing parallel computation processing on the data at corresponding positions of each data block corresponding to the reduction operation operator to obtain the target data block includes: S601, The vector processor obtains a data block from each data block to initialize the historical target data block; Specifically, a data block is obtained from each of the data blocks to initialize the historical target data block, that is, the data of the obtained data block is the initial value of the data at the corresponding position of the historical target data block.

[0082] S602. Perform parallel computation on each data block of the remaining data block and the corresponding data of the historical target data block to obtain the computation result data block corresponding to each data block of the remaining data block. After obtaining a new computation result data block, update the historical target data block based on the latest obtained computation result data block until all data blocks included in the remaining data block have been processed. Herein, the remaining data block refers to the data block other than the data block used to initialize the historical target data block. Specifically, the vector processor retrieves a data block from the remaining data blocks and performs parallel computation on the corresponding positions of the historical target data block. This allows for parallel computation on the corresponding positions of two data blocks, retaining the computation results to obtain a result data block. The historical target data block is then updated based on this result data block, replacing the data in the historical target data block with the data at the corresponding positions of the result data block. Next, another data block is retrieved from the remaining data blocks and performed parallel computation on the corresponding positions of the updated historical target data block, retaining the computation results to obtain a new result data block. This new result data block is then used to update the historical target data block. This process is repeated until all data blocks within the remaining data blocks have been processed with the historical target data block, resulting in a final result data block. This final result data block is obtained by processing the last remaining data block that has not yet been processed with the historical target data block.

[0083] The remaining data blocks refer to the data blocks other than those used to initialize the historical target data blocks.

[0084] S603. The final calculation result data block is used as the target data block.

[0085] Specifically, the last block of data resulting from the operation, i.e. the final block of data resulting from the operation, is used as the target data block.

[0086] In this embodiment of the invention, the data at corresponding positions of two data blocks are processed in parallel. Since the parallel operation uses element-wise instructions, the use of fully vector-dependent instructions is avoided, reducing the instruction overhead of the vector processor.

[0087] In one embodiment, the reduction operator is used to obtain the maximum value of each data block. Assume there are a total of 7 data blocks, namely the 0th data block, the 1st data block, the 2nd data block, the 3rd data block, the 4th data block, the 5th data block, and the 6th data block.

[0088] The vector processor initializes the historical maximum value data block by acquiring the 0th data block. The vector processor then uses a "larger value" instruction to compare the data at corresponding positions in the 1st data block and the historical maximum value data block in parallel. Specifically, it compares the data at position 1 of the 1st data block with the data at position 1 of the historical maximum value data block, the data at position 2 of the 1st data block with the data at position 2 of the historical maximum value data block, the data at position 3 of the 1st data block with the data at position 3 of the historical maximum value data block, and so on, up to comparing the data at position 20 of the 1st data block with the data at position 20 of the historical maximum value data block. This yields the operation result data block corresponding to the 1st data block. Each position in the operation result data block represents the larger value between the corresponding positions of the 1st data block and the historical maximum value data block. The "larger value" instruction is used to compare the data at corresponding positions of the two data blocks in parallel, taking the larger value at each corresponding position to form the comparison result data block. The vector processor updates the historical maximum value data block based on the operation result data block corresponding to the 1st data block.

[0089] The result data block compares the corresponding data positions of the second data block and the historical maximum value data block in parallel using a larger value instruction. The specific comparison process is similar to that of the first data block and the historical maximum value data block, and will not be elaborated here. This yields the result data block corresponding to the second data block. The vector processor then updates the historical maximum value data block based on the result data block corresponding to the second data block.

[0090] The vector processor uses a larger-value instruction to compare the data at the corresponding positions of the third data block with the historical maximum value data block in parallel, obtaining the operation result data block corresponding to the third data block. The vector processor then updates the historical maximum value data block based on the comparison result data block corresponding to the third data block.

[0091] This process continues until the result data block corresponding to the 6th data block is obtained. The result data block corresponding to the 6th data block is the target maximum value data block.

[0092] Figure 7 This is a flowchart illustrating a method for processing reduction operators according to another embodiment of the present invention, as shown below. Figure 7 As shown, based on the above embodiments, further, the step of performing parallel computation processing on the data at corresponding positions of each data block corresponding to the reduction operation operator to obtain the target data block includes: S701. If it is determined that the total number of data blocks of each data block is greater than or equal to 4, then initialize the operation dataset based on each data block, wherein the operation dataset includes an operation data block with a number greater than or equal to 4. Specifically, the vector processor determines whether the total number of data blocks in each data block is greater than or equal to 4. If it is greater than or equal to 4, then the computation dataset is initialized based on each data block, that is, each data block in each data block is used as a computation data block in the computation dataset. If the total number of data blocks in each data block is less than 4, then the processing flow of steps S601 to S603 can be directly adopted.

[0093] S702. Obtain 2n operation data blocks from the operation dataset and perform parallel operation processing to obtain n intermediate operation data blocks; wherein, the data at corresponding positions of every two operation data blocks in the 2n operation data blocks are processed in parallel; n is a positive integer greater than or equal to 1 and less than or equal to N, where N represents the number of operation units in the vector processor that can perform parallel operation processing, and N is greater than or equal to 2; Specifically, the vector processor obtains 2n operation data blocks from the operation dataset and allocates them to n operation units. Each operation unit performs parallel operations on the corresponding positions of the two operation data blocks, and the n operation units perform parallel operations on the 2n operation data blocks. n is a positive integer greater than or equal to 1 and less than or equal to N, where N represents the number of operation units in the vector processor capable of parallel processing, and N is greater than or equal to 2.

[0094] In one embodiment, when M / 2 is greater than N, n is N; when M / 2 is less than or equal to N, n is the result of rounding down M / 2. Here, M is the number of computational data blocks included in the computational dataset.

[0095] S703. Update the computation dataset based on the 2n computation data blocks and the n intermediate computation data blocks; Specifically, the vector processor removes the 2n processed data blocks from the computation dataset and adds the n intermediate data blocks obtained by processing the 2n processed data blocks to the computation dataset, thereby updating the computation dataset.

[0096] S704. Repeat the process of obtaining 2n operation data blocks from the operation dataset and performing parallel operation processing to obtain n intermediate operation data blocks, and updating the operation dataset based on the 2n operation data blocks and the n intermediate operation data blocks, until the updated operation processing dataset contains only one operation data block. Specifically, the vector processor continuously and repeatedly obtains 2n operation data blocks from the operation dataset and performs parallel operation processing to obtain n intermediate operation data blocks. The operation dataset is then updated based on the 2n operation data blocks and the n intermediate operation data blocks. That is, steps S702 and S703 are repeated. When the updated operation processing dataset contains only one operation data block, the operation processing of each data block ends.

[0097] S705. Take one of the operation data blocks included in the operation processing dataset obtained in the last update as the target data block.

[0098] Specifically, the vector processor uses the last updated data block of the computational processing dataset as the target data block. The last update of the computational processing dataset will only result in one computational data block.

[0099] In this embodiment of the invention, the data at corresponding positions of two data blocks are processed in parallel, and multiple computing units process the two data blocks in parallel. The parallel computing uses element-wise instructions, which avoids the use of fully vector-dependent instructions and reduces the instruction overhead of the vector processor, thereby further improving the data processing efficiency of the vector processor.

[0100] In one embodiment, the reduction operator is used to obtain the result of the logical AND operation of each data block, and the vector processor includes three arithmetic units. Assume there are a total of 9 data blocks, namely the 0th data block, the 1st data block, the 2nd data block, the 3rd data block, the 4th data block, the 5th data block, the 6th data block, the 7th data block, and the 8th data block.

[0101] The total number of data blocks is 9, which is greater than 4. The computation dataset is initialized based on 7 data blocks, and the computation dataset includes 9 computation data blocks.

[0102] In the first round of computation, since 9 / 2 is greater than 3, n is set to 3. Six computation data blocks are obtained from the above computation dataset and processed in parallel to obtain three intermediate computation data blocks. Each computation unit performs a logical AND operation on the corresponding positions of two computation data blocks in parallel to obtain the corresponding intermediate computation data blocks. After completing the computation processing of the six computation data blocks, the six computation data blocks are removed from the computation dataset, and the three intermediate computation data blocks are added to the computation dataset, thus updating the computation dataset. At this point, the computation dataset contains six computation data blocks.

[0103] In the second round of computation, since 6 / 2 equals 3, n is set to 3. Six computation data blocks are obtained from the first updated computation dataset and processed in parallel to obtain three intermediate computation data blocks. Each computation unit performs a parallel logical AND operation on the corresponding positions of two computation data blocks to obtain the corresponding intermediate computation data blocks. After completing the computation processing of the six computation data blocks, the six computation data blocks are removed from the computation dataset, and the three intermediate computation data blocks are added to the computation dataset, thus updating the computation dataset. At this point, the computation dataset contains three computation data blocks.

[0104] In the third round of computation, since 3 / 2 is less than 3, n is set to 1. Two computation data blocks are obtained from the updated computation dataset and processed in parallel to obtain one intermediate computation data block. A single computation unit performs a parallel logical AND operation on the corresponding positions of the two computation data blocks to obtain one corresponding intermediate computation data block. After processing the two computation data blocks, these two blocks are removed from the computation dataset, and the one intermediate computation data block is added to the computation dataset, thus updating the computation dataset. At this point, the computation dataset contains two computation data blocks.

[0105] In the fourth round of computation, since 2 / 2 is less than 3, n is set to 1. Two computation data blocks are obtained from the computation dataset updated in the third round and processed in parallel to obtain one intermediate computation data block. A single computation unit performs a parallel logical AND operation on the corresponding positions of the two computation data blocks to obtain one corresponding intermediate computation data block. After processing the two computation data blocks, they are removed from the computation dataset, and the one intermediate computation data block is added to the computation dataset, thus updating the computation dataset. At this point, the computation dataset contains one computation data block.

[0106] Use the last obtained data block as the target data block.

[0107] Based on the above embodiments, each data block is further obtained by dividing the data to be processed corresponding to the reduction operation operator according to the processing data volume of a single vector instruction.

[0108] Specifically, for the data to be processed corresponding to a given reduction operation operator, the vector processor can divide the data to be processed into multiple data blocks based on the amount of data processed by a single vector instruction. For example, if the data to be processed includes m data items, and the amount of data processed by a single vector instruction is k, where k is less than m, then the vector processor can divide the data to be processed into... One data block, This indicates rounding up m / k. The amount of data processed by a single vector instruction is predetermined.

[0109] Figure 8 This is a flowchart illustrating a method for processing reduction operators according to another embodiment of the present invention, as shown below. Figure 8 As shown, based on the above embodiments, the method for processing reduction operators provided in this embodiment of the invention further includes: S801. Obtain the first index position of the target data from the target data block; Specifically, the vector processor compares each piece of data in the target data block with the target data to obtain the positions of the data that are equal to the target data, and generates a first mask. The first mask marks the positions of the data in the target data block that are equal to the target data. From the first mask, the first valid position is obtained as the first index position of the target data.

[0110] In one embodiment, the vector processor obtains the maximum value 'a' of the target data block from the vector register using a maximum value instruction (e.g., vredmax). The vector register stores the target data block. The vector processor compares the maximum value 'a' with each data item in the target data block using a comparison instruction. If a data item in the target data block is equal to the maximum value 'a', the corresponding position in the first mask is marked as 1; if a data item in the target data block is not equal to the maximum value 'a', the corresponding position in the first mask is marked as 0. The vector processor uses a vector mask processing instruction (e.g., vfirst) to find the first marked value of '1' in the first mask. The position corresponding to the first marked value of '1' is the valid position, and the index position of the marked value '1' is returned as the first index position of the target data. The maximum value instruction, comparison instruction, and vector mask processing instruction are set according to actual needs, and this embodiment of the invention does not impose any limitations.

[0111] S802: Obtain the second index position of the target data based on the target data, the first index position of the target data, and each data block; Specifically, the vector processor obtains the data corresponding to the first index position of the target data from each data block, that is, it obtains the data at the first index position in each data block, and the data corresponding to the multiple first index positions obtained constitute a data vector.

[0112] The data vector is traversed, and each data point in the data vector is compared with the target data to obtain the positions of data points that are equal to the target data, generating a second mask. The second mask marks the positions of data points in the data vector that are equal to the target data. From the second mask, the first valid position is obtained and used as the second index position of the target data.

[0113] In one embodiment, the vector processor reads data at the first index position j from x data blocks and forms a vector B containing x data points. A comparison instruction compares the maximum value a with each data point in vector B. If a data point in vector B is equal to the maximum value a, then the corresponding position of that data point in the second mask is marked as 1; if a data point in vector B is not equal to the maximum value a, then the corresponding position of that data point in the second mask is marked as 0. A vector mask processing instruction finds the first marked value of 1 in the second mask and returns the index position of that marked value as the second index position of the maximum value a. The comparison instruction and the vector mask processing instruction are set according to actual needs, and this embodiment of the invention does not impose any limitations.

[0114] S803. Determine the data index corresponding to the target data based on the number of data blocks, the first index position, and the second index position.

[0115] Specifically, the vector processor determines the data index corresponding to the target data based on the number of data blocks in each data block, the first index position, and the second index position.

[0116] In one embodiment, the number of data blocks in each data block is x, the first index position of the target data is j, and the second index position is k. Then the index of the target data in the data to be processed is represented as: xk+j.

[0117] Based on the above embodiments, the opcode is further defined as an addition opcode, a multiplication opcode, a logical AND opcode, a logical OR opcode, a maximum value opcode, or a minimum value opcode.

[0118] Specifically, when the opcode is an addition opcode, the operation is performed as an addition operation. When the opcode is a multiplication opcode, the operation is performed as a multiplication operation. When the opcode is a logical AND opcode, the operation is performed as a logical AND operation. When the opcode is a logical OR opcode, the operation is performed as a logical OR operation. When the opcode is a maximum value opcode, the operation is performed as a maximum value operation. When the opcode is a minimum value opcode, the operation is performed as a minimum value operation.

[0119] This invention provides a vector processor for executing the reduction operation operator processing method described in any of the above embodiments.

[0120] The specific structure of the vector processor can be set according to actual needs, and this embodiment of the invention does not limit it.

[0121] This invention provides an artificial intelligence chip, including the vector processor described in the above embodiments.

[0122] This invention provides a board that includes at least one artificial intelligence chip as described in the above embodiments.

[0123] This invention provides an electronic device, including at least one artificial intelligence chip or at least one board as described in the above embodiments.

[0124] The electronic devices mentioned include, but are not limited to, artificial intelligence servers, mobile devices, handheld devices, vehicle-mounted devices, drones, monitoring equipment, cameras, and other devices with AI application requirements.

[0125] The following describes the specific implementation process of the reduction operation operator processing method provided in this embodiment of the invention, using the reconstruction of the argmax operator as an example. Figure 9 As shown.

[0126] The first step is to divide the data to be processed. The Argmax operator corresponds to m data points, and a single vector instruction can process k data points, meaning the amount of data processed by a single vector instruction is k. Therefore, the data to be processed is divided into x = There are x data blocks, namely the 0th data block, the 1st data block, the 2nd data block, ..., the (x-1)th data block.

[0127] The second step is to initialize the historical maximum data block. The 0th data block is read from memory, and the historical maximum data block is initialized.

[0128] The third step is to compare data blocks. The i-th data block is read from memory as the current data block. The data at corresponding positions in the current data block is compared in parallel with the data at the corresponding positions in the historical maximum data block. The larger value between the current data block and the historical maximum data block is obtained, forming the result data block corresponding to the i-th data block. The parallel comparison of corresponding positions in two data blocks can use a larger value instruction, which is a typical element-wise operation. The initial value of i is 1.

[0129] Step 4: Update the historical maximum data block. Based on the computation result data block corresponding to the i-th data block, update the historical maximum data block.

[0130] Step 5: Determine if there are any uncompared data blocks. Compare i with x-1. If i is less than x-1, then there are still uncompared data blocks, i = i+1, and return to step 3; if i equals x-1, then the comparison of x data blocks is complete, and proceed to step 6.

[0131] Step 6: Obtain the maximum value of the historical maximum data block. The maximum value can be obtained from the historical maximum data block using the `get maximum value` command, which retrieves the maximum value of each data element within the historical maximum data block.

[0132] Step 7: Obtain the first index of the maximum value. This can be done by comparing the maximum value with each data point in the historical maximum data block. If a data point in the historical maximum data block is equal to the maximum value, its corresponding position in the first mask is marked as 1; otherwise, it is marked as 0. Then, using the vector mask processing instruction `vfirst`, the first marked value of 1 is found in the first mask, and the index of this marked value is returned as the first index position `j` of the maximum value.

[0133] Step 8: Obtain the second index of the maximum value. Extract the data corresponding to the first index position j from x data blocks, forming a data vector B containing x data points. Using a comparison instruction, compare the maximum value with each data point in data vector B. If a data point in data vector B is equal to the maximum value, then the corresponding position of that data point in the second mask is marked as 1; if a data point in data vector B is not equal to the maximum value, then the corresponding position of that data point in the second mask is marked as 0. Then, using the vector mask processing instruction vfirst, find the first marked value of 1 in the second mask and return the index position of that marked value as the second index position k of the maximum value.

[0134] Step 9: Calculate the data index of the maximum value. Based on the number of data blocks x, the first index position j of the maximum value, and the second index position k of the maximum value, the data index corresponding to the maximum value can be determined. That is, the data index of the maximum value in the data to be processed is represented as: xk+j.

[0135] Step 10: Output the data index of the maximum value.

[0136] The improved implementation of the argmax operator in this invention, compared to the existing implementation, eliminates the need for local reduction instructions for each data block. Instead, it executes the maximum value instruction only once in step six, meaning the fully vector-dependent instruction is called only once, thus avoiding pipeline bubbling issues. For comparing two data blocks, corresponding data positions are compared in parallel using element-wise operation instructions. This fully leverages the high parallelism and low latency of element-wise instructions, significantly improving overall throughput while ensuring correctness.

[0137] The reduction operation operator processing method provided in this embodiment of the invention can significantly improve the execution efficiency of target-type operators such as argmax / argmin under the RVV architecture, and is especially suitable for large-scale data processing scenarios.

[0138] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0140] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0141] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0142] In the description of this specification, the references to terms such as "an embodiment," "a specific embodiment," "some embodiments," "for example," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0143] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for processing reduction operators, characterized in that, The hardware instruction execution layer applied to vector processors includes: Receive processing requests from reduction operation operators; If the reduction operator belongs to the target type operator, then the data at the corresponding positions of each data block corresponding to the reduction operator are processed in parallel to obtain the target data block; wherein, each data block includes data at multiple positions, and the data at each position of the target data block is the result of the operation processing of the data at the corresponding positions of each data block; the operation processing corresponds to the opcode corresponding to the reduction operator; The target data is obtained by traversing each piece of data in the target data block and performing calculations. The step of performing parallel computation on the data at corresponding positions of each data block corresponding to the reduction operator to obtain the target data block includes: Initialize the historical target data block by obtaining one data block from each of the aforementioned data blocks; Each data block of the remaining data block is processed in parallel with the corresponding data in the historical target data block to obtain the result data block for each data block of the remaining data block. After obtaining a new result data block, the historical target data block is updated based on the latest result data block until all data blocks included in the remaining data block have been processed. The remaining data block refers to the data blocks other than the data blocks used to initialize the historical target data block. The final result data block is used as the target data block.

2. The method according to claim 1, characterized in that, The step of performing parallel computation on the data at corresponding positions of each data block corresponding to the reduction operator to obtain the target data block includes: If it is determined that the total number of data blocks of each data block is greater than or equal to 4, then the operation dataset is initialized based on each data block, and the operation dataset includes an operation data block with a number greater than or equal to 4. 2n operation data blocks are obtained from the operation dataset and processed in parallel to obtain n intermediate operation data blocks; wherein, the data at corresponding positions of every two operation data blocks in the 2n operation data blocks are processed in parallel; n is a positive integer greater than or equal to 1 and less than or equal to N, where N represents the number of operation units in the vector processor that can perform operation processing in parallel, and N is greater than or equal to 2; The computation dataset is updated based on the 2n computation data blocks and the n intermediate computation data blocks; Repeat the process of obtaining 2n computation data blocks from the computation dataset in parallel and processing them to obtain n intermediate computation data blocks, and updating the computation dataset based on the 2n computation data blocks and the n intermediate computation data blocks, until the updated computation processing dataset contains only one computation data block; The target data block is one of the computational data blocks included in the last updated computational processing dataset.

3. The method according to claim 1, characterized in that, Each data block is obtained by dividing the data to be processed corresponding to the reduction operation operator based on the amount of data to be processed according to a single vector instruction.

4. The method according to claim 3, characterized in that, Also includes: Obtain the first index position of the target data from the target data block; Based on the target data, the first index position of the target data, and each data block, the second index position of the target data is obtained; The data index corresponding to the target data is determined based on the number of data blocks, the first index position, and the second index position.

5. The method according to claim 1, characterized in that, The opcode is an addition opcode, a multiplication opcode, a logical AND opcode, a logical OR opcode, a maximum value opcode, or a minimum value opcode.

6. A vector processor, characterized in that, The vector processor is used to execute the processing method of the reduction operation operator as described in any one of claims 1 to 5.

7. An artificial intelligence chip, characterized in that, Includes the vector processor as described in claim 6.

8. A circuit board, characterized in that, It includes at least one artificial intelligence chip as described in claim 7.

9. An electronic device, characterized in that, It includes at least one artificial intelligence chip as described in claim 7 or at least one board as described in claim 8.

Images