Top-K acceleration device and method based on RISC-V vector extension

Abstract

The invention provides a Top-K acceleration device and method based on RISC-V vector extension. The method comprises the following steps: generating configuration information in a configuration information lookup table through a software algorithm; the method comprises the following steps: in a hardware process, receiving a VTOPK instruction and decoding the VTOPK instruction so as to parse an operation parameter; according to the analyzed operation parameters, corresponding gating configuration information is loaded from the configuration information lookup table; according to the loaded gating configuration information, dynamically activating a comparison and exchange unit in the full-function double-modulation sequencing network through a gating clock technology; reading input vector data from the vector register file; performing anti-disruption processing and sequence processing on the input vector data; inputting the processed input vector data into the activated sorting network to carry out Top-K selection operation; and writing the result of the Top-K selection operation back to the vector register file so as to solve the problem that the Top-K selection algorithm cannot consider performance, flexibility and universality at the same time.

Description

Technical Field

[0001] This application relates to the field of hardware acceleration technology, and in particular to a Top-K acceleration device and method based on RISC-V vector extension. Background Technology

[0002] In edge computing fields such as autonomous driving and robotics, deep learning models based on the Mixture-of-Experts (MoE) structure have become a research hotspot due to their ability to significantly expand model capacity while maintaining a relatively constant inference computation. The core operation of the MoE model is to dynamically select the top K highest-scoring experts from N experts using the Top-K selection algorithm, thereby enhancing the model's expressive power. However, the highly diverse configurations of the number of experts (N) and the number of activations (K) in MoE models pose significant challenges to the computational efficiency and flexibility of Top-K operations in different application scenarios. Especially in resource-constrained edge devices (such as embedded systems and mobile terminals), achieving Top-K acceleration that balances high performance, low power consumption, and parameter adaptability has become a key requirement for the practical application of MoE models.

[0003] To improve the efficiency of Top-K computation, academia and industry have proposed various hardware acceleration solutions. These solutions can be mainly divided into two categories: one is based on fixed hardware sorting structures or application-specific integrated circuits (ASICs), which achieve high-throughput sorting under specific NK configurations through customized hardware; the other is based on field-programmable gate arrays (FPGAs) with dynamically configurable parameters, which support a limited range of parameter adjustments through hardware reconfiguration.

[0004] While the aforementioned solutions demonstrate effectiveness in specific scenarios, their inherent limitations severely restrict their widespread application in edge-side MoE models. Fixed hardware solutions lack adaptability to diverse NK configurations, making it difficult to achieve parameter flexibility while maintaining high performance. FPGA solutions, although offering some reconfigurability, lack software programmability, resulting in high porting costs and long development cycles. Furthermore, none of the relevant solutions effectively resolve the contradiction between dynamic power consumption and control simplification in Top-K operations, particularly when processing multi-precision data (such as integer INT4 / 8 / 16 and floating-point FP16 / 32), failing to provide a unified and efficient processing mechanism. Therefore, there is an urgent need for a Top-K acceleration architecture that deeply integrates software programmability and hardware efficiency, supports multi-parameter configuration, multi-data types, and offers controllable power consumption. Summary of the Invention

[0005] This application provides a Top-K acceleration device and method based on RISC-V vector extension to solve the problem that the Top-K selection algorithm cannot balance performance, flexibility and versatility.

[0006] The first aspect of this application provides a Top-K acceleration device based on RISC-V vector extension, comprising: The software algorithm section includes a reconfigurable simplified bitonic sorting network generator, which is used to generate configuration information in a configuration information lookup table. The configuration information is configuration information for multiple simplified bitonic sorting network structures. The hardware design includes: a custom instruction processing unit integrated into the RISC-V vector processor, including an instruction decoder configured to decode VTOPK instructions. The instruction code of the VTOPK instruction includes an operation parameter control field, which includes a sign control field, a bit width control field, a data type control field, a total length control field, and a total number of groups control field. The Top-K hardware functional unit is connected to the custom instruction processing unit and the vector register file. The Top-K hardware functional unit includes an instruction queue, a data processing pipeline, a top-level module of the sorting network, and a result queue. The top-level module of the sorting network includes a full-function bitone sorting network and the configuration information lookup table. The full-function bitone sorting network includes multiple comparison and exchange units, and the top-level module of the sorting network includes a gated clock circuit. The gated clock circuit controls the clock signal of the comparison and exchange units according to the gated configuration information loaded from the configuration information lookup table.

[0007] The software algorithm section uses a reconfigurable simplified bitone sorting network generator to generate configuration information in a lookup table. Combined with a custom instruction processing unit in the hardware design section, the VTOPK instruction is decoded to support flexible configuration of multiple parameters. The top-level module of the sorting network in the Top-K hardware functional unit dynamically controls the clock signal of the comparison and exchange unit according to the loaded gating configuration information. This reduces the dynamic power consumption of the sorting network, improves hardware resource utilization, and enhances adaptability to different input sizes and data types while maintaining the correctness of Top-K operation.

[0008] Optionally, the reconfigurable simplified bitonic sorting network generator includes a pruning module and a verification module; the pruning module is configured to iteratively attempt to remove each comparison and swap unit using the full-featured bitonic sorting network as an initial template; the verification module is configured to verify the functional correctness of the pruned network using a test vector set after each removal attempt, and permanently remove or retain the comparison and swap unit based on the verification result. The verification module employs a combination of full verification based on the zero-one principle, fast verification based on random sequences, and parallel verification.

[0009] The reconfigurable simplified bitone sorting network generator iteratively removes comparison and swap units through a pruning module, and then verifies the functional correctness of the pruned network through a verification module combining zero-one principle verification, fast verification based on random sequences, and parallel verification. This reduces the number of redundant comparison and swap units in the network while ensuring that the network output meets the Top-K constraints, which helps to reduce the power consumption and resource overhead of the final hardware sorting network and improves the efficiency of the network generation process.

[0010] Optionally, the verification module is configured to divide the test vector set into multiple subsets and use multiple computing cores to assign each subset to an independent verification process for parallel verification.

[0011] The verification module divides the test vector set into multiple subsets through parallel verification and uses multiple computing cores to perform parallel verification of independent processes. This ensures the correctness of the verification results while shortening the time required for functional verification during network generation.

[0012] Optionally, the VTOPK instruction is encoded in the OPIVX format of the RISC-V vector instruction set, and the operation parameter control field is encoded in the scalar register operand.

[0013] The VTOPK instruction is encoded in the OPIVX format of the RISC-V vector instruction set, and the operation parameter control fields are encoded in scalar register operands, thus maintaining compatibility with the existing RISC-V vector instruction set architecture. This helps reduce the complexity of compiler support and simplifies the software programming interface.

[0014] Optionally, the data processing pipeline includes a descrambling unit for restoring the input sequence from the vector processor data arrangement format to its natural order so that the correct sequence index can be generated subsequently.

[0015] The data processing pipeline restores the input sequence from the vector processor's specific data arrangement format to its natural order through a descrambling unit, thus providing the correct sequence order basis for subsequent index generation and data comparison by the top-level module of the sorting network, and supporting the accurate execution of Top-K operations.

[0016] Optionally, the top-level module of the sorting network further includes an index generation unit and a sequence complementation unit; the index generation unit is connected to the data processing pipeline and is used to generate an original position index for each element in the input sequence; the sequence complementation unit is connected to the index generation unit and the full-function bitonic sorting network and is used to perform complementation processing on input sequences whose length is less than the maximum processing length of the full-function bitonic sorting network.

[0017] The top-level module of the sorting network generates original position indices for the elements of the input sequence through the index generation unit to preserve the original order information of the elements. At the same time, it performs complementation on short sequences through the sequence complementation unit, thereby supporting the processing of input sequences of arbitrary length and adapting to full-featured bitone sorting networks with fixed input size, improving the versatility of the hardware unit and the integrity of data processing.

[0018] Optionally, the top-level module of the sorting network is configured to: concatenate multiple short input sequences and input them in parallel into the full-function bitone sorting network or its sub-network according to the total number of groups control field.

[0019] The top-level module of the sorting network concatenates multiple short input sequences and processes them in parallel based on the total number of groups control field. This makes fuller use of the hardware computing resources of the full-featured bitone sorting network or its sub-networks, improving hardware utilization and overall throughput when processing multiple small-scale tasks.

[0020] A second aspect of this application provides a Top-K acceleration method based on RISC-V vector extension, applied to the Top-K acceleration device based on RISC-V vector extension described in the first aspect, the method comprising: Software algorithm steps: Generate configuration information in the configuration information lookup table; wherein, the configuration information is gated configuration information of multiple simplified bitone sorting network structures; Hardware execution steps: Receive the VTOPK instruction and decode the VTOPK instruction to parse out the operation parameters; Based on the parsed operation parameters, load the corresponding gate control configuration information from the configuration information lookup table; Based on the loaded gating configuration information, the comparison and exchange units in the full-function bitone sorting network are dynamically activated using gating clock technology. Read the input vector data from the vector register file; The input vector data is subjected to descrambling and sequence processing; The processed input vector data is fed into the activated sorting network for Top-K selection. The result of the Top-K selection operation is written back to the vector register file.

[0021] The configuration information is generated and retrieved from the lookup table through software algorithm steps. In the hardware execution steps, gated clock technology is used to dynamically activate the comparison and exchange units required in the full-function bitone sorting network. Combined with the descrambling and sequence processing of the input vector data, the Top-K selection operation is realized. This reduces the dynamic power consumption of the sorting operation while ensuring functional correctness and improves the adaptability to different input scales and configurations.

[0022] Optionally, the steps for generating configuration information in the configuration information lookup table include: Using a full-featured bitone sorting network as the initial template, we iteratively attempt to remove each comparison and swap unit; After each removal attempt, the pruned network is quickly validated using random sequence test vectors; After completing the fast verification of all units using random sequences, a complete verification of the simplified network is performed based on the zero-one principle. Both the complete verification and the fast verification of random sequences adopt parallel computing, distributing the test vector set to multiple processes for simultaneous verification.

[0023] The steps for generating configuration information in the configuration information lookup table are achieved through iterative pruning comparison and swapping units, followed by rapid verification using random sequence test vectors and parallel complete verification based on the zero-one principle. This reduces the overall verification time of the network generation process and lowers the logical resource overhead of the final hardware sorting network while ensuring the correctness of the generated simplified network functions.

[0024] Optionally, the steps of descrambling and sequence processing the input vector data include: Based on the parsed total number segments, identify multiple short input sequences; Concatenate the multiple short input sequences; The concatenated data stream is input into the top-level module of the sorting network and processed in parallel.

[0025] The steps of de-scrambling and sequence processing of the input vector data involve identifying and concatenating multiple short input sequences, and then inputting the concatenated data stream into the sorting network for parallel computation. This allows for more efficient use of the hardware resources of the top-level module of the sorting network, thereby improving the processing throughput when performing Top-K selection on batches of short sequences.

[0026] As can be seen from the above technical solutions, this application provides a Top-K acceleration device and method based on RISC-V vector extension. The method generates configuration information in a configuration information lookup table through a software algorithm. In the hardware process, a VTOPK instruction is received and decoded to parse the operation parameters. Based on the parsed operation parameters, the corresponding gating configuration information is loaded from the configuration information lookup table. Based on the loaded gating configuration information, the comparison and exchange units in the full-function bitone sorting network are dynamically activated using gating clock technology. Input vector data is read from the vector register file. The input vector data undergoes descrambling and sequence processing. The processed input vector data is input to the activated sorting network for Top-K selection. The result of the Top-K selection operation is written back to the vector register file, thus solving the problem that the Top-K selection algorithm cannot simultaneously achieve performance, flexibility, and versatility. Attached Figure Description

[0027] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating the Top-K acceleration method based on RISC-V vector extension provided in this application embodiment; Figure 2 A flowchart illustrating the overall implementation of the design scheme for the Top-K acceleration device based on RISC-V vector extension provided in this application embodiment; Figure 3 A schematic diagram illustrating the parallelization verification principle of a Top-K acceleration device based on RISC-V vector extension, provided for embodiments of this application. Figure 4 A schematic diagram of an 8-2 simplified bitone sorting network for a Top-K acceleration device based on RISC-V vector extension provided in this application embodiment; Figure 5 A schematic diagram of a custom VTOPK instruction for a Top-K acceleration device based on RISC-V vector extension provided in this application embodiment; Figure 6 A schematic diagram of multiple parallel inputs of a Top-K acceleration device based on RISC-V vector extension provided in an embodiment of this application; Figure 7 This is a schematic diagram of the overall hardware architecture of the Top-K acceleration device based on RISC-V vector extension provided in the embodiments of this application. Detailed Implementation

[0029] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application.

[0030] In models such as Vision Transformer (ViT), the MoE structure expands the model capacity without increasing inference computation by replacing the multilayer perceptron (MLP) layers with layers composed of multiple expert subnetworks. During inference, it dynamically selects only the most relevant expert subnetworks for each input. The core of this implementation is to use the Top-K selection algorithm to select the K highest-scoring experts from N available experts for subsequent computation.

[0031] RISC-V is an emerging instruction set architecture (ISA). As a representative instruction system of Reduced Instruction Set Computers (RISC), RISC-V has brought about changes and opportunities in many aspects. RISC-V's fixed instruction length and simple instruction set simplify the design of processor microarchitecture. Its open-source, modular, and scalable characteristics significantly lower the barrier to chip development, providing an ideal platform for embedded systems, personal mobile devices, and other fields.

[0032] To address the issue that the Top-K selection algorithm cannot simultaneously balance performance, flexibility, and generality, see [link to relevant documentation]. Figures 1-7 This application provides a Top-K acceleration device based on RISC-V Vector Extension (RVV) in some embodiments, including: The software algorithm part includes a reconfigurable simplified bitonic sorting network generator, and the configuration information in the reconfigurable simplified bitonic sorting network generator. The configuration information is the configuration information of multiple simplified bitonic sorting network structures.

[0033] See Figure 2The inherent trade-off between flexibility and efficiency in Top-K accelerators can be resolved by combining offline process optimization with hardware process optimization. In the offline process, a reconfigurable simplified bitone sorting network (R-RBS) generator performs automated pruning analysis on a full-featured bitone sorter (FBS) based on a preset (N, K) requirement set, generating the optimal hardware gating configuration and storing it in a configuration information lookup table. In the hardware process, a custom VTOPK instruction passes key parameters such as N, K values, and data types to the hardware, guiding the Top-K hardware functional units to dynamically load the pre-stored configuration. The gating mechanism shields redundant logic, simplifying the full-featured sorting network into an energy-optimized simplified version for computation. This accelerator is ultimately integrated into a RISC-V vector processor and, with complete software toolchain support (including RVV operator development and C language application compilation), achieves an end-to-end solution from hardware acceleration to software deployment, effectively resolving the inherent trade-off between flexibility and efficiency in Top-K accelerators.

[0034] To address the high power consumption of FBS on edge processors, this invention proposes a reconfigurable simplified bitone sorting network generator. Since Top-K selection does not require a complete sort of the sequence, a large number of Compare-and-Swap (CAS) units in the FBS are redundant. By removing these redundant units, the power consumption of the sorting network can be significantly reduced while maintaining correct functionality. Specifically, using a standard FBS as the initial template, each CAS unit node is pruned and removed one by one. After each unit is removed, the algorithm verifies that the network output still satisfies the Top-K constraints, thus ensuring correct Top-K selection. If the verification passes, the CAS unit is permanently removed and the network structure is updated. This iterative process continues until there are no more redundant CAS units to be removed from the network. The verification process uses the zero-one principle, that is, each successful verification inputs 2 to the network to be verified. N By using different 0-1 sequences to cover all possible input sequence features, the network verified by the zero-one principle satisfies the necessary and sufficient condition for functional correctness. When the network size is large, this invention proposes random sequence verification and parallelized verification based on the zero-one principle to accelerate the verification speed. See also Figure 4 This is a schematic diagram of an 8-2 simplified bitonic sorting network obtained from a reconfigurable simplified bitonic sorting network generator, where the dashed lines represent the redundant CAS units that have been removed. Through simplification, the original 24 CAS units are reduced to only 14.

[0035] The hardware design includes a custom instruction processing unit integrated into the RISC-V vector processor, which includes an instruction decoder configured to decode VTOPK instructions. The instruction code of a VTOPK instruction includes an operation parameter control field, which includes a sign control field, a bit width control field, a data type control field, a total length control field, and a total number of groups control field.

[0036] The Top-K hardware functional unit is connected to a custom instruction processing unit and a vector register file. The Top-K hardware functional unit includes an instruction queue, a data processing pipeline, a top-level module of the sorting network, and a result queue. The top-level module of the sorting network includes a full-featured bitonal sorting network and a configuration information lookup table. The full-featured bitonal sorting network includes multiple comparison and exchange units, and the top-level module of the sorting network includes a gated clock circuit. The gated clock circuit controls the clock signals of the comparison and exchange units according to the gated configuration information loaded from the configuration information lookup table.

[0037] See Figure 7 , Figure 7 (a) shows the architecture diagram after integrating the Top-K hardware functional units into the vector processor. Figure 7 Figure (b) shows a schematic diagram of the Top-K hardware functional unit. In the vector processor, the instruction dispatcher receives and decodes vector instructions from the scalar processor, the instruction sequencer is used to eliminate data hazards and issue instructions in an ordered manner, and the channel contains various vector processing units and a vector register file. The sliding unit and the masking unit are used to process vector arrangement instructions and vector masking instructions, respectively. The vector loading and storage unit is responsible for data exchange between the internal vector register file and external memory. As an off-channel functional unit, the Top-K hardware functional unit receives VTOPK instructions from the instruction sequencer through a unidirectional control interface and exchanges data with the vector register file within the channel through a bidirectional data interface.

[0038] It should be noted that the implementation of this invention is not limited to a specific hardware platform or sorting network structure. For other hardware architectures that support SIMD or vectorized processing, Top-K acceleration can be achieved by combining the sorting network proposed in this invention with dedicated hardware units. Furthermore, the sorting network structure can also employ other sorting network structures such as odd-even sorting networks, hybrid sorting networks, or pairwise comparison sorting networks. In addition, by adjusting the size of the sorting network or juxtaposing multiple networks, different application scenarios can be adapted. For sequences exceeding the hardware processing capacity, segmented processing can be used. These alternative solutions can all achieve the objectives of this invention.

[0039] In some embodiments, the reconfigurable simplified bitonic sorting network generator includes a pruning module and a verification module; the pruning module is configured to iteratively attempt to remove each comparison and swap unit using a full-featured bitonic sorting network as an initial template.

[0040] It should be understood that the iterative process of the pruning module is not blind, but follows clear logic and rules. After using a fully functional bitonic sorting network as the initial template, it attempts to remove each comparison and swap (CAS) unit in a predetermined order. This order can be based on the physical arrangement of the network or the order generated by a specific algorithm, ensuring that each unit has a chance to be evaluated for removal.

[0041] The verification module is configured to: after each removal attempt, use the test vector set to verify the functional correctness of the pruned network, and permanently remove or retain the comparison and swap units based on the verification results; the verification module adopts a combination of full verification based on the zero-one principle, fast verification based on random sequences, and parallel verification.

[0042] It should be understood that the verification module initiates verification immediately after each attempt by the pruning module to remove a Compare-and-Swap (CAS) unit. This zero-one principle-based verification performs comprehensive and meticulous testing on the pruned network, ensuring it correctly executes Top-K operations and outputs the expected results under all possible input conditions. While this verification method is time-consuming, it provides the most reliable verification results.

[0043] Specifically, a small amount (much less than 2) is used when verifying each CAS unit. N A random sequence is used instead of a complete sequential 0-1 sequence to quickly filter out CAS units that fail verification and retain them. If all random sequences pass verification, the CAS unit is directly removed, and the verification of the next CAS unit continues. The more random sequences there are, the lower the probability of direct removal errors, but the verification speed will decrease. After all CAS units have been verified by random sequences, a final complete zero-one principle verification is performed to ensure network correctness. This method of quickly filtering redundant CAS units using random sequence input significantly reduces the overall verification workload, thereby accelerating the verification speed.

[0044] In some embodiments, the verification module is configured to divide the test vector set into multiple subsets and use multiple computing cores to assign each subset to an independent verification process for parallel verification.

[0045] It should be understood that the design of parallelized verification aims to fully utilize the computing power of modern multi-core processors by decomposing verification tasks that originally needed to be executed sequentially into multiple subtasks that can be processed in parallel, thereby significantly improving the overall efficiency of the verification process. Specifically, the verification process first decomposes the test vector set into several non-overlapping subsets according to a preset partitioning strategy, such as equal partitioning or partitioning according to specific rules. Then, each subset is assigned to an independent verification process, which can run simultaneously on different computing cores, achieving true parallel verification. This process not only speeds up verification but also reduces the risk of overall verification interruption due to the failure of a single process through parallel processing, enhancing the robustness of the verification.

[0046] Specifically, see Figure 3 By using random test vectors and 2 N The test vectors are divided into 0-1 segments, and verification is performed using multi-process parallelism on the CPU. Each process verifies only a portion of the complete test vector set. If any process fails, all verification processes are immediately terminated, and the network structure is restored. Verification is only successful when all verification processes pass. The number of verification processes depends on the number of CPU cores used; therefore, using a server platform can maximize the parallelization speed improvement.

[0047] In some embodiments, the VTOPK instruction is encoded in the OPIVX format of the RISC-V vector instruction set, and the operation parameter control field is encoded in the scalar register operand.

[0048] It should be understood that the VTOPK instruction uses the OPIVX format encoding of the RISC-V vector instruction set. This format is a typical operand type combination in the RISC-V vector instruction set, where OP represents the operation type, I represents an immediate operation (which can be understood as a specific instruction operation mode in this context), V represents a vector register, and X represents a scalar register. Through this format, the VTOPK instruction can efficiently handle operations between vectors and scalars. Furthermore, encoding the operation parameter control fields in the scalar register operands means that these control parameters (such as the selection of the K value, the setting of the operation mode, etc.) can be flexibly configured through the scalar register, thereby enhancing the instruction's versatility and configurability.

[0049] See Figure 5The VTOPK instruction is based on the OPIVX format encoding of the standard RVV instruction set. When the instruction is executed, the configuration information in the scalar register rs1 is decoded, and scalar operands are read from the scalar processor for fine-tuning the input vector information in the vector register vs2. The configuration information contains multiple control fields: the sign control field indicates whether the input sequence is signed or unsigned; the bit width field indicates the element bit width of the input sequence, allowing dynamic switching between multiple data precisions such as 4, 8, 16, and 32 bits; and the data type field indicates whether the input sequence is integer or floating-point. Based on these fields, appropriate comparison logic is selected at the hardware level to implement multi-type data processing under a unified instruction format. To maximize hardware throughput, support for multi-sequence parallel processing is added to the instruction. Multiple short sequences are concatenated and processed in parallel using the total length and total number of data segments, fully utilizing the modularity of the bitonic sorting network, which significantly improves hardware utilization when processing small batches of tasks. See the diagram for a schematic of multiple sets of parallel inputs. Figure 6 .

[0050] In some embodiments, the data processing pipeline includes a descrambling unit for restoring the input sequence from the vector processor data arrangement format to its natural order so that the correct sequence index can be generated subsequently.

[0051] It should be understood that the descrambling unit plays a crucial role in the data processing pipeline. Since data is stored in the vector registers of the vector processor according to certain rules, the sequence needs to be restored to its natural order in order to correctly generate the sequence index during sorting. The descrambling unit is designed to achieve this function; it connects the vector register file and the top-level module of the sorting network, ensuring that the data can be accurately restored to its original natural order after a series of processing steps, thus guaranteeing the accuracy and effectiveness of data processing. When the data is stored back in the vector registers, it needs to be scrambled again in the original manner.

[0052] In some embodiments, the top-level module of the sorting network further includes an index generation unit and a sequence complementation unit; the index generation unit is connected to the data processing pipeline and is used to generate an original position index for each element in the input sequence; the sequence complementation unit is connected to the index generation unit and the full-featured bitone sorting network and is used to perform complementation processing on input sequences whose length is less than the maximum processing length of the full-featured bitone sorting network.

[0053] It should be understood that the index generation unit plays a crucial role in the top-level module of the sorting network. During data processing, the input sequence may change its original positional order due to various operations, which can cause problems for subsequent data processing and analysis. The index generation unit, by connecting to the data processing pipeline, can accurately generate original position indices for each element in the input sequence. Thus, regardless of how the data has changed during processing, its original position can be quickly and accurately found using these indices, providing strong support for data reconstruction and further analysis.

[0054] The sequence complement unit is equally indispensable. In practical applications, the lengths of input sequences may vary. When encountering an input sequence shorter than the maximum processing length of a full-featured bitonic sorting network, directly inputting it into the sorting network without any processing may lead to inaccurate or erroneous sorting results. The sequence complement unit connects the index generation unit and the full-featured bitonic sorting network. It performs complementation on these shorter input sequences, bringing their length to the maximum processing length of the full-featured bitonic sorting network. In this way, it ensures that all input sequences can be processed correctly and effectively in the sorting network, improving the stability and reliability of the entire data processing system.

[0055] In some embodiments, the top-level module of the sorting network is configured to concatenate multiple short input sequences and input them in parallel into a full-featured bitone sorting network or its sub-networks, based on the total number of groups control field.

[0056] It should be understood that after receiving the total number of groups control field, the top-level module of the sorting network performs a concatenation operation on multiple short input sequences according to the group number information indicated by the field. The concatenation process is not arbitrary, but rather follows specific rules and order, combining short input sequences that meet the conditions to form a longer sequence that satisfies the input requirements of the full-featured bitonic sorting network or its subnetworks. Subsequently, these concatenated sequences are input in parallel into the full-featured bitonic sorting network or its subnetworks to fully utilize the network's parallel processing capabilities and improve sorting efficiency. This approach effectively handles input sequences of varying lengths, ensuring the accuracy and efficiency of the entire sorting process.

[0057] Specifically, see Figure 7 The Top-K hardware functional unit adopts a pipelined design and includes four main parts: 1. Instruction Queue: Essentially a first-in-first-out (FIFO) memory with a depth of 4, it is an instruction buffer area where VTOPK instructions dispatched to functional units are cached in the instruction queue, with pointers indicating the instructions being issued and submitted.

[0058] 2. Data Processing Pipeline: The data processing pipeline reads the data sequence from the vector register according to the instruction configuration information, restores the sequence from the specific data arrangement of the vector processor to the natural order through the descrambling unit, which facilitates index generation, and automatically processes multi-cycle data transmission according to the interface bit width. After the data processing is completed, it is sent to the top-level module of the sorting network.

[0059] 3. Top Layer of the Sorting Network: This module is responsible for processing the complement and index generation of the input data to support inputs of arbitrary length while preserving the original positions of elements. This module employs a physically complete 32-input full-featured bitone sorter. Its modular hierarchical structure allows for parallel reuse of internal sub-networks when processing small batches, thereby maximizing hardware utilization. Furthermore, this module includes a controller containing a lookup table (LUT) pre-generated by a simplified bitone sorting network generator. At runtime, the controller selects the corresponding network structure through instruction configuration parameters and activates only the necessary CAS units using gated clock technology, thus logically and dynamically reconstructing the FBS into a lower-power RBS.

[0060] 4. Result queue: The result queue has a depth of 2 and serves as a buffer area for the results, responsible for writing these data back to the vector register.

[0061] The Top-K unit, as a high-efficiency and independent execution unit, provides complete hardware acceleration support for the VTOPK instruction and is seamlessly integrated into the processor's execution architecture.

[0062] See Figure 1 This application also provides a Top-K acceleration method based on RISC-V vector extension in some embodiments, applied to the Top-K acceleration device based on RISC-V vector extension in the above embodiments, the method including: S100: Software algorithm steps: Generate configuration information by looking up the configuration information in the table.

[0063] The configuration information consists of gating configuration information for multiple simplified bitone sorting network structures.

[0064] S200: Hardware execution steps: Receive VTOPK instructions and decode them to parse out the operation parameters.

[0065] It should be understood that the VTOPK instruction is a specific instruction used to accelerate Top-K calculations. Upon receiving the instruction, the system will decode it to accurately parse out the various operational parameters related to Top-K calculations, such as the value of K, the range of input data, and sorting requirements. These operational parameters will provide necessary guidance and basis for subsequent Top-K acceleration calculations.

[0066] S300: Based on the parsed operation parameters, load the corresponding gate control configuration information from the configuration information lookup table.

[0067] S400: Based on the loaded gating configuration information, the comparison and switching units in the full-function bimodal sorting network are dynamically activated using gating clock technology.

[0068] S500: Reads input vector data from the vector register file.

[0069] S600: Performs descrambling and sequence processing on the input vector data.

[0070] S700: Input the processed input vector data into the activated sorting network to perform Top-K selection.

[0071] It should be understood that Top-K refers to the operation of selecting the K largest or K smallest elements from a dataset. It plays a crucial role in many applications, such as identifying the K most popular products in data mining and selecting the K features that have the greatest impact on the model in machine learning. In the RISC-V vector extension-based scenario described in this implementation, the Top-K operation can be achieved more efficiently and accurately through the application of relevant instructions and techniques.

[0072] S800: Writes the result of the Top-K selection operation back to the vector register file.

[0073] It should be understood that the step of generating configuration information in the configuration information lookup table is to pre-store gating configuration information for various possible simplified bitone sorting network structures. This allows for rapid retrieval of the required configuration in subsequent operations, improving processing efficiency. Receiving and decoding the VTOPK instruction is to accurately parse out key operational parameters from the instruction, which are crucial for subsequent sorting operations. Loading the corresponding gating configuration information from the configuration information lookup table based on the parsed operational parameters ensures that the activated sorting network structure meets the requirements of the current operation. Dynamically activating the comparison and swap units in the full-function bitone sorting network using gating clock technology avoids unnecessary energy consumption while ensuring the accuracy of the sorting operation. Reading the input vector data from the vector register file is the initial step of the sorting operation, providing the data foundation for subsequent processing. De-scrambling and sequence processing of the input vector data ensures that the data meets the processing requirements of the sorting network, guaranteeing the correctness of the sorting result. Inputting the processed input vector data into the activated sorting network for Top-k selection allows for rapid filtering of key data from a large amount of data. Finally, the result of the Top-k selection operation is written back to the vector register file for further processing or use.

[0074] In some embodiments, the step of generating configuration information in the configuration information lookup table includes: Using a full-featured bitone sorting network as the initial template, we iteratively attempt to remove each comparison and swap unit.

[0075] After each removal attempt, the pruned network is quickly validated using random sequence test vectors.

[0076] After completing the fast verification of all units using random sequences, a complete verification of the simplified network is performed based on the zero-one principle.

[0077] Among them, the full verification and the fast verification of random sequences adopt parallel computing, which distributes the test vector set to multiple processes for simultaneous verification.

[0078] It should be understood that when iteratively trying with a full-featured bitone sorting network as the initial template, only one comparison and swap unit is removed each time. This allows for precise evaluation of the impact of each unit on the overall network functionality. Using random sequence test vectors to quickly filter the pruned network is to determine in a short time whether the network can still maintain its basic sorting function after removing a unit; if not, the unit cannot be removed. After completing the random sequence quick filtering of all units, a complete verification based on the zero-one principle is performed on the simplified network. The zero-one principle states that if a sorting network can correctly sort any sequence of 0s and 1s, then it can correctly sort any sequence of numerical values. During verification, all possible 2s need to be input into the network. N The algorithm iterates through several distinct 0-1 sequences (from all 0s to all 1s) and determines whether the output matches expectations. It should be noted that N is the length of the sorting network input. By employing parallel computing to distribute the test vector set across multiple processes for simultaneous verification, verification efficiency is significantly improved, and the time required to generate the configuration information lookup table is reduced. It should be understood that this method of generating the configuration information lookup table ensures that the final lookup table is accurate and efficient, providing a reliable basis for subsequently loading gating configuration information based on this lookup table.

[0079] In some embodiments, the steps of descrambling and sequence processing the input vector data include: Based on the parsed total number segments, identify multiple short input sequences.

[0080] Concatenate multiple short input sequences.

[0081] The concatenated data stream is input into the top-level module of the sorting network and processed in parallel.

[0082] It should be understood that in the above steps, identifying multiple short input sequences based on the parsed total number segments is a process based on prior understanding of the input vector data structure and the established parsing rules. The total number segments act as an identifier, guiding us to accurately segment each short input sequence from complex input vector data. Concatenating multiple short input sequences integrates the scattered data into a coherent data stream, facilitating unified processing in the top-level module of the sorting network. Inputting the concatenated data stream into the top-level module of the sorting network for parallel computation leverages the efficiency of parallel computing, simultaneously performing sorting-related operations on multiple elements in the data stream, thereby improving overall processing speed and efficiency.

[0083] As can be seen from the above technical solutions, the embodiments of this application provide a Top-K acceleration device and method based on RISC-V vector extension. Configuration information in a configuration information lookup table is generated through a software algorithm. In the hardware process, a VTOPK instruction is received and decoded to parse the operation parameters. Based on the parsed operation parameters, the corresponding gating configuration information is loaded from the configuration information lookup table. Based on the loaded gating configuration information, the comparison and exchange units in the full-function bitone sorting network are dynamically activated using gating clock technology. Input vector data is read from the vector register file. The input vector data undergoes descrambling and sequence processing. The processed input vector data is input to the activated sorting network for Top-K selection. The result of the Top-K selection operation is written back to the vector register file, thus solving the problem that the Top-K selection algorithm cannot simultaneously achieve performance, flexibility, and versatility.

[0084] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A Top-K acceleration device based on RISC-V vector extension, characterized in that, include: The software algorithm section includes a reconfigurable simplified bitonic sorting network generator, which is used to generate configuration information in a configuration information lookup table. The configuration information is configuration information for multiple simplified bitonic sorting network structures. The hardware design includes: A custom instruction processing unit, integrated into the RISC-V vector processor, includes an instruction decoder configured to decode VTOPK instructions. The instruction code of the VTOPK instruction includes an operation parameter control field, which includes a sign control field, a bit width control field, a data type control field, a total length control field, and a total number of groups control field. The Top-K hardware functional unit is connected to the custom instruction processing unit and the vector register file. The Top-K hardware functional unit includes an instruction queue, a data processing pipeline, a top-level module of the sorting network, and a result queue. The top-level module of the sorting network includes a full-function bitone sorting network and the configuration information lookup table. The full-function bitone sorting network includes multiple comparison and exchange units, and the top-level module of the sorting network includes a gated clock circuit. The gated clock circuit controls the clock signal of the comparison and exchange units according to the gated configuration information loaded from the configuration information lookup table.

2. The Top-K acceleration device based on RISC-V vector extension according to claim 1, characterized in that, The reconfigurable simplified bitone sorting network generator includes a pruning module and a verification module; The pruning module is configured to iteratively attempt to remove each comparison and swap unit using the full-featured bitone sorting network as an initial template. The verification module is configured to: after each removal attempt, use a test vector set to verify the functional correctness of the pruned network, and permanently remove or retain the comparison and exchange unit based on the verification result; The verification module employs a combination of full verification based on the zero-one principle, fast verification based on random sequences, and parallel verification.

3. The Top-K acceleration device based on RISC-V vector extension according to claim 2, characterized in that, The verification module is configured to divide the test vector set into multiple subsets and use multiple computing cores to assign each subset to an independent verification process for parallel verification.

4. The Top-K acceleration device based on RISC-V vector extension according to claim 1, characterized in that, The VTOPK instruction is encoded in the OPIVX format of the RISC-V vector instruction set, and the operation parameter control field is encoded in the scalar register operand.

5. The Top-K acceleration device based on RISC-V vector extension according to claim 1, characterized in that, The data processing pipeline includes a descrambling unit for restoring the input sequence from the vector processor data arrangement format to its natural order so that the correct sequence index can be generated subsequently.

6. The Top-K acceleration device based on RISC-V vector extension according to claim 1, characterized in that, The top-level module of the sorting network also includes an index generation unit and a sequence complementation unit; The index generation unit is connected to the data processing pipeline and is used to generate an original position index for each element in the input sequence. The sequence complementation unit is connected to the index generation unit and the full-function bitonic sorting network, and is used to perform complementation processing on input sequences whose length is less than the maximum processing length of the full-function bitonic sorting network.

7. The Top-K acceleration device based on RISC-V vector extension according to claim 1, characterized in that, The top-level module of the sorting network is configured as follows: Based on the total number of groups control field, multiple short input sequences are concatenated and input in parallel into the full-function bitone sorting network or its sub-network.

8. A Top-K acceleration method based on RISC-V vector extension, characterized in that, The method, applied to any one of claims 1-7, for a Top-K acceleration device based on RISC-V vector extension, comprises: Software algorithm steps: Generate configuration information in the configuration information lookup table; wherein, the configuration information is gated configuration information of multiple simplified bitone sorting network structures; Hardware execution steps: Receive the VTOPK instruction and decode the VTOPK instruction to parse out the operation parameters; Based on the parsed operation parameters, load the corresponding gate control configuration information from the configuration information lookup table; Based on the loaded gating configuration information, the comparison and exchange units in the full-function bitone sorting network are dynamically activated using gating clock technology. Read the input vector data from the vector register file; The input vector data is subjected to descrambling and sequence processing; The processed input vector data is fed into the activated sorting network for Top-K selection. The result of the Top-K selection operation is written back to the vector register file.

9. The Top-K acceleration method based on RISC-V vector extension according to claim 8, characterized in that, The steps for generating configuration information in the configuration information lookup table include: Using a full-featured bitone sorting network as the initial template, we iteratively attempt to remove each comparison and swap unit; After each removal attempt, the pruned network is quickly validated using random sequence test vectors; After completing the fast verification of all units using random sequences, a complete verification of the simplified network is performed based on the zero-one principle. Both the complete verification and the fast verification of random sequences adopt parallel computing, distributing the test vector set to multiple processes for simultaneous verification.

10. The Top-K acceleration method based on RISC-V vector extension according to claim 8, characterized in that, The steps for descrambling and sequence processing the input vector data include: Based on the parsed total number segments, identify multiple short input sequences; Concatenate the multiple short input sequences; The concatenated data stream is input into the top-level module of the sorting network and processed in parallel.

Images