Computing unit, hardware accelerator and electronic chip with scalable computing power

Abstract

This invention discloses a scalable computing unit, hardware accelerator, and electronic chip. The scalable computing unit includes: local memory, a core controller, and a configurable PE-LUT array for computation and storage. The PE-LUT array includes multiple lookup table columns and a processing unit column for performing computations. Under the control of the core controller, the PE-LUT array can be configured into a computation-intensive mode or a storage-intensive mode. In computation-intensive mode, each lookup table column is used to pre-store the product results of low-precision multiplication operations and accumulation operations. In storage-intensive mode, each lookup table column is used to cache data participating in the computation of the processing unit column. This invention can be used in visual autoregressive models to flexibly expand computing power according to task size and computational load.

Description

Technical Field

[0001] This invention relates to the field of integrated circuit technology, and in particular to a computing unit with scalable computing power, a hardware accelerator, and an electronic chip. Background Technology

[0002] With the development of artificial intelligence technology, computers are now able to automatically generate visual content such as images and videos based on user input. This type of technology is commonly referred to as visual generation technology. The core of visual generation technology is the visual generation model, which learns from a large amount of visual data to establish a mapping relationship between input information and visual content, thereby achieving the goal of automatically generating images or videos.

[0003] As one of the mainstream visual generation techniques, the visual autoregressive model employs a hierarchical design for the image generation process. Specifically, in the generation process, the visual autoregressive model first starts with an extremely low-resolution image representation, such as containing only one or a few image patches; then, in each generation stage, it progressively interpolates and expands the image representation, gradually increasing its resolution until a complete high-resolution image is generated.

[0004] The aforementioned coarse-to-fine staged generation method results in distinct stage-specific characteristics in the visual autoregressive model's inference process: the computational scale is relatively small in the low-resolution stage, while it rapidly increases in the high-resolution stage. During the generation process, the number of image patches, computational scale, and storage access patterns involved in different generation stages of the visual autoregressive model vary significantly. However, existing hardware acceleration solutions for this model are mostly designed with fixed computing power, making it difficult for accelerators to dynamically adjust computing resources according to different generation stages in the application scenario. This limits computing power utilization and energy efficiency, thus hindering the full realization of the performance advantages of the visual autoregressive model. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a scalable computing unit, hardware accelerator, and electronic chip, which enables visual autoregressive models to flexibly expand their computing power according to task scale and computational load.

[0006] The present invention provides a computing unit with scalable computing power, comprising: local memory, core controller and computing and storage configurable PE-LUT array; The PE-LUT array includes multiple lookup table columns and a processing unit column for performing calculations. Under the control of the core controller, the PE-LUT array can be configured into a computation-intensive mode or a storage-intensive mode. In computation-intensive mode, each lookup table column is used to pre-store the product results of low-precision multiplication operations and accumulation operations. In storage-intensive mode, each lookup table column is used to cache the data participating in the calculation of the processing unit column. The local memory is used to store data and intermediate results accessed frequently by the PE-LUT array; The core controller is used to control the local memory and PE-LUT array according to the computing load.

[0007] This invention also provides a hardware accelerator, comprising: a precision-aware vector module, a configurable accumulator, and multiple PE-LUT clusters capable of simultaneously executing computational tasks of different precisions, each PE-LUT cluster containing two of the aforementioned computationally scalable computing units. The configurable accumulator is used to dynamically select the accumulation path for accumulation processing based on the accuracy of the calculation results output by multiple PE-LUT clusters; The precision-aware vector module is used to support nonlinear operations and quantization operations with different precision requirements.

[0008] This invention also provides an electronic chip, including the aforementioned hardware accelerator.

[0009] In this embodiment of the invention, the PE-LUT core with scalable computing power adopts a hybrid array structure design. By refactoring the functions of the lookup table columns, it becomes highly configurable, capable of being configured as either computing units or register resources. This allows for flexible adjustment of the computing and storage ratio at different computing stages to adapt to the computing structure requirements of different model layers. During task execution, by analyzing and statistically analyzing the distribution characteristics of the activated data, the main control processor can dynamically configure the computing precision for each PE-LUT core based on the obtained data distribution information. This enables different computing units to execute computing tasks with appropriate precision, thereby achieving multi-precision parallel processing. This ensures numerical accuracy on critical computing paths while allowing lower precision in non-critical computing to reduce power consumption and improve computing efficiency. In terms of nonlinear and quantization operations, the precision-aware vector unit integrates high-precision computing units and low-precision lookup table computing units. Through the collaborative work of these two types of computing units, the accelerator can improve hardware resource utilization and reduce overall energy consumption while ensuring computational accuracy. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a diagram illustrating the computational requirements of a visual generative model.

[0012] Figure 2 This is a schematic diagram illustrating the dynamic changes in the distribution characteristics of computational data in a visual generative model.

[0013] Figure 3 This is a schematic diagram of the structure of a visual autoregressive model accelerator chip provided in an embodiment of the present invention.

[0014] Figure 4 for Figure 3 A schematic diagram of the structure of the PE-LUT core in computationally intensive mode.

[0015] Figure 5 for Figure 3 A schematic diagram of the structure of the PE-LUT core in storage-intensive mode.

[0016] Figure 6 This is a schematic diagram of a mixed-precision parallel processing data stream in the preprocessing stage provided in an embodiment of the present invention.

[0017] Figure 7 This is a schematic diagram of a mixed-precision parallel processing data stream in the post-processing stage provided by an embodiment of the present invention.

[0018] Figure 8 A schematic diagram of the structure of the W4A12 multiplication extension provided in an embodiment of the present invention.

[0019] Figure 9 A schematic diagram of the W4A8 multiplication extension provided in an embodiment of the present invention.

[0020] Figure 10 This is a schematic diagram illustrating the energy consumption and latency benefits of the hybrid precision hardware accelerator provided in this embodiment of the invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0022] As mentioned earlier, with the emergence of visual generative models, especially visual autoregressive models employing a staged generation mechanism, the structural and functional limitations of existing solutions have gradually become apparent. Figure 1The diagram illustrates the computational requirements of a visual generative model. Existing hardware accelerators typically have their computational power fixed during the design phase, making it difficult to adjust the number of processing units, parallelism, and data path configuration during operation. When the computational load during model inference changes significantly with the generation phase, a fixed-power architecture struggles to match the computational demands of different stages. This can lead to idle computing power during low-load phases and insufficient computing power during high-load phases, resulting in decreased overall resource utilization and limited inference efficiency.

[0023] In terms of numerical representation and computational accuracy, existing solutions mostly rely on fixed-precision quantization or statically configured hierarchical mixed-precision quantization techniques to reduce computational complexity and power consumption. Fixed-precision quantization typically employs a uniform numerical precision across the entire model, lacking adaptability to different computational stages and data distribution characteristics. While hierarchical mixed-precision quantization introduces precision differences at the model or module level, its precision configuration is usually determined during the compilation or deployment phase, making dynamic adjustment based on runtime data changes difficult. Figure 2 The diagram illustrates the dynamic changes in the computational data distribution characteristics in a visual generative model. In the visual autoregressive generative model, as the generation process progresses, the number of image patches involved in the computation continuously increases, the matrix size rapidly expands, and the range and complexity of the data numerical distribution significantly increase. Under these circumstances, there may be significant numerical differences between different levels and different matrix sub-regions. Low-precision representation cannot simultaneously take into account the data characteristics of each sub-region, and quantization errors are further amplified after multiple cumulative calculations, ultimately significantly impacting the overall accuracy and quality of the generated results.

[0024] To address this, the present invention proposes a scalable computing unit, a hardware accelerator, and an electronic chip. It provides a hybrid PE-LUT core, which combines a processing unit (PE) with a lookup table (LUT) and supports dynamic configuration of computing performance. This allows the accelerator to flexibly expand its computing power according to the task size and computing load, thereby effectively coping with the high computational pressure brought about by the large-scale tile generation in visual autoregressive models.

[0025] To make the technical solution of the present invention clearer, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0026] Figure 3 This is a schematic diagram of the structure of a visual autoregressive model accelerator chip provided in an embodiment of the present invention. Figure 4 for Figure 3 A schematic diagram of the structure of the PE-LUT core in computationally intensive mode. Figure 5 for Figure 3 A schematic diagram of the structure of the PE-LUT core in storage-intensive mode. (Example) Figures 3-5As shown, the accelerator chip mainly includes a precision-aware vector module, a configurable accumulator, and multiple PE-LUT clusters that can simultaneously execute computational tasks with different precisions. Each PE-LUT cluster contains two computationally scalable computing units (PE-LUT cores). The configurable accumulator is used to dynamically select the accumulation path for accumulation processing based on the precision of the computation results output by multiple PE-LUT clusters. The precision-aware vector module is used to support nonlinear operations and quantization operations with different precision requirements.

[0027] The PE-LUT core is the core computing unit in the visual autoregressive model accelerator architecture, employing a hybrid array structure design. Specifically, each PE-LUT core consists of one column of processing units (PE columns) and eight columns of lookup tables (LUT columns). The lookup table columns are highly configurable, serving as either computing units or register resources, allowing for flexible adjustments to the computation-to-storage ratio at different computational stages to meet the computational needs of different model layers. The PE-LUT array supports flexible data retrieval from local on-chip memory or global buffers. The local static memory has a capacity of 8KB and is used to store frequently accessed data and intermediate results, thereby reducing memory access latency and power consumption. By flexibly scheduling between local storage and global buffers, the accelerator can effectively control the efficiency of storage resource utilization while ensuring performance.

[0028] The aforementioned scalable computing unit includes: local memory, a core controller, and a configurable PE-LUT array for computation and storage. The PE-LUT array comprises multiple lookup table columns and a processing unit column for performing computations. Under the control of the core controller, the PE-LUT array can be configured into a compute-intensive mode or a storage-intensive mode. In compute-intensive mode, each lookup table column is used to pre-store the product results of low-precision multiplication operations and accumulation operations. In storage-intensive mode, each lookup table column is used to cache data participating in the computation of the processing unit column. The local memory stores frequently accessed data and intermediate results of the PE-LUT array. The core controller controls the local memory and PE-LUT array according to the computational load. In other words, the PE-LUT core can achieve efficient support for large-scale low-precision multiplication and addition operations by reconfiguring the functions of the lookup table columns. Each lookup table column is internally configured with multiple sets of dual-function lookup table units (DF-LUTs), which can be configured as computation units or storage units. The PE-LUT core corresponds to compute-intensive mode and storage-intensive mode.

[0029] In practical use, under computationally intensive mode, the processing unit column calculation generates a pre-stored product result. This product result represents all possible product values ​​of weights and low-precision multiplication operations. This product result is loaded into a lookup table column for pre-stored calculations. The weights of the multiplication operations serve as the addresses of the lookup table columns. The product result obtained by weight-based addressing is then fed into an addition tree for accumulation operations to complete vector dot products or matrix multiplication and addition calculations. For example... Figure 4 As shown, in computationally intensive mode, the dual-function lookup table unit pre-stores the product results for low-precision multiplication operations. For example, for multiplication of four-digit integers, the system loads all possible product results into the lookup table unit before execution. During actual computation, weight data no longer directly participates in the multiplication operation, but is instead input as an address to the lookup table unit for indexing and retrieving the corresponding product result. This approach avoids the numerous switching activities generated by traditional multipliers during computation, thus significantly reducing computational energy consumption. After the lookup operation is completed, the obtained product result is fed into the addition tree structure for accumulation to complete vector dot product or matrix multiplication and addition calculations. By transforming multiplication operations into lookup operations and combining them with accumulation using an addition tree, the PE-LUT core can significantly improve computational parallelism and energy efficiency while maintaining computational correctness.

[0030] In storage-intensive mode, lookup table columns are configured as register-type storage units that load data from the global cache, and data is read from the lookup table columns during processing unit column calculations. For example... Figure 5 As shown, in storage-intensive mode, the lookup table columns are no longer used to perform lookup-driven multiplication operations. Instead, they are reconfigured as register-type storage units to store activation data, intermediate calculation results, or weight data. In this mode, the lookup table units provide data access functionality similar to local memory, thereby expanding on-chip available storage capacity without introducing additional independent storage modules. In this mode, the lookup table columns first read cached data from the global buffer. Then, the PE-LUT array control logic uniformly manages the data read / write order and access timing to ensure that the PE columns read and write during computation. In this way, the PE-LUT core can flexibly cache some data in the lookup table columns during storage-intensive phases, reducing the number of accesses to local memory and the global buffer, thereby reducing memory access latency and improving overall execution efficiency.

[0031] In the aforementioned accelerator, to address the challenges of a wide distribution of activation data in visual autoregressive models and significant differences in numerical precision requirements across different spatial regions, the system employs a mixed-precision parallel processing data flow organization method. This allows computational tasks of varying precision to be executed simultaneously in the hardware, thereby improving overall computational efficiency and energy efficiency. Before the data enters the computation phase, the activation data is first divided into fixed-size segments.

[0032] To support multi-precision computing operations, the aforementioned accelerator introduces a main control RISC-V processor at the system control level, namely... Figure 3 The processor utilizes a RISC-V core. This core is responsible for the overall control and scheduling of the model inference process, and analyzes and statistically analyzes the distribution characteristics of the activated data during runtime. Based on the obtained data distribution information, the main control processor can dynamically configure the computational precision for each PE-LUT core, enabling different computational units to execute computational tasks with appropriate precision, thereby achieving multi-precision parallel processing. This approach ensures numerical precision on critical computational paths while allowing the use of lower precision in non-critical computations to reduce power consumption and improve computational efficiency.

[0033] In nonlinear and quantization operations, the precision-aware vector module integrates high-precision computation units and low-precision lookup table computation units. The high-precision computation units perform calculations requiring high numerical accuracy, while the low-precision lookup table computation units perform calculations with relatively lower precision requirements but higher frequency of operations. Through the collaborative work of these two types of computation units, the accelerator can improve hardware resource utilization and reduce overall energy consumption while ensuring computational accuracy.

[0034] The aforementioned accelerator architecture also includes a RISC-V processor, used to configure a corresponding computational precision level for each data block based on the activation values, forming a precision mapping table. Specifically, the RISC-V processor statistically analyzes the activation values ​​in each data block, extracts the maximum absolute value of the activation values ​​within that block, and compares this absolute value with preset first and second thresholds to label the data block with a computational precision level. These precision levels include low-precision, medium-precision, and high-precision computation, with the first threshold being less than the second threshold. In practical applications, such as... Figure 6The diagram illustrates the mixed-precision parallel processing data flow in the preprocessing stage. In this stage, the main RISC-V processor statistically analyzes the activation values ​​in each data block, extracts the maximum absolute value of the activation values ​​within that block, and compares it with two pre-defined thresholds. Based on the comparison results, each data block is assigned a corresponding computational precision level. When the maximum absolute value is below the first threshold, the data block is marked as low-precision computation; when it falls between the two thresholds, it is marked as medium-precision computation; and when it exceeds the second threshold, it is marked as high-precision computation. This method allows the system to utilize as much low-precision computation as possible while maintaining numerical accuracy, thus improving energy efficiency. After precision determination, the system generates a precision mapping table to guide subsequent data block scheduling and computation. This mapping table records the precision level corresponding to each data block and serves as the basis for allocating data blocks of different precisions to computing resources. To balance the differences in execution cycles between different precision computations, the system further concatenates two low-precision data blocks to form a larger data block, aligning low-precision computation with medium and high-precision computations in time as much as possible, thereby avoiding idle computing resources.

[0035] like Figure 7 The diagram illustrates the mixed-precision parallel processing data flow in the post-processing stage. In this stage, each data block is independently scheduled to a PE-LUT core for computation. Data blocks of different precisions can be simultaneously allocated to different PE-LUT cores, thus achieving multi-precision parallel computation at the hardware level. The PE-LUT core dynamically configures its internal lookup table and accumulation structure based on scheduling information to match the precision requirements of the corresponding data block. The system employs a reconfigurable accumulation structure to uniformly process the computation results from PE-LUT cores of different precisions. This accumulation structure consists of multiple sets of addition trees and addition units with different bit widths, which can dynamically select the appropriate accumulation path based on the precision of the input result, thereby avoiding unnecessary bit width expansion or precision loss. In this way, computation results of different precisions can be efficiently accumulated in the same hardware structure and output as intermediate results with higher bit widths. The accumulated output vector takes different processing paths depending on its bit width. For output vectors with a large bit width, the system sends them to the precision-aware vector unit, where the high-precision computing unit (floating-point processing unit) performs operations such as nonlinear activation functions and then converts them back to integer form. For output vectors with a small bit width, nonlinear processing and quantization operations are performed directly through a lookup table to reduce computational overhead.

[0036] like Figure 8The schematic diagram of the W4A12 multiplication extension shown indicates that the weight W is 4 bits and the activation value A is 12 bits. To support multi-precision calculation operations, the accelerator introduces a combinational computing mechanism based on a dual-function lookup table within the lookup table columns. Specifically, in each lookup table column, every two dual-function lookup table units are grouped and share the same address input, thus jointly participating in the generation of a multiplication result. By concatenating and combining the output results of the two lookup table units, this structure can support multiplication operations with four-bit weights and eight-bit or twelve-bit activation values. In the four-bit weight and eight-bit activation value calculation mode and the four-bit weight and twelve-bit activation value calculation mode, the system no longer relies on a single lookup table unit to complete the multiplication operation, but instead expands the bit width of the output result through the collaborative work of two lookup table units. In the specific implementation, the two lookup table units receive the same weight address input and output partial product results respectively. Then, the two output results are concatenated to form the complete product value required for the corresponding high-precision multiplication operation.

[0037] like Figure 9 The diagram illustrates the W4A8 multiplication extension, where the weight W is 4 bits and the activation value A is 8 bits. In the calculation mode with 4-bit weights and 8-bit activation values, only a 12-bit multiplication result needs to be generated. Therefore, the highest four bits of the output of the first lookup table in the grouped lookup table unit are masked. Simultaneously, the addition tree structure is re-divided into three parts: 4-bit, 4-bit, and 8-bit. By bypassing the accumulation path of the highest four bits, invalid calculations are avoided, thereby improving overall computational efficiency. For the calculation of 8-bit weights, the system completes it by combining two 4-bit partial products within two calculation cycles, thus supporting higher-precision weight calculations without increasing hardware complexity.

[0038] Figure 10 A schematic diagram illustrating the energy and latency gains of the hybrid precision hardware accelerator provided in this embodiment of the invention, as shown below. Figure 10 The benefit decomposition results shown demonstrate that all key technologies have made significant contributions to the improvement of system energy efficiency and performance. Among them, the scalable PE-LUT architecture reduces both computational energy consumption and on-chip memory access energy consumption, achieving an energy consumption reduction of approximately 1.4 times; the hybrid precision processing mechanism, by adaptively selecting computational precision based on data distribution, further brings an energy consumption reduction of approximately 2.5 times, and the accelerator achieves an overall system performance improvement of approximately 2.0 times.

[0039] This invention employs a hybrid array structure design for the scalable PE-LUT core, making it highly configurable through functional refactoring of the lookup table columns. These columns can be configured as either computational units or register resources, allowing for flexible adjustment of the computation-to-storage ratio at different computational stages to meet the computational needs of different model layers. During task execution, the main control processor analyzes and statistically processes the distribution characteristics of the activated data. Based on this data distribution information, the main control processor dynamically configures the computational precision for each PE-LUT core, enabling different computational units to execute computational tasks with appropriate precision. This achieves multi-precision parallel processing, ensuring numerical accuracy on critical computational paths while using lower precision in non-critical computations to reduce power consumption and improve computational efficiency. In nonlinear and quantization operations, the precision-aware vector unit integrates high-precision computational units and low-precision lookup table computational units. Through the collaborative work of these two types of computational units, the accelerator can improve hardware resource utilization and reduce overall energy consumption while ensuring computational accuracy.

[0040] The aforementioned hardware accelerator can be used in electronic chips, such as accelerator chips for visual autoregressive models, for data inference computation of deploying visual autoregressive models. Because this accelerator has scalable computing power and multi-precision parallel processing capabilities, it can meet the deployment needs of visual generative models with different levels and stages of computing power requirements and complex data numerical distributions.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A computing unit with scalable computing power, characterized in that, include: Local memory, core controller, and compute and storage configurable PE-LUT array; The PE-LUT array includes multiple lookup table columns and a processing unit column for performing computations. Under the control of the core controller, the PE-LUT array can be configured into a compute-intensive mode or a storage-intensive mode. In computationally intensive mode, each lookup table column is used to pre-store the product results of low-precision multiplication operations and accumulation operations; In storage-intensive mode, each lookup table column is used to cache data that participates in the calculation of the processing unit column; The local memory is used to store data and intermediate results accessed frequently by the PE-LUT array; The core controller is used to control the local memory and PE-LUT array according to the computing load.

2. The computing unit according to claim 1, characterized in that, In computationally intensive mode, the processing unit column calculation produces a pre-stored product result, which is all possible product values ​​of the weights and low-precision multiplication operations. This product result is loaded into the lookup table column in the form of a lookup table. The weights of the multiplication operations are used as the addresses of the lookup table columns. The product result obtained by weight addressing is sent to the addition tree for accumulation operation to complete the vector dot product or matrix multiplication and addition calculation.

3. The computing unit according to claim 1, characterized in that, In storage-intensive mode, lookup table columns are configured as register-type storage units that load data from the global cache, and data is read from lookup table columns when processing unit columns are computed.

4. A hardware accelerator, characterized in that, include: The precision-aware vector module, the configurable accumulator, and the multiple PE-LUT clusters capable of simultaneously performing computational tasks of different precisions, each PE-LUT cluster containing two computationally scalable computing units as described in any one of claims 1-3; The configurable accumulator is used to dynamically select the accumulation path for accumulation processing based on the accuracy of the calculation results output by multiple PE-LUT clusters; The precision-aware vector module is used to support nonlinear operations and quantization operations with different precision requirements.

5. The accelerator according to claim 4, characterized in that, It also includes a RISC-V processor, which is used to configure the corresponding computational precision level for each data block based on the activation value in each data block, forming a precision mapping table.

6. The accelerator according to claim 5, characterized in that, The RISC-V processor is specifically used to count the activation values ​​in each data block, extract the maximum absolute value of the activation values ​​in the data block, and compare the absolute value with a preset first threshold and a second threshold to mark the data block with a calculation precision level. The calculation precision level includes low precision calculation, medium precision calculation and high precision calculation, and the first threshold is less than the second threshold.

7. The accelerator according to claim 6, characterized in that, The RISC-V processor is also used to combine low-precision computed data blocks into larger data blocks.

8. The accelerator according to claim 4, characterized in that, The precision-aware vector module is specifically used to allocate high-precision data blocks to the floating-point processing unit for nonlinear and quantization operations according to different precision requirements, while allocating low-precision data blocks to the lookup table to complete nonlinear processing and quantization operations.

9. The accelerator according to any one of claims 4-8, characterized in that, Each lookup table column includes two DF-LUT units that share the same address input for the group. The outputs of the two DF-LUT units used by the group are concatenated and combined to support multiplication operations of four-bit weights and eight-bit or twelve-bit activation values.

10. An electronic chip, characterized in that, Includes the hardware accelerator as described in any one of claims 4-9.

Images