A quantization precision adaptive switching hardware acceleration architecture

Abstract

The application relates to the field of integrated circuit design and artificial intelligence hardware acceleration technology, and particularly discloses a kind of quantization precision adaptive switching hardware acceleration architecture.The application divides hardware into static area and dynamic area: the static area includes always running pre-processing, post-processing, control module and reconfigurable logic loading control block;The dynamic area carries the quantization and dequantization module composed of a plurality of reconfigurable logic units.The innovation lies in that the model precision instruction is parsed by the control module, the reconfigurable logic file of the corresponding precision is dynamically loaded from the external starting device to the dynamic area by the loading control block according to the internal address mapping table, so that seamless switching of different quantization precision is realized at the hardware level, and the whole chip does not need to be reprogrammed.The design effectively solves the limitations of fixed-precision hardware when dealing with diversified AI models through the modular and reconfigurable hardware structure, and significantly improves the deployment flexibility and energy efficiency.

Description

Technical Field

[0001] This invention relates to the fields of integrated circuit design and artificial intelligence hardware acceleration technology, specifically to a hardware acceleration architecture with adaptive switching of quantization accuracy. Background Technology

[0002] With the rapid development of artificial intelligence technology, large models are gradually changing our current production and lifestyles. Currently, the main factors limiting the development of large models are twofold: memory bandwidth bottlenecks and computing resource and power consumption bottlenecks. The massive weights and data of the models need to be frequently transferred between off-chip storage and on-chip computing units, and memory bandwidth severely limits the inference speed and throughput of the models. Furthermore, as the computational complexity of the models continues to increase, maintaining throughput while controlling power consumption is also a major challenge.

[0003] To address these challenges, quantization has become a core technology in the field of large-scale model hardware acceleration. Quantization reduces model weights and activation values ​​from the standard 32-bit floating-point format to a low-bit-width format, thereby achieving storage optimization, increased computational density, and improved energy efficiency.

[0004] However, most current hardware accelerator designs employ static structures to support quantization operations. Static solutions face the following key problems when handling mixed-precision models or switching between different precision models: 1. Resource waste and power consumption due to static hardware redundancy. To support multiple quantization precisions, existing accelerators typically employ two strategies: a general-purpose MAC or a fixed hybrid array. A general-purpose MAC designs the MAC (multiply-accumulate) unit to handle the highest precision and then uses a multiplexer to achieve compatibility with lower precisions; a fixed hybrid array statically allocates and constructs multiple MAC arrays of different precisions on the chip. Both solutions require significant resource redundancy, increasing static power consumption and wasting resources. 2. Insufficient runtime flexibility and inability to achieve optimal adaptation. The physical structure of existing solutions is fixed after hardware compilation. Although the operating mode of the MAC array can be switched via software registers, physical adjustments to the hardware logic are not possible. This results in the hardware being unable to perform physical-level customization optimization for the newly switched model when switching between different precision models, causing the model to always run in a suboptimal configuration. In cloud environments, it is difficult to dynamically release or reuse unnecessary hardware according to task requirements, leading to low accelerator card efficiency. Summary of the Invention

[0005] To overcome the problems of resource waste, high power consumption, complex layout and routing, difficulty in timing convergence, and inability to dynamically adapt to different quantization precision models caused by the static fixed structure of existing hardware acceleration architectures, this invention provides a hardware acceleration architecture with adaptive switching of quantization precision. By designing the core logic of quantization, inference, and dequantization as dynamically reconfigurable logic independent of the static area, and dynamically loading the corresponding precision logic file to the hardware based on the model precision requirements by the control module, this invention achieves the time-division multiplexing of hardware logic resources, adaptive and rapid switching of different quantization precision models, and operation under optimal timing.

[0006] The specific technical solution of this application is as follows:

[0007] A hardware-accelerated architecture with adaptive quantization precision switching includes:

[0008] The preprocessing module receives the raw input data, performs preprocessing operations, packages the processed data into a data packet of a uniform format, and outputs it through the data channel.

[0009] The quantization and dequantization module receives the data packet, performs unpacking, quantization, inference, and dequantization processing on the data according to the current precision configuration, and outputs the processing results as a data packet in a unified format.

[0010] The post-processing module receives the data packets output by the quantization and dequantization modules, performs post-processing operations, and outputs the inference results.

[0011] The control module, which communicates with the preprocessing module, the quantization and dequantization module, and the postprocessing module, is configured to: analyze the model accuracy requirements; dynamically configure the hardware logic of the quantization and dequantization modules according to the model accuracy requirements to adapt to the target quantization accuracy; and schedule the data flow through each module to avoid data blockage.

[0012] The preprocessing module, quantization and dequantization module, and postprocessing module transmit data packets of a unified format through a data channel, and exchange control information through a status and handshake signal channel to achieve collaborative operation between modules.

[0013] As a further option of the present invention, the quantization and dequantization module includes a reconfigurable logic region and a reconfigurable logic loading control block;

[0014] The reconfigurable logic loading control block is configured to generate a reconfigurable logic read address based on the model accuracy requirements, and dynamically load the reconfigurable logic of corresponding accuracy from the external storage device into the reconfigurable logic region.

[0015] As a further option of the present invention, the reconfigurable logic loading control block maintains a precision address mapping table, which records the starting address and data size of the reconfigurable logic file corresponding to each quantization precision in the external storage device.

[0016] As a further option of the present invention, the unified format data packet adopts a data packet-based interface protocol, and each data packet includes a header and a payload;

[0017] Among them, the Baotou bears control information and metadata, while the load bears valid data.

[0018] As a further option of the present invention, the metadata includes data length, data type and batch information, which is used to standardize data interaction between modules.

[0019] As a further option of the present invention, the control module receives a loading completion signal after the reconfigurable logic is loaded, and configures quantization parameters to the quantization and dequantization module;

[0020] The quantization parameters include the scaling factor of the weights, the zero point, the scaling factor of the activation value, and the quantization range.

[0021] As a further option of the present invention, the control module monitors the empty and full status of the input and output buffers of each module in real time, and dynamically adjusts the data flow scheduling strategy based on the status to ensure the continuity of data processing.

[0022] As a further option of the present invention, the post-processing operations performed by the post-processing module include at least one of applying an activation function, performing a Softmax operation, performing nonmaximum suppression, or data format conversion, to generate the final inference result.

[0023] The beneficial effects of this invention are:

[0024] The quantization / dequantization and inference logic of different precisions are separated from the entire model deployment system and integrated into dynamic reconfigurable logic. At runtime, the corresponding reconfigurable logic is loaded according to the model precision, so as to achieve compatibility and rapid dynamic adaptation to models with different precision requirements.

[0025] Quantization / dequantization and inference logic of different precisions are compiled into independent reconfigurable logic files, which realizes efficient time-division multiplexing of logic resources while being compatible with different precision models, effectively reducing resource consumption and overall power consumption.

[0026] The quantization / dequantization and inference logic of different precisions are all independent static area combinations. The quantization / dequantization and inference logic corresponding to each precision are synthesized and placed and routed separately. This effectively avoids the high fan-out and placement and routing congestion problems introduced by integrating all the above logic resources of all precisions together using the MUX method. It also ensures that the dynamic area logic corresponding to each precision runs under the optimal timing, which can further improve the system's operating frequency and optimize the overall performance. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the hardware architecture;

[0028] Figure 2 This is a schematic diagram illustrating the division of dynamic and static areas and the loading of the dynamic area;

[0029] Figure 3 This is a schematic diagram illustrating the process of switching from INT8 to BF16.

[0030] Figure 4 This is a diagram illustrating the data types used in the interfaces between modules.

[0031] Figure 5 It is a multi-buffer cache structure. Detailed Implementation

[0032] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. In addition, in order to clearly describe the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with basically the same function and effect.

[0033] Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or order of execution, and that "first," "second," etc., are not necessarily different. Furthermore, in some embodiments of this application, words such as "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0034] Furthermore, the device architecture and business scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of device architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0035] This disclosure relates to the fields of integrated circuit design and artificial intelligence hardware acceleration technology, and in particular to a hardware acceleration architecture that supports dynamic switching of multiple quantization precisions. With the widespread application of deep learning models in resource-constrained scenarios such as edge computing and mobile devices, higher requirements are placed on the energy efficiency, speed, and flexibility of model inference.

[0036] According to the inventive layout described in this disclosure, a hardware-accelerated architecture with adaptive quantization precision switching is provided. This hardware-accelerated architecture enables dynamic and seamless switching between different quantization precisions, thereby allowing for the compatible deployment of neural network models with multiple precisions on a single hardware platform. This architecture provides great flexibility while significantly reducing logic resource overhead and system power consumption, and optimizing data path timing.

[0037] In the example implementation, the described circuit architecture can be used with various hardware accelerators and the accelerator engines contained therein. A hardware accelerator having one or more accelerator engines implemented therein can be communicatively linked to the host central processing unit via a communication bus. The host CPU is capable of executing operating software that allows host applications executed by the host CPU to dynamically configure the quantization precision of the hardware accelerators based on the characteristics of the model to be deployed. Furthermore, the hardware acceleration architecture provides standardized hardware interfaces and dynamically reconfigurable computing cores to the accelerator engines, allowing the system to load the optimal precision processing unit as needed at runtime, thereby achieving high performance, low power consumption, and sustained data throughput.

[0038] Example 1: Figure 1The overall block diagram of the hardware-accelerated architecture with adaptive quantization precision switching proposed in this application is shown. The architecture can dynamically load and execute quantization, inference, and dequantization operations with different precisions according to model requirements.

[0039] like Figure 1 As shown, the architecture mainly consists of four core modules: a preprocessing module, a quantization and dequantization module, a post-processing module, and a control module. These four core modules work together to achieve adaptive accelerated processing throughout the entire process, from data input to result output.

[0040] The control module is responsible for the initial configuration, operational status monitoring, and overall scheduling of the entire system. For example... Figure 1 As indicated by arrow ①, the control module interacts with the preprocessing module, quantization and dequantization module, and postprocessing module. After the system is powered on or reset, the control module first initializes and configures each module. During operation, the control module continuously reads the operating status of each module and performs intelligent scheduling based on the operating status information to ensure smooth data flow and avoid blockage.

[0041] Specifically, the control module is responsible for parsing the model accuracy information from the host CPU or internal configuration memory and sending the accuracy command to the quantization and dequantization modules.

[0042] The preprocessing module is responsible for receiving the raw input data and performing necessary preprocessing operations. Preprocessing includes, but is not limited to, data format conversion, normalization, padding, and size scaling. Its core function is to uniformly convert diverse input data into a standardized format defined within the architecture. The preprocessing module connects to the quantization and dequantization modules via the data channel shown by arrow ③. Before data transmission, the preprocessing module packages the processed data according to a preset interaction interface format, forming a standard data packet. This data packet typically contains metadata such as data body, data length, data type, and batch information. As shown by arrow ②, there is a status and handshake signal channel between the preprocessing module and the quantization / dequantization modules for interaction between them.

[0043] The quantization and dequantization modules are the core and innovation of the entire architecture. Figure 1 Highlighted by a dashed box. The quantization and dequantization module is responsible for unpacking the standardized data packets from the preprocessing module and performing corresponding quantization, inference, and dequantization calculations according to the currently configured precision. Internally, the quantization and dequantization module is not a single fixed circuit, but rather consists of multiple reconfigurable logic units, abbreviated as .loc. For example... Figure 1 As shown, the reconfigurable logic units are labeled as 1.loc, 2.loc, 3.loc, ..., n.loc, with each .loc file corresponding to a specific quantization precision.

[0044] by Figure 1 Taking the 1.loc file as an example, its internal structure will be explained in detail. Each .loc unit mainly contains the following sub-components:

[0045] An input buffer is used to cache data packets transmitted from the preprocessing module through the data channel. Using an input buffer can effectively mitigate the processing speed differences between upstream and downstream modules, thereby improving system throughput.

[0046] The output buffer is used to temporarily store the result data after quantization, inference, and dequantization, and then pass it to the post-processing module.

[0047] Multiple buffer structures are typically used within the input buffer Ibuf and the output buffer Obuf. For example... Figure 1 As shown in B1 and B2, read / write wait times are avoided by cyclically switching the target buffer for read / write operations. While one buffer is being written to or read from, another buffer can be read from or written to simultaneously, thereby greatly improving the efficiency of the data processing pipeline.

[0048] The quantization unit is responsible for converting high-precision data or pre-processed normalized data from the input buffer into a low-precision fixed-point or floating-point format supported by the current .loc unit. The quantization process typically involves the calculation and application of scaling factors and zeros.

[0049] The inference unit is the hardware engine that performs the core neural network operations. It receives quantized, low-precision data and performs high-speed, efficient mathematical calculations. The inference unit can be a general-purpose vector processing unit or a dedicated circuit optimized for specific operators.

[0050] The dequantization unit is responsible for converting the low-precision results output by the inference unit back into a higher-precision format for processing by subsequent modules. Dequantization is the inverse process of quantization.

[0051] All .loc units are managed by a reconfigurable logic loading control block. The control module passes the parsed model accuracy requirements to this loading control block. The loading control block then generates the read address of the corresponding .loc file based on the accuracy information and dynamically loads the corresponding reconfigurable logic from the external boot device into the reconfigurable area of ​​the FPGA or similar programmable hardware.

[0052] The post-processing module is responsible for further processing the dequantized results output by the quantization and dequantization modules. This processing includes applying activation functions, performing softmax operations, applying non-maximum suppression, and converting the result data format to the format required for the final output. Similar to the pre-processing module, the post-processing module interacts with the quantization and dequantization modules through the data channel shown by arrow ⑤ and the status and handshake signal channel shown by arrow ④. Data transmission also follows a unified interaction interface format; the quantization and dequantization modules package and send the result data, which the post-processing module receives, unpacks, and processes.

[0053] In summary, the overall data flow of the architecture is as follows: the raw data stream enters the preprocessing module for preprocessing and format standardization, and is packaged into a fixed format; the packaged data is passed to the quantization and dequantization module through the data channel; the quantization and dequantization module performs unpacking, quantization, inference, and dequantization calculations on the data according to the currently loaded reconfigurable logic; the calculation results are packaged and passed to the post-processing module through the data channel; the post-processing module performs final processing and outputs the inference results.

[0054] Figure 2 This section provides a more detailed explanation of the division between dynamic and static regions in the architecture, as well as the loading process for quantization logic of different precisions. The entire hardware-accelerated architecture is divided into two main regions:

[0055] The static area contains common modules that need to run continuously and have fixed functions within the system. In this architecture, the pre-processing module, post-processing module, control module, and reconfigurable logic loading control block are placed in the static area. These modules are configured during system startup and remain stable throughout operation, ensuring the continuity of basic system functions and management.

[0056] The dynamic area is a resource area specifically reserved to carry reconfigurable logic. For example... Figure 2 As shown, multiple reconfigurable logic units of the quantization and dequantization modules are deployed in the dynamic area. The logic content of the dynamic area is dynamically erased and reloaded as needed.

[0057] The quantization and dequantization logics of different precisions are compiled into independent reconfigurable logic files, i.e., .loc files, during the compilation stage and stored in an external non-volatile boot device.

[0058] The specific process of loading the dynamic area is as follows, corresponding to Figure 3 Processes ① to ⑧ in the middle:

[0059] Process ①: Before the system runs, the logic in the static area is first configured, putting the preprocessing, post-processing, and control modules in an initialized and ready state. Then, when initializing the dynamic area or switching precision, the processor analyzes the precision requirements of the model to be deployed. The processor then passes the analyzed target precision information to the reconfigurable logic loading control block located in the static area.

[0060] Process ②: The reconfigurable logic loading control block searches the internal address mapping table based on the target precision information and generates the corresponding .loc reconfigurable logic file's read address in the external boot device.

[0061] Process ③: The reconfigurable logic loading control block initiates a read request to the external boot device through the memory interface. The external boot device, based on the provided address, sends back the corresponding reconfigurable logic file. The reconfigurable logic data is then loaded into the hardware resources of the dynamic area via the configuration port.

[0062] Process 4: Once the target reconfigurable logic is successfully loaded into the dynamic area and configured, the reconfigurable logic loading control block will send a loading completion signal to the processor.

[0063] Step 5: After receiving the loading completion signal, the processor confirms that the hardware logic of the quantization and dequantization modules is in place. Next, the processor needs to configure the vectorization and dequantization modules with specific quantization parameters related to accuracy. These parameters include, but are not limited to: the scaling factor and zeros of the weights, the scaling factor and zeros of the activation values, and the quantization range. These parameters are written into the configuration registers within the quantization and dequantization modules.

[0064] Processes 6 and 7: Once the quantization parameters are configured, the entire quantization and dequantization module enters a normal operating state. At this time, the control module can schedule the preprocessing module to start inputting data. The data flow begins to follow the aforementioned path: the preprocessing module processes and packages the data (process 6), and then sends the data to the quantization and dequantization module (process 7).

[0065] Process ⑧: The quantization and dequantization module processes the data using the newly loaded hardware logic and configured parameters, and outputs the final inference result to the post-processing module.

[0066] Through the aforementioned dynamic reconfiguration mechanism, this architecture achieves adaptive switching of quantization precision at the hardware level. When different precision models need to be run, it is not necessary to reprogram the entire chip; simply load the corresponding .loc file and configure the appropriate parameters, greatly improving deployment flexibility and efficiency.

[0067] supply Figure 1 and Figure 2The architecture and operations described herein are for illustrative purposes only and not as limiting. It should be understood that the architecture described above can be extended to support more types of quantization precision, more complex pre- and post-processing functions, or integrated into larger heterogeneous computing systems. The various parts of the method can also operate concurrently or in parallel with other parts. For example, while the dynamic region loads logic for a certain precision, the static region can continue to handle other administrative tasks or communicate with the host; or, while the previous inference task is still in the post-processing module, new data can be processed by the pre-processing module and cached in the input buffer of the quantization module.

[0068] In one or more example implementations, the hardware acceleration architecture described above can be implemented on a programmable integrated circuit, exemplarily a field-programmable gate array (FPGA). In this case, the preprocessing module, quantization and dequantization module, post-processing module, and control module are implemented using the programmable circuitry of the IC. The loading process of the reconfigurable logic is the dynamic reconfigurable logic programming process for the reconfigurable regions of the FPGA.

[0069] On the other hand, hardware-accelerated architectures can also be implemented as application-specific integrated circuits (ASICs). In ASIC implementations, the concept of dynamic reconfiguration can be achieved through techniques such as multiplexing or power gating. For example, quantization, inference, and dequantization circuits of different precisions can all be fabricated on a single chip, forming a hardware resource pool. Then, a centralized configuration network selects the computation path corresponding to the precision based on the precision instruction, and shuts down other paths to save power. In this case, the loading process becomes the selection and activation of internal circuit paths.

[0070] On the other hand, the architecture can function as part of a hardware acceleration device, connecting to the host computer via a high-speed bus such as PCIe. Drivers and application software on the host computer are responsible for sending model accuracy instructions and quantization parameters to the architecture's control modules.

[0071] Optionally, the above and other embodiments may each include one or more of the following features individually or in combination. Some example embodiments include all of the following features in combination.

[0072] In one aspect, the unified format data interface is a packet-based interface, where each packet contains a header and a payload. The header is used to transmit control information and metadata, while the payload carries the actual data.

[0073] On the other hand, the reconfigurable logic loading control block maintains a precision address mapping table, which records the starting address and size of the reconfigurable logic file corresponding to each quantization precision in the external boot device.

[0074] On the other hand, during scheduling, the control module monitors the empty / full status of the input and output buffers. If the input buffer is about to fill up, the control module can pause the data inflow from the preprocessing module; if the output buffer is about to empty, the control module can notify the postprocessing module to prepare to receive data or temporarily enter a low-power state. This state-based flow control mechanism ensures the efficient and stable operation of the system.

[0075] On the other hand, the configuration parameters of the quantization unit and the dequantization unit are dynamically updated by the processor at runtime, thereby supporting more complex quantization strategies, such as per-pass quantization, dynamic quantization, etc.

[0076] On the other hand, the architecture also includes a performance monitoring unit for collecting performance metrics such as throughput, latency, and power consumption of each module in real time. The control module or external processor can intelligently select the quantization precision best suited to the current power consumption and performance requirements based on these metrics, achieving finer-grained adaptive optimization.

[0077] The inventive layout described herein provides a highly flexible, efficient, and low-power hardware acceleration architecture. It successfully addresses the limitations of fixed-precision hardware in handling diverse AI models, providing a powerful hardware platform solution for intelligent computing at both the edge and device sides.

[0078] Example 2: As Figure 1 As shown, this application mainly comprises four parts: a preprocessing module, a quantization and dequantization module, a postprocessing module, and a control module. The preprocessing module is mainly responsible for preprocessing the data to be processed and converting it into the unified format required by this application; the quantization and dequantization module is responsible for quantizing, inferring, and dequantizing the preprocessed data; the postprocessing module is responsible for further processing the dequantized results according to the model requirements; and the control module is responsible for the configuration and overall scheduling of the entire system.

[0079] The following uses an image recognition process as a specific example to describe the processing procedure of the above modules in detail. When using a model to recognize or extract features from an image, the preprocessing module first performs processing such as cropping and reshaping on the input image, converting the original image into a size that meets the model's input requirements. Then, it normalizes and converts the data type of the image to be processed using the following formula:

[0080]

[0081] Where X norm This represents the normalization result, where x represents the current image pixel value, and Mean and STD are normalization parameters configured by the processor as needed. Based on the above calculation process, the image to be processed is normalized, and the data type is converted from the original image's UINT8 format to FP32 format.

[0082] The converted image data is packetized according to the specified transmission bit width and passed to the subsequent quantization / inverse quantization module. The quantization module quantizes the input FP32 format raw data. Taking INT8 quantization as an example, the quantization process can be represented by the following formula:

[0083]

[0084] In the above formula, INT8 out The quantization result for INT8, FP32 in The input is in FP32 format, where S is the scaling factor and Z is the zero point. Both S and Z are in FP32 format, configured by the processor. `Round()` rounds to the nearest integer, and `Clip()` is the saturation function, limiting the result to the INT8 range. In hardware implementation, the above process is broken down into a three-stage pipeline: the first stage performs multiplication and addition of FP32 floating-point numbers; the second stage performs the conversion and rounding from floating-point to fixed-point input; and the third stage performs the saturation and output of the result. Furthermore, to avoid the division operation introduced by S, (1 / S) is treated as a whole and sent to simplify the hardware implementation.

[0085] The dequantization process from INT8 to FP32 can be represented by the following formula:

[0086]

[0087] INT8 in This represents the inverse quantization result input of type INT8, FP32. out This indicates the output of the FP32 type dequantization result, where Z represents the quantization zero point and S represents the scaling factor. Both S and Z are in FP32 format and are configured uniformly by the processor. In hardware implementation, the above process is also broken down into a three-stage pipeline: the first stage converts the INT8 quantized value into a 32-bit floating-point number; the second stage performs floating-point subtraction to subtract the zero-point offset; and the third stage performs floating-point multiplication, multiplying by the scaling factor S.

[0088] The quantization and dequantization processes for other precision levels are similar to those for INT8 precision. When, in practical applications, INT8 precision cannot meet the detection accuracy requirements, necessitating a switch to higher precision quantization / dequantization logic; or when the precision of the INT8 precision quantization / dequantization process exceeds the actual accuracy requirements, and lower precision quantization / dequantization logic is needed to reduce power consumption, then... Figure 2 The reconfigurable logic switching and loading process dynamically switches and adapts the quantization / dequantization logic. The specific process is as follows: Figure 3 As shown: Figure 3The demonstration shows the switching process from INT8 precision quantization / dequantization to BF16 precision quantization / dequantization. The processor sends the new precision requirement to the reconfigurable logic loading control block, which generates the read address of the corresponding reconfigurable logic file, reads the corresponding reconfigurable logic file from the external boot device, and loads it. After loading, the reconfigurable logic loading control block sends a loading completion signal back to the processor. Upon receiving the loading completion signal, the processor resends the corresponding precision quantization configuration parameters to the new quantization / dequantization module. After configuration, the system restarts, using the new precision to quantize, infer, dequantize, and output the data. The entire process does not require recompiling or reprogramming the hardware, nor does it change the logic and state of the static area; the entire switching process can be controlled within milliseconds.

[0089] To ensure compatibility with quantization / dequantization logic of varying precision, a unified interface and data type are used for data transfer between the quantization / dequantization module and its preceding and following modules. The specific process is as follows: Figure 4 As shown. Figure 4 In the diagram, ① and ② represent handshake signals between the pre-processing and post-processing stages. Data transmission can only begin after a successful handshake. ③ represents the input to the pre-processing module. The data type of the input to the pre-processing module is not restricted here; it can be uint8 / uint16 / fp32 / … etc. These types of input need to be converted and packaged in the pre-processing module to ultimately generate fp32 type data, which is then concatenated into 512 bits and passed to the quantization / dequantization module through the interface shown in ④. The dequantized output data is also in fp32 format, concatenated into 512 bits, and passed to the post-processing module through the interface shown in ⑤. The output of the post-processing module is also not restricted; it can be converted according to the final output requirements within the post-processing module, such as… Figure 4 As shown in ⑥.

[0090] To improve the processing efficiency between the quantization / dequantization module and the preceding and following stages, and to reduce the overall latency, input and output multi-buffer management is added to the quantization / dequantization module. Figure 5 The diagram in the middle shows a multi-buffer structure. Figure 5 Both input and output use 3-buffer buffers. Reading and writing switch cyclically between the 3 buffers to reduce pipeline processing latency. Unified scheduling and control are achieved through buffer scheduling control logic. The processor adjusts the input rate of the previous stage or the output rate of the next stage according to the state of the buffer scheduling control logic. Figure 5 Q, IF, and UQ in the equation correspond to the quantization, inference, and dequantization processes for each precision, respectively.

[0091] Those skilled in the art will understand that the embodiments of this application are provided as methods, systems, or computer program products. Therefore, this application takes the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application takes the form of a computer program product implemented on one or more computer storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer program code. The solutions in the embodiments of this application are implemented using various computer languages, exemplified by the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0092] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0093] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A quantization precision adaptive switching hardware acceleration architecture, comprising: include: The preprocessing module receives the raw input data, performs preprocessing operations, packages the processed data into a data packet of a uniform format, and outputs it through the data channel. The quantization and dequantization module receives the data packet, performs unpacking, quantization, inference, and dequantization processing on the data according to the current precision configuration, and outputs the processing results as a data packet in a unified format. The post-processing module receives the data packets output by the quantization and dequantization modules, performs post-processing operations, and outputs the inference results. The control module, which communicates with the preprocessing module, the quantization and dequantization module, and the postprocessing module, is configured to: analyze the model accuracy requirements; dynamically configure the hardware logic of the quantization and dequantization modules according to the model accuracy requirements to adapt to the target quantization accuracy; and schedule the data flow through each module to avoid data blockage. Among them, the preprocessing module, quantization and dequantization module and postprocessing module transmit data packets of the same format through the data channel, and exchange control information through the status and handshake signal channel to realize the collaborative operation between modules; The quantization and dequantization module includes a reconfigurable logic region and a reconfigurable logic loading control block; The reconfigurable logic loading control block is configured to generate a reconfigurable logic read address based on the model accuracy requirements, and dynamically load the reconfigurable logic of corresponding accuracy from the external storage device into the reconfigurable logic region.

2. The quantization precision adaptive switching hardware acceleration architecture of claim 1, wherein, The reconfigurable logic loading control block maintains a precision address mapping table, which records the starting address and data size of the reconfigurable logic file corresponding to each quantization precision in the external storage device.

3. The quantization precision adaptive switching hardware acceleration architecture of claim 1, wherein, The unified format data packets use a data packet-based interface protocol, and each data packet includes a header and a payload; Among them, the head carries control information and metadata, while the load carries valid data.

4. The quantization precision adaptive switching hardware acceleration architecture of claim 3, wherein, The metadata includes data length, data type, and batch information, which is used to standardize data interaction between modules.

5. The quantization precision adaptive switching hardware acceleration architecture of claim 1, wherein, After the reconfigurable logic is loaded, the control module receives the loading completion signal and configures the quantization parameters to the quantization and dequantization module. The quantization parameters include the scaling factor of the weights, the zero point, the scaling factor of the activation value, and the quantization range.

6. The quantization precision adaptive switching hardware acceleration architecture of claim 1, wherein, The control module monitors the empty and full status of the input and output buffers of each module in real time, and dynamically adjusts the data flow scheduling strategy based on the status to ensure the continuity of data processing.

7. The quantization precision adaptive switching hardware acceleration architecture of claim 1, wherein, The post-processing module performs post-processing operations including applying an activation function, performing a Softmax operation, performing nonmaximum suppression, or converting the data format to generate the final inference result.

Images