Design method, device, medium and product of bit-level combinable accelerator

Abstract

The present invention relates to the field of computer technology, and discloses a design method, device, medium, and product for a bit-level combinable accelerator. The method comprises: performing bit-level combinable hardware modeling on the accelerator to be designed to obtain a bit-level combinable hardware structure; performing bit-level data flow modeling on the bit-level combinable hardware structure by fusing software and hardware parameterized descriptions to obtain data flow description parameters; establishing a design space based on the data flow description parameters, and obtaining a target data flow by searching the design space for the best design point. The solution of this embodiment, by performing bit-level combinable hardware modeling, bit-level data flow modeling, and hardware scheduling design search, can add bit-level computing support to the hardware design and data scheduling design of the accelerator, can provide quantitative evaluation and automated guidance for hardware and scheduling design, and can realize automatic hardware design and scheduling optimization of the bit-level combinable accelerator.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a design method, device, medium and product of a bit-level combinable accelerator. Background Art

[0002] Neural networks are widely used in fields such as image classification, recommendation systems, object detection, and natural language understanding. Behind their remarkable achievements lies complex and deep network models, massive parameters, and a significant amount of computational effort. Large models can contain hundreds of millions of parameters, and the increasing computational effort increases model training and inference time, resource consumption, and costs.

[0003] To increase the speed of neural networks while reducing costs, model quantization has become a relatively effective solution. Model quantization involves replacing high-bit-width floating-point numbers with low-bit-width fixed-point numbers or integers as network parameters and input data. On the one hand, a smaller data bit width reduces the amount of data and computation required, significantly improving performance and power consumption. Furthermore, a lower bit width helps reduce hardware complexity. On the other hand, reduced data precision can lead to a decrease in model accuracy. The trade-off between reduced accuracy and performance gains must be considered during quantization. Because neural networks are tolerant of data precision, data bit width can be significantly reduced without exceeding a certain accuracy loss threshold. After model quantization, the ideal minimum bit width varies between different network models. Even different parts and layers of the same network model may require different bit widths, resulting in tasks with variable bit widths or multiple bit width requirements.

[0004] In order to efficiently process neural networks of different bit widths, hardware support acceleration and software scheduling optimization have become key research topics, namely the design of efficient hardware accelerators that support multiple data bit widths and the scheduling optimization of neural networks on this hardware. Existing machines with fixed high bit widths obviously cannot take advantage of low bit width and variable bit width calculations. In most devices, low-precision data still needs to be expanded to the machine bit width before calculation. In graphics processors, there are multiple computing cores with different precision sizes that can fully utilize low bit width data, but designing a separate type of computing core for each precision will result in a waste of resources, extremely low hardware utilization, and high cost. Driven by the above problems, there is a need for dedicated hardware devices that can support multiple data bit widths at the same time, and some hardware accelerator design work has already been done.

[0005] To achieve a single architecture supporting calculations with multiple bit widths, hardware typically employs a method similar to "large number multiplication," combining "small number" operations to implement "large number" operations. High-bit-width data is split into multiple groups of lower-bit-width data, or even single-bit data, which are then processed through multiple stages to form multi-bit calculations. Hardware that supports this type of calculation is called a bit-level composable architecture. This composable architecture adapts well to scenarios with varying bit-width requirements. When high precision is required, low-precision calculations can be combined to create high-precision calculations. When low precision is required, calculations can be performed directly using low-precision components, eliminating the waste of hardware resources. Compared to providing separate computational components for different bit widths, a bit-level composable architecture allows for the execution of varying numbers of multiplication and addition calculations of varying sizes on the same device. Hardware can be dynamically configured to operate as multiplication and addition units of varying bit widths, meeting varying bit width requirements while improving component utilization and throughput.

[0006] Although the composable architecture has additional overhead in the microstructure, this does not prevent data bit width from being a consideration dimension in hardware design, and some accelerator designs have been able to gain benefits from the combined structure. In addition to hardware support, the data scheduling process of neural network applications on hardware affects the final operating efficiency. Scheduling, also known as data flow, describes when data is transmitted to various levels of storage and computing units, and describes the scale and flow of data. In actual deployment, scheduling is reflected in the reordering of instructions or data. Obviously, the same application has many different scheduling methods, and the data scheduling process of the application needs to be optimized during design. Furthermore, when optimizing scheduling, the characteristics of the underlying hardware should be considered, and the underlying hardware resources should be used rationally. At present, there has been work in the field to evaluate and analyze the execution effect of neural networks on hardware, as well as methods and frameworks for optimizing scheduling, but there are important problems in past work:

[0007] First, some hardware and scheduling designs lack support for bit-level computing, meaning they lack support for computing at multiple bit widths. To accelerate neural network inference and reduce costs, bit-level computing has become a key design consideration. Low bit-width computing directly improves performance and reduces power consumption, and it also helps reduce hardware complexity. Neural network quantization methods have created a practical need for bit-level computing, and neural networks with varying bit widths have become more common. Beyond neural networks, tensor computing in a broader sense also requires computing at multiple bit widths.

[0008] Secondly, traditional neural network accelerators supporting various precisions are based on empirical, qualitative designs. Hardware architectures involve numerous variations and trade-offs, and the component types, quantities, interconnect topologies, and storage structures are difficult to determine. Feasible designs are typically developed empirically. The design process becomes even more complex when considering bit-level computation and the need for various bit widths. Previous accelerator core designs were manually designed based on application characteristics, which was time-consuming, labor-intensive, and inefficient. The rapid development of neural network applications also demands rapid updates and iterations of the underlying hardware. Due to the time-consuming nature of hardware verification, synthesis, simulation, and emulation, some design trade-offs are difficult to quickly assess.

[0009] Finally, even after the hardware architecture is determined, the scheduling process needs to be designed and optimized for different applications. Previously, accelerator design scheduling was manually designed and lacked scheduling optimization. New applications required manual design of new schedules. The inefficiency of hardware design and scheduling optimization severely hindered the iteration of hardware and software. While data flow modeling, evaluation, and design automation efforts have alleviated this problem, these efforts lack bit-level computational support and cannot be applied to scenarios involving bit-level composable architectures. Summary of the Invention

[0010] The present invention provides a design method, device, medium and product for a bit-level combinable accelerator, which can add bit-level computing support to the hardware design and data scheduling design of the accelerator, provide quantitative evaluation and automated guidance for hardware and scheduling design, and realize automatic hardware design and scheduling optimization of the bit-level combinable accelerator.

[0011] According to one aspect of the present invention, a method for designing a bit-level combinable accelerator is provided, comprising:

[0012] Perform bit-level composable hardware modeling on the accelerator to be designed to obtain the bit-level composable hardware structure;

[0013] According to the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters;

[0014] A design space is established according to the data flow description parameters, and a target data flow is obtained by searching for an optimal design point in the design space.

[0015] According to another aspect of the present invention, a device for designing a bit-level combinable accelerator is provided, comprising:

[0016] A hardware modeling module is used to perform bit-level composable hardware modeling on the accelerator to be designed and obtain the bit-level composable hardware structure;

[0017] A data flow modeling module is used to perform bit-level data flow modeling based on the bit-level combinable hardware structure by fusing software and hardware parameterized descriptions to obtain data flow description parameters;

[0018] The design point search module is used to establish a design space according to the data flow description parameters, and obtain the target data flow by searching the design space for the best design point.

[0019] According to another aspect of the present invention, an electronic device is provided, comprising:

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method for designing a bit-level combinable accelerator according to any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the method for designing a bit-level combinable accelerator according to any embodiment of the present invention when executed.

[0024] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method for designing a bit-level combinable accelerator according to any embodiment of the present invention.

[0025] The technical solution of the embodiment of the present invention is to obtain a bit-level combinable hardware structure by performing bit-level combinable hardware modeling on the accelerator to be designed; based on the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters; a design space is established according to the data flow description parameters, and the target data flow is obtained by searching the design space for the best design point; by performing bit-level combinable hardware modeling, bit-level data flow modeling and hardware scheduling design search, bit-level computing support can be added to the hardware design and data scheduling design of the accelerator, quantitative evaluation and automated guidance can be provided for hardware and scheduling design, and automatic hardware design and scheduling optimization of the bit-level combinable accelerator can be realized.

[0026] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 This is a flow chart of a method for designing a bit-level combinable accelerator according to a first embodiment of the present invention;

[0029] Figure 2 2 is a schematic diagram of a bit-level combinable hardware structure provided according to the first embodiment of the present invention;

[0030] Figure 3 is a calculation flow chart of the bit grouping and vectorization method provided according to the first embodiment of the present invention;

[0031] Figure 4 is a calculation flow chart of a vector dot product or multiplication-addition operation according to the first embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of a storage structure design provided according to the first embodiment of the present invention;

[0033] Figure 6 is a schematic diagram of a bit width configuration algorithm provided according to embodiment 1 of the present invention;

[0034] Figure 7 is a schematic diagram of a shifter type configuration algorithm provided according to embodiment 1 of the present invention;

[0035] Figure 8 1 is a schematic diagram of a shifter multiplexing process according to a first embodiment of the present invention;

[0036] Figure 9 Schematic diagram of a data multiplexing design solution provided according to the first embodiment of the present invention;

[0037] Figure 10 is a schematic diagram of a hardware parameter expression provided according to the first embodiment of the present invention;

[0038] Figure 11 is a schematic diagram of an array arrangement index and a data sharing area provided in Embodiment 1 of the present invention;

[0039] Figure 12 Schematic diagram of an operand index relationship matrix provided according to the first embodiment of the present invention;

[0040] Figure 13 is a schematic diagram of hardware description parameters provided according to embodiment 1 of the present invention;

[0041] Figure 14 is a schematic diagram of a parameterized description of a classic accelerator topology provided according to the first embodiment of the present invention;

[0042] Figure 15 This is a diagram illustrating an example of a data flow provided according to the first embodiment of the present invention;

[0043] Figure 16 is a flowchart of a hardware and scheduling design search method provided in accordance with a first embodiment of the present invention;

[0044] Figure 17 is a schematic diagram of a backtracking pruning search algorithm provided according to the first embodiment of the present invention;

[0045] Figure 18 2 is a schematic diagram of heterogeneity analysis verification results provided in accordance with the first embodiment of the present invention;

[0046] Figure 19 2 is a schematic diagram of data flow for optimal scheduling provided by embodiment 1 of the present invention;

[0047] Figure 20 2 is a schematic diagram of reliability verification results provided according to the first embodiment of the present invention;

[0048] Figure 21 2. It is a schematic diagram of the neural network structure parameters provided according to the first embodiment of the present invention;

[0049] Figure 22 2 is a schematic diagram of the evaluation results of the design search verification provided in accordance with the first embodiment of the present invention;

[0050] Figure 23 2 is a schematic structural diagram of a device for designing a bit-level combinable accelerator according to a second embodiment of the present invention;

[0051] Figure 24 It is a schematic structural diagram of an electronic device for implementing the method for designing a bit-level combinable accelerator according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first," "second," "target," etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products, or apparatus.

[0054] Example 1

[0055] Figure 1 A flowchart of a method for designing a bit-level combinable accelerator is provided for the first embodiment of the present invention. This embodiment is applicable to the case of hardware design and scheduling optimization of a bit-level combinable accelerator. The method can be executed by a design device for a bit-level combinable accelerator. The design device for a bit-level combinable accelerator can be implemented in the form of hardware and / or software. Typically, the design device for a bit-level combinable accelerator can be configured in an electronic device, such as a computer device or a server. Figure 1 As shown, the method includes:

[0056] S110 , performing bit-level composable hardware modeling on the accelerator to be designed to obtain a bit-level composable hardware structure.

[0057] In this embodiment, bit-level composable hardware modeling is performed to achieve bit-level composable hardware structure abstraction, supporting multiplication and addition operations on data of different bit widths while maintaining hardware resource utilization. Hardware structure abstraction can include computational component design, shifter optimization through component reuse, and bandwidth and complexity optimization through data reuse.

[0058] Specifically, the accelerator to be designed can be modeled by bit-level composable computing unit modeling and storage structure modeling in sequence to obtain a comprehensive composable design, component structure design, bandwidth design, and storage structure design. Then, based on the above design, multi-layer bit-level spatial accelerator modeling can be performed to give a bit-level composable hardware structure abstraction, thereby obtaining a bit-level composable hardware structure. For example, the bit-level composable hardware structure can be as follows Figure 2As shown in Figure 1, a basic bit-level composable accelerator can be composed of a computing array, on-chip interconnect, and on-chip storage. The computing array is an array of PEs (Processing Elements), which are bit-level composable structures. The on-chip interconnect can be a memory access control module or a combination of data feed methods. The relevant parameters of this basic structure include storage size and maximum bandwidth, interconnect topology and broadband latency, and the computing array number of PEs, register size, vector length, and width.

[0059] S120 , performing bit-level data flow modeling based on the bit-level combinable hardware structure by fusing software and hardware parameterized descriptions to obtain data flow description parameters.

[0060] Specifically, based on the acquisition of the bit-level combinable hardware structure, the tensor computing application and the bit-level combinable hardware can be parameterized to obtain software description parameters and hardware description parameters; then, by fusing the software description parameters, hardware description parameters and bit-level calculations, the execution process of the application on the hardware can be parameterized to obtain data flow description parameters, for example, the process and parameter items required to describe the data flow.

[0061] S130 , establishing a design space according to the data flow description parameters, and obtaining a target data flow by searching for an optimal design point in the design space.

[0062] In this embodiment, the hardware and scheduling design search can be divided into two parts: design space construction and space search. Specifically, all feasible data flows can be automatically generated based on the data flow description parameters to construct a design space. For example, different feasible data flows can be generated by assigning different parameter values ​​to each parameter in the data flow description parameters. Among them, when constructing the design space, the software description parameters can be used as the main input, and the hardware description parameters other than the number of multipliers can be used as optional inputs in the form of constraints. These feasible data flows can then be evaluated, for example, based on indicators such as hardware utilization efficiency and delay, to search for the best design point in the design space, thereby finding the feasible data flow with the best evaluation result as the final target data flow.

[0063] It can be understood that based on the determined target data flow, the hardware topology and scale of the accelerator, as well as the data scheduling process, can be determined, thereby realizing the automatic design of the accelerator.

[0064] The technical solution of the embodiment of the present invention is to obtain a bit-level combinable hardware structure by performing bit-level combinable hardware modeling on the accelerator to be designed; based on the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters; a design space is established according to the data flow description parameters, and the target data flow is obtained by searching the design space for the best design point; by performing bit-level combinable hardware modeling, bit-level data flow modeling and hardware scheduling design search, bit-level computing support can be added to the hardware design and data scheduling design of the accelerator, quantitative evaluation and automated guidance can be provided for hardware and scheduling design, and automatic hardware design and scheduling optimization of the bit-level combinable accelerator can be realized.

[0065] In an optional implementation of this embodiment, bit-level combinable hardware modeling is performed on the accelerator to be designed to obtain the bit-level combinable hardware structure, which may include: based on data bit grouping and vectorization methods, bit-level combinable computing unit modeling is performed to obtain a computing unit structure design, and storage structure modeling is performed to obtain a storage structure design; and according to the computing unit structure design and the storage structure design, the bit-level combinable hardware structure is obtained.

[0066] In this embodiment, for the data bit grouping and vectorization method, the bit grouping and vectorization process can be given by mathematical symbols, providing a theoretical basis for bit-level calculation and modeling of combination structures. For example, the process of the bit grouping and vectorization method can be as follows Figure 3 As shown in the figure, G represents the bit group size, that is, the bit width of the data after grouping. After the bit grouping, it can continue to participate in external vector multiplication and addition calculations. At this time, the data has an additional dimension. The new dimension brings new calculation methods and introduces new shift operations when performing multiplication. After the data is vectorized, the calculation process of the classic vector dot product or multiplication and addition operation can be as follows Figure 4 As shown. This calculation formula can be converted into another calculation method by transforming the inner and outer layers of the summation sign, namely

[0067] Secondly, based on the data bit grouping and vectorization methods, bit-level composable computing unit modeling can be carried out. Typically, a bottom-up approach can be adopted to build a composable computing unit design paradigm, enabling the computing unit (PE) to support a variety of different bit widths while maintaining hardware resource utilization.

[0068] It should be noted that the design goal of the bit-level combinable computing unit is to allow the computing unit PE to support a variety of different bit widths, and the constraint is to keep the hardware fully loaded as much as possible. In order for PE to support low-bit-width data operations, the bottom layer must be a low-bit-width computing unit, and operations with a bit width of G can only be combined into operations with a bit width that is an integer multiple of G. By configuring the shifter, multiplication and addition operations with different vector lengths and different bit widths can be achieved. In this embodiment, the number of multipliers and the bit grouping size can be determined according to the initial structure of the accelerator to be designed, and the required shifter configuration can be automatically calculated based on the number of multipliers and the bit grouping size, thereby realizing the modeling of the bit-level combinable computing unit and obtaining the computing unit structure design.

[0069] In this embodiment, while modeling the bit-level combinable computing unit, it is also possible to model the storage structure based on a preset storage body structure, and use parameters to describe the storage structure to obtain a storage structure design. Optionally, the storage structure design may include a global storage structure design using multi-level cache and a local storage structure design using separate storage. This embodiment proposes the idea of ​​multi-level cache design and separate storage, which can utilize data access locality and data reuse to reduce repeated and inefficient external memory accesses. At the same time, through a double buffer design, memory access and calculation can be pipelined to increase hardware execution efficiency. The accelerator in this embodiment can adopt a storage structure design with multi-level cache, separate storage, and a double buffer pipeline design, which includes registers, local storage around the computing unit, on-chip global storage, and external storage from bottom to top.

[0070] In a specific example, the storage structure design can be as follows Figure 5 As shown in the figure, the left side shows multi-level cache and separate storage. A global memory bank is located on the hardware chip, which interacts with external memory and sends data to local storage. Local storage uses separate storage, with each type of data corresponding to a separate memory bank. Below the local storage are the compute units, which store data into the compute unit registers, perform calculations, and then write it back. Memory hotspots in the multi-level cache—those locations with intensive data reads and writes—are concentrated in the more efficient storage at the bottom layer, with memory access becoming less efficient as you move further out. The arrows in the figure represent memory buses or specific data transmission paths.

[0071] Secondly, the right side of the figure illustrates a double-buffered "ping-pong" mechanism. The memory banks are doubled, with two memories performing the same function. During memory access, the two memories perform reads and writes separately. This eliminates structural conflicts between the inner and outer layers, allowing for streamlined memory access. The inner layer can serve as a computational unit, thus streamlining both memory access and computation.

[0072] Optionally, based on the data bit grouping and vectorization method, bit-level combinable computing unit modeling is performed to obtain the computing unit structure design, which may include: obtaining the number of multipliers and the bit group size, and calculating the combined bit width configuration according to the number of multipliers and the bit group size through a bit width configuration algorithm; obtaining the shifter type configuration according to the combined bit width configuration by adopting spatial multiplexing and time multiplexing of the shifter; obtaining a data multiplexing design by combining internal and external parallel data multiplexing, and obtaining the computing unit structure design according to the shifter type configuration and the data multiplexing design.

[0073] In this embodiment, the bit width configuration algorithm can be as follows: Figure 6 As shown, the input is the number of multipliers and the bit grouping size, and the output is the combined bit width configuration, including the number of shifts, the number of shifters required, etc. It is understandable that the number and type of shifters determine the achievable bit width configuration. To reduce hardware complexity, shifters generally cannot support an arbitrary number of shifts, but rather support a few specific types of shifts, such as 2-bit and 4-bit left shifts. To obtain a more efficient computing component, this embodiment proposes a shifter multiplexing structure.

[0074] In order to reduce the number of shifters as much as possible, use fixed-width shifting as much as possible, try to reuse the shifter in the operation, and reduce the complexity of the calculation components, the shifter type configuration algorithm can be as follows Figure 7 As shown in the figure, the algorithm takes as input a list of shift numbers, shiftlist, and outputs a list of shifter types. The shift list specifies the number of left shifts required for each subsequent multiplication to achieve a specific bitwidth. If this list is viewed as a matrix, the number of rows equals the number of configured bitwidths, and the number of columns equals the number of multipliers.

[0075] First, the algorithm sorts the rows of the matrix and then permutes the rows to produce an ordered matrix. Each row and column of an ordered matrix is ​​an ordered sequence, with each column representing the total number of shifts a shifter needs to support. Next, the matrix is ​​grouped and packed by column. The packing process combines consecutive, identical columns into a single column. Identical columns mean that the shifter configurations are identical, sufficing for a single shifter. Multiple multiplications can share a single shifter. After multiplication, addition can be performed first, followed by shifting. This order of operations, from multiplication, shift, and addition, becomes multiplication and addition first, followed by shifting. This optimization enables component reuse by sharing a single shifter, also known as shifter spatial multiplexing. Spatial multiplexing reduces the number of shifters and reduces hardware complexity. The packed matrix is ​​then subtracted in reverse column order. This subtraction process is used to extract the commonality between two different types of shifters and reuse this commonality, also known as shifter temporal multiplexing. The purpose of time multiplexing is to reduce the complexity of the shifter and try to convert multiple shifts into a fixed number of shifts. For example, the process of shifter multiplexing can be as follows: Figure 8 shown.

[0076] In addition, in addition to the difference in vector length and bit width, another important difference between different bit width configurations is the different bandwidth requirements. In order to alleviate the problem of bandwidth imbalance in different bit width configurations, this embodiment proposes a data multiplexing hardware design method. The core idea is to combine internal and external data parallelism, incorporate data multiplexing into hardware design, increase the number of operations performed on a single data access, and thus alleviate bandwidth pressure. For example, the data multiplexing design solution can be as follows Figure 9 shown.

[0077] In another optional implementation of this embodiment, according to the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters, which may include: performing software and hardware modeling according to the bit-level combinable hardware structure to obtain software description parameters and hardware description parameters; wherein the software description parameters include tensor operation loop body characteristic parameters, the tensor operation loop body characteristic parameters include loop range, stride and tensor index relationship matrix, the hardware description parameters include multi-level hardware structure parameters, topology description and operand index relationship matrix, the topology description includes array arrangement index and data sharing area; scheduling process modeling is performed according to the software description parameters and the hardware description parameters to obtain an intermediate expression primitive format, and data flow description parameters are obtained according to the intermediate expression primitive format.

[0078] In this embodiment, bit-level dataflow modeling is proposed, which integrates software and hardware features with bit-level computation to describe the execution process of applications on hardware and enable quantitative evaluation of the running processes of different applications and hardware structures. Typically, bit-level dataflow modeling can include mapping intermediate expression primitives, complete dataflow description, and index matching. Bit-level dataflow modeling is based on software and hardware description parameters and is targeted at tensor computing applications and multi-layer array-type bit-level composable hardware structures, reflecting when and where tensor data participates in computation.

[0079] Specifically, bit-level data flow modeling can include two stages: software and hardware modeling and scheduling process modeling. Software and hardware modeling primarily involves parameterized descriptions of tensor computing applications and bit-level composable hardware. This includes loop computational features of neural network operators and general tensor computing applications in software descriptions, as well as parameterization of multi-level hardware structures, topological descriptions, and operand index relationship matrices in hardware descriptions.

[0080] When parameterizing a tensor operation loop, the loop body is determined by two parameters: the loop range and the stride. The computational statements are determined by the tensor index relationship matrix. The tensor index matrix is ​​a relationship matrix that describes the correspondence between tensors and indices, with indices corresponding one-to-one to computational statements. When an index has an access relationship to a tensor, that is, when the tensor subscript contains the index, the corresponding position in the matrix is ​​1, indicating that the computational statement corresponding to the index should be used. If the tensor subscript does not contain the index, the corresponding position in the matrix is ​​0.

[0081] Secondly, when describing hardware abstraction parameterized, the main part of the hardware description can use nested definitions. For example, the hardware parameter expression can be as follows: Figure 10 As shown, L1, L2, and L3 are nested layer by layer. Typically, each hardware layer is described separately, with parameters for basic arithmetic units, storage, and topology required for each layer. Basic arithmetic unit parameters include the number of units, operation type, bit width, and bit width configuration type. Storage parameters include size, number of banks, number of ports, word size, read / write bandwidth, and read / write latency.

[0082] The topological interconnection is relatively complex and cannot be defined by simple numbers. This embodiment models the topological interconnection of the tensor operation accelerator. In order to parameterize the topology, this embodiment proposes the concepts of array arrangement index and data sharing area. The array arrangement index refers to the fact that in a hardware layer, the basic operation units or PEs are usually arranged in a two-dimensional planar array, and two indexes are specified in the array to indicate the row and column arrangement direction, and all units are divided according to the row and column indexes. The data sharing area refers to a group of several operation units, and all operation units in a group share the same data, such as sharing an input operand or being reduced to the same output.

[0083] For example, the array arrangement index and data sharing area can be shown as follows: Figure 11 As shown in Figure 1, different operands can have different shared areas. Multiple operand shared areas can be combined to obtain various topological structures, ultimately achieving complete modeling of the topology of a layer of hardware structure. This topological structure can be described by the relationship vectors of all operands. All vectors are combined into a relationship matrix, which is the definition of the hardware operand index relationship matrix. For example, the operand index relationship matrix can be as follows Figure 12 By combining the parameterized description of each hardware layer's computing unit, storage, and topology, the complete hardware description parameters can be obtained as follows: Figure 13 Typically, the parameterized description of the classic accelerator topology of Dadianno, Stripes and BitFusion can be as follows Figure 14 shown.

[0084] Furthermore, after completing the software and hardware modeling, the intermediate representation (IR) format can be determined based on the software and hardware description parameters, and the data flow description parameters are composed of the software description parameters, hardware description parameters and IR format. IR can describe the characteristics of cyclic calculations in the spatiotemporal dimensions and map data to hardware components. Each IR format is unified and defined by four fields: mapping type, bit-level calculation, index and range. For example, IR can be expressed as representation = [MappingType, Bitwise, Index, Range & Stride], MappingType∈{Spatial, Temporal}, Bitwise∈{0,1}.

[0085] IR primitives can be used to describe the spatiotemporal characteristics of data transmission and operation processes. Multiple IR primitives can be nested from bottom to top to represent more complex mapping situations. The first two items of the IR primitive determine the IR type. Depending on whether the mapping type is spatial or temporal and whether it is a bit-level description, the IR primitive can be subdivided into four statements, represented by Tmap, Smap, bTmap, and bSmap respectively. Therefore, the primitive can also be expressed as three items: type, index, and size. For example, a complete data flow example can be as follows Figure 15 shown.

[0086] In another optional implementation of this embodiment, establishing a design space based on the data flow description parameters and obtaining a target data flow by searching for an optimal design point in the design space may include: obtaining multiple feasible data flows based on the data flow description parameters, and establishing a design space based on the multiple feasible data flows; evaluating each feasible data flow in the design space, and obtaining a feasible data flow with the best evaluation result as the target data flow.

[0087] In this embodiment, the process of the hardware and scheduling design search method can be as follows: Figure 16 As shown in Figure 1, the parameter values ​​corresponding to the software and hardware description parameters are used as algorithm inputs. The software description parameters include the tensor index relationship matrix of the tensor operation loop body, the range, stride, and input data bit width of each loop level. The hardware description parameters include the number of components, bit width, component arrangement scale, storage size and bandwidth, and the operand index relationship matrix representing the topological interconnection. Among them, the software description parameters are the main inputs, and the hardware description parameters, except for the size and number of multipliers, are optional inputs in the form of constraints.

[0088] The entire process includes two parts: design space construction and space search. The design space construction process starts with the software description, first adding a bit-level description and expanding the operation to the bit level. This process mainly expands the software's relationship matrix and adds two layers of bit-level loops in the loop body. Next, the index for spatial dimension mapping is selected from the loop body to generate the data flow Smap primitive. At the same time, the same data flow relationship matrix is ​​generated based on the software relationship matrix, and the index is not matched first. The mapping process is followed by layered sharding of the loop to determine which indexes are Smap and which are Tmap. The corresponding mappings for the bit-level loop are bTmap and bSmap. If the hardware description parameter value restricts the topology at this time, it is necessary to determine whether the mapped Smap meets the hardware topology restrictions based on the index matching. Data flows that do not meet the constraints are illegal and will be skipped.

[0089] If the hardware parameters do not restrict the topology, or if the array supports arbitrary interconnection, there is no data sharing area, and the relationship values ​​are all 1, then the data flow generated previously is legal by default, and the sub-relationship matrix corresponding to the Smap will be used as a reference for the hardware topology structure. Finally, the nesting order of the IR is transformed to obtain a feasible data flow. After the design space is constructed, it is necessary to search for the optimal design point, which can be summarized as a sampling, evaluation, and arbitration process. The space construction process is precisely the sampling process that generates a feasible data flow based on the input. The sampling results are then evaluated and an arbitration judgment is made. Arbitration includes two aspects. The first is a second legality judgment based on the evaluation results. The bandwidth and storage requirements in the evaluation results must meet the hardware parameter restrictions. The second is to determine the subsequent search strategy, whether to further sample or determine the termination of the search and output the final result.

[0090] The algorithm output is the data flow with the optimal evaluation result, which is represented by nested IR primitives and relational matrices. The IR and sub-relational matrix corresponding to Smap and bSmap determine the hardware topology and scale. The evaluation results of the data flow provide bandwidth and storage requirements. The sub-relational matrix gives a reference topology for the hardware topology, and the actual hardware topology cannot be stricter than this topology. The scope of Smap determines the number and arrangement of hardware components. The actual bandwidth and storage size of the hardware design cannot be lower than the required values ​​of the data flow evaluation. The entire data flow provides a detailed data scheduling process. As an intermediate expression, the data flow is a link in the automated design and tool chain. The actual hardware structure and scheduling can be further refined based on the data flow description.

[0091] Secondly, in the unrestricted case, starting from the software description, space mapping and layered sharding are performed to automatically generate possible topologies and data flows. In the case of hardware constraints, the validity of the hardware description parameter values ​​is checked to determine the feasible data flow and search for the optimal data broadcast order.

[0092] Optionally, evaluating each feasible data flow in the design space and obtaining the feasible data flow with the best evaluation result as the target data flow can include: pruning the design space using redundant pruning and greedy strategies to obtain the pruned design space; and evaluating each feasible data flow in the pruned design space using a backtracking pruning search algorithm, and obtaining the feasible data flow with the best evaluation result as the target data flow.

[0093] In this embodiment, redundant pruning and greedy strategies can be used to prune the design space to improve search efficiency. Furthermore, a backtracking pruning search algorithm is proposed to rapidly optimize the nesting order, make legality judgments and search strategy decisions based on the evaluation results, and output the final results.

[0094] Specifically, by limiting dataflow primitives, the design space is pruned to improve search efficiency. For example, the relative order between spatial mappings and temporal primitives can be ignored, and the order between spatial primitives can also be ignored. It should be noted that spatial and temporal mappings do not affect each other. Dataflow modeling can centralize spatial loops, reorder them, and move them to the innermost layer. The relative order between spatial primitives only affects the hardware arrangement and does not affect the final analysis results. Spatial and temporal primitives do not affect each other, as this is determined by the dataflow definition.

[0095] In addition to redundant space pruning, greedy strategies can also be designed to further reduce the design space. Specifically, first limit the layering to two layers, and adopt a non-layering strategy for the outermost third layer unless necessary. Cache layering is only necessary when data reuse in the outer primitives cannot be utilized. Therefore, there is no need to search for all layered sharding situations. Only when the outer primitives have unusable data reuse should further layering be considered. Otherwise, the default is one layer of sharding, and the data flow is limited to two layers. Secondly, the shard size is also limited. When performing single-layer sharding, the shard size is limited to a power of 2.

[0096] It is understandable that a large part of the design space is contributed by the order. In the data flow evaluation process, the data volume is analyzed layer by layer. Based on this process, this embodiment proposes a backtracking pruning search algorithm that is consistent with the evaluation process to quickly optimize the nesting order. The backtracking pruning search algorithm can be as follows: Figure 17 As shown in the figure, during evaluation and search, IR primitives are selected layer by layer from the bottom up for the current order, using the current optimal total data access amount as a heuristic. If the evaluation of a layer exceeds the heuristic threshold, it can be directly pruned and backtracked, and subsequent outer layer primitives do not need to be searched again.

[0097] The technical solution of this embodiment provides hardware design parameterization results for bit-level combinable computing arrays and automatically designs data scheduling for tensor operations and neural network applications with different bit widths, thereby achieving the goal of adding bit-level computing support to hardware design and data scheduling design, and providing quantitative evaluation and automated guidance for hardware and scheduling design.

[0098] In this embodiment, the following verification experiments are designed to verify the effectiveness of the technical solution of this embodiment. All experiments are run on the Ubuntu 18.04.1 operating system, kernel version 5.4.0-122-generic. Data flow evaluation, operation simulation and design search experiments are implemented in Python language, Python version 3.9.7. The hardware function verification and simulation experiments use the Verilog language, the tool is Vivado 2021.2, and the auxiliary FPGA platform is Zynq UltraScale + ZCU104 Board. Among them, the difference analysis verifies the effectiveness of the design search and analyzes the differences caused by different scheduling strategies. In the reliability verification part, the bit-level evaluation and RTL simulation results are compared to verify the reliability of the evaluation results. In the design search verification part, the search results of the technical solution of this embodiment are compared with the existing manual design solution.

[0099] During the variance analysis and verification, the differences between different data flows were analyzed to verify the effectiveness of the data flow design. A two-dimensional convolution tensor operation was used as the test program, with a convolution scale of (48, 256, 3, 3, 27, 27) and a batch size of 4. The hardware description is single-layer slicing, with a component width of 16 bits and a two-dimensional array size of 16×16. There are no storage or bandwidth restrictions. The array topology is constrained to [i, j] = [[0, 1, 1], [1, 0, 1]], similar to DaDiannao. O is shared in the i direction, with each 16 PEs reducing to one O. I is shared in the j direction, with all 16 PEs sharing the same I. W is not shared and is unicast to each PE.

[0100] Since only OC and IC exceed 16 in size, sharding is required only when these two loops are mapped to spatial IR. The range of other loops does not exceed 16, and direct mapping does not require sharding. The shard size should be sufficient to fill the hardware as much as possible, so the size is limited to 16. After mapping, a legality check is performed based on the topology restrictions. In addition, further restrictions are made on the data flow, and the restrictions are implemented using the Weight-Stationary (WS) method. In the data flow IR, one of the three indexes x, b, and y must be located in the nested inner layer. The heterogeneous analysis verification results can be as follows Figure 18 As shown. Figure (a) shows the latency results and the average utilization results of the PE components, sorted by utilization. The horizontal axis represents different data streams, which are distinguished by numbers. The vertical axis represents latency, and the latency unit is the number of cycles (Mcycles). The smaller the latency, the better the performance. The auxiliary vertical axis represents utilization. The higher the utilization, the better the performance. Utilization is calculated based on the amount of data and the number of multiplication operations. Without considering data reuse, the number of accesses to each tensor is the multiplication of each index range. Each access performs one multiplication, and the theoretical total number of multiplications can be obtained. Component utilization is the PE multiplier utilization, which represents the efficiency of hardware resources. It is obtained by dividing the theoretical total number of multiplications by the number of multiplications at runtime. The hardware may not be full at runtime, and there may be some empty operations. Therefore, the number of operations at runtime exceeds the total number of multiplications. The ratio of the two is the utilization. Utilization and latency are inversely correlated.

[0101] Figure (b) shows the data reuse rate of each tensor in these data flows. The reuse rate refers to the ratio of the amount of reused data to the total amount of data. In a certain batch of operations, if there is data reuse, there is no need to access the memory, and the data is already in the register of the computing unit. This part of the data volume is the reuse volume. The higher the data reuse rate, the higher the memory access efficiency, and the higher the overall efficiency of the data flow. Among various data flows, the optimal data flow reached 1412208 cycles, or 1.4Mcycles, with a PE utilization rate of 89%, and the hardware as a whole ran at a high load. An optimally scheduled data flow can be as follows Figure 19As shown, the relationship matrix is ​​omitted. In this data flow, spatial mapping and slicing are performed using indices oc and ic, representing the number of input and output channels. In the nested order, x and y are located in the inner layers, followed by b, kx, and ky in the outer layers. This prioritizes access to feature map rows and columns, then batch data, and finally weights. This data flow allows weights W to participate in as many calculations as possible before being swapped out, resulting in high temporal reuse of W. Furthermore, O is reused as much as possible.

[0102] The worst-case data flow requires 115Mcycles, with a component utilization rate of only 0.01087. An example is the spatial mapping of indices oc and kx, with the nesting order from the inside out being oc0, kx0, b, ky, x, ic, and oc1. The kx range is small, only 3 in size, which means that 13 / 16 of the multipliers are always idle, resulting in low hardware utilization. Furthermore, the nesting order shows that this data flow has little data reuse and a low reuse rate. First, the b index causes frequent swapping in and out of I and O, with no reuse, and only a small amount of reuse of W. Then the ky index swaps W out again, and W is not reused. The scheduler is always waiting for new data to be swapped in. When calculating the delay, the larger delay is used as the final delay for a round of calculation, resulting in a high overall delay. Despite the poor performance, this data flow is still a legal data flow on the specific hardware in this example, and the two-dimensional convolution can be performed correctly.

[0103] In summary, for a simple 2D convolution and a specific hardware architecture, there are many different dataflow designs, and the differences are significant. It is difficult to estimate the differences between dataflows based on intuition and experience, so a design process that models, quantitatively evaluates, and searches for dataflows is necessary and effective.

[0104] Secondly, in the reliability verification phase, previous related evaluation work did not support bit-level combinability. Therefore, for bit-level computation, this embodiment uses RTL to implement the hardware computation core design for combinable hardware abstraction, and performs functional verification and cycle-level simulation on an FPGA (Field Programmable Gate Array). The hardware design language is Verilog, implementing 2-bit multipliers, shifters, registers, and data exchange paths. 4-bit and 8-bit multiplication and addition computations are implemented based on 16 2-bit multipliers, each followed by a configurable shifter, and the data bit width type can be changed by configuring the shift number. FPGA-assisted design simulation and synthesis verification are utilized. The Vivado pre-simulation tool is used to perform cycle-level simulation of the application scheduling process, including 2b×2b, 2b×4b, 4b×4b, 4b×8b, and 8b×8b matrix multiplications. The simulated scheduling process is generated from the data flow IR, where the IR index corresponds to the loop index, the Smap portion corresponds to the parallel data transmission, and the nesting order indicates the data transmission order. For a 16×16 array, select a general matrix to multiply the three indices to a range of [16×256×16]. The simulation frequency is 200MHz, or 5ns / cycle. The reliability verification results can be shown as follows: Figure 20 shown.

[0105] As can be seen from the figure, the evaluation of the computing core delay by the technical solution of this embodiment is close to the RTL (register-transfer level) cycle-level simulation results, with an accuracy rate of more than 93% and an average accuracy rate of 97%. The main source of error is the pipeline filling and emptying time under pulsation. This embodiment does not analyze pulsation control, so this part of the delay produces a large error. The main errors evaluated in this embodiment come from two aspects, pipeline filling and irregular boundaries. As the scale of calculation increases, the proportion of filling becomes smaller and smaller, and the accuracy will gradually improve.

[0106] Finally, in design search and verification, BitFusion is a typical bit-parallel, combinable deep neural network accelerator, and its combined structure is still widely applicable today. The bottom layer is a two-bit multiplier, and 16 multipliers form a single PE. Each PE can be configured as a multiplier-accumulator of different bit widths. 16×16 PEs form a complete computing core. I rows share systolic propagation, O columns share systolic propagation, and W is unshared. For two-dimensional convolution, the original BitFusion paper adopted two methods: using im2col to convert the convolution into a matrix multiplication, or using a channel-wise data flow. This data flow expands the channel dimension of the output feature map to both directions of the array.

[0107] Use Figure 21The neural network shown in the figure is used for design search, where LeNet, Cifar, VGG7, and SVHN are 2b×2b data. LSTM, AlexNet, and ResNet are 4b×4b data. The evaluation results are statistically analyzed in the simulator. The performance is calculated by dividing the data access volume by the bandwidth to obtain the number of cycles. The number of cycles is divided by the operating frequency of the computing core to obtain the maximum bandwidth limit of a single separated storage of 192bit / cycle. The frequency is set to 500MHz (megahertz), which is consistent with BitFusion. There is no restriction on storage for the time being. It is assumed that all data is cached on the chip and a minimum value of the storage requirement is taken. The storage power consumption is obtained by multiplying the total data access volume by the dynamic power consumption of data reading and writing. The dynamic power consumption is obtained by evaluating the storage using a network traffic monitoring graphic analysis tool. The computing core power consumption is obtained from the array database. The final evaluation results of the design search verification are shown in the figure. Figure 22 As shown, the performance was improved by an average of 1.07 times, the power consumption was improved by an average of 1.1 times, and the overall energy efficiency, or performance-to-power ratio, was improved by 1.17 times. This proves that the scheduling strategy automatically derived by the data flow design search method of this embodiment is as efficient as the most advanced manual scheduling design.

[0108] Example 2

[0109] Figure 23 This is a schematic diagram of a design device for a bit-level combinable accelerator provided in the second embodiment of the present invention. Figure 23 As shown, the apparatus includes: a hardware modeling module 210, a data flow modeling module 220 and a design point search module 230; wherein,

[0110] The hardware modeling module 210 is used to perform bit-level composable hardware modeling on the accelerator to be designed and obtain the bit-level composable hardware structure;

[0111] A data flow modeling module 220 is configured to perform bit-level data flow modeling based on the bit-level combinable hardware structure by fusing software and hardware parameterized descriptions to obtain data flow description parameters;

[0112] The design point search module 230 is used to establish a design space according to the data flow description parameters, and obtain a target data flow by searching for an optimal design point in the design space.

[0113] The technical solution of the embodiment of the present invention is to obtain a bit-level combinable hardware structure by performing bit-level combinable hardware modeling on the accelerator to be designed; based on the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters; a design space is established according to the data flow description parameters, and the target data flow is obtained by searching the design space for the best design point; by performing bit-level combinable hardware modeling, bit-level data flow modeling and hardware scheduling design search, bit-level computing support can be added to the hardware design and data scheduling design of the accelerator, quantitative evaluation and automated guidance can be provided for hardware and scheduling design, and automatic hardware design and scheduling optimization of the bit-level combinable accelerator can be realized.

[0114] Optionally, a hardware modeling module 210 is specifically configured to perform bit-level combinable computing unit modeling based on data bit grouping and vectorization methods to obtain a computing unit structure design, and perform storage structure modeling to obtain a storage structure design;

[0115] According to the computing unit structure design and the storage structure design, a bit-level combinable hardware structure is obtained.

[0116] Optionally, the hardware modeling module 210 is specifically configured to obtain the number of multipliers and the bit group size, and calculate a combined bit width configuration based on the number of multipliers and the bit group size using a bit width configuration algorithm;

[0117] According to the combined bit width configuration, obtaining a shifter type configuration by adopting spatial multiplexing and temporal multiplexing of the shifter;

[0118] By combining internal and external data for parallel data multiplexing, a data multiplexing design is obtained, and based on the shifter type configuration and the data multiplexing design, the computing unit structure design is obtained.

[0119] Optionally, the storage structure design includes a global storage structure design using multi-level cache and a local storage structure design using separate storage.

[0120] Optionally, a data flow modeling module 220 is specifically configured to perform software and hardware modeling according to the bit-level combinable hardware structure, and obtain software description parameters and hardware description parameters;

[0121] The software description parameters include tensor operation loop body characteristic parameters, including loop range, stride and tensor index relationship matrix; the hardware description parameters include multi-level hardware structure parameters, topology description and operand index relationship matrix, and the topology description includes array arrangement index and data sharing area;

[0122] The scheduling process modeling is performed according to the software description parameters and the hardware description parameters, an intermediate expression primitive format is obtained, and data flow description parameters are obtained according to the intermediate expression primitive format.

[0123] Optionally, a design point search module 230 is specifically configured to obtain a plurality of feasible data flows according to the data flow description parameters, and establish a design space based on the plurality of feasible data flows;

[0124] Each feasible data flow in the design space is evaluated, and the feasible data flow with the best evaluation result is obtained as the target data flow.

[0125] Optionally, a design point search module 230 is specifically configured to prune the design space using redundant pruning and greedy strategies to obtain a pruned design space;

[0126] A backtracking pruning search algorithm is used to evaluate each feasible data flow in the pruned design space, and the feasible data flow with the best evaluation result is obtained as the target data flow.

[0127] The device for designing a bit-level combinable accelerator provided in an embodiment of the present invention can execute the method for designing a bit-level combinable accelerator provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0128] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0129] Example 3

[0130] Figure 24 A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. Electronic device 30 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are provided for example only and are not intended to limit the implementation of the present inventions described and / or claimed herein.

[0131] like Figure 24As shown, the electronic device 30 includes at least one processor 31 and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32 and a random access memory (RAM) 33. The memory stores a computer program that can be executed by the at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory 32 or the computer program loaded from the storage unit 38 to the random access memory 33. Various programs and data required for the operation of the electronic device 30 can also be stored in the RAM 33. The processor 31, ROM 32, and RAM 33 are connected to each other via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0132] Multiple components in the electronic device 30 are connected to the I / O interface 35, including an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a magnetic disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0133] Processor 31 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit, a graphics processing unit, various dedicated artificial intelligence computing chips, various processors running machine learning model algorithms, a digital signal processor, and any appropriate processor, controller, microcontroller, etc. Processor 31 executes the various methods and processes described above, such as the design method for a bit-level combinable accelerator.

[0134] In some embodiments, the method for designing a bit-level combinable accelerator may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the method for designing a bit-level combinable accelerator described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to execute the method for designing a bit-level combinable accelerator in any other appropriate manner (e.g., by means of firmware).

[0135] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays, application specific integrated circuits, application specific standard products, system-on-a-chip systems, on-load programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0137] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 30 having: a display device (e.g., a cathode ray tube or a liquid crystal display) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device 30. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0139] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks, wide area networks, blockchain networks, and the Internet.

[0140] A computing system may include clients and servers. The client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server.

[0141] This embodiment may also include a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for designing a bit-level combinable accelerator provided by any embodiment of the present invention.

[0142] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0143] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A design method for a bit-level combinatorial accelerator, characterized in that: include: Perform bit-level composable hardware modeling on the accelerator to be designed to obtain the bit-level composable hardware structure; According to the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters; Establishing a design space according to the data flow description parameters, and obtaining a target data flow by searching for an optimal design point in the design space; The bit-level composable hardware modeling of the accelerator to be designed is performed to obtain the bit-level composable hardware structure, including: Based on data bit grouping and vectorization methods, bit-level combinable computing unit modeling is performed to obtain the computing unit structure design, and storage structure modeling is performed to obtain the storage structure design; Acquire a bit-level combinable hardware structure according to the computing unit structure design and the storage structure design; Among them, based on the data bit grouping and vectorization method, bit-level combinable computing unit modeling is carried out to obtain the computing unit structure design, including: Obtaining the number of multipliers and the bit group size, and calculating a combined bit width configuration based on the number of multipliers and the bit group size using a bit width configuration algorithm; wherein the combined bit width configuration includes the number of shifts and the number of required shifters; According to the combined bit width configuration, a shifter type configuration is obtained by using spatial multiplexing and temporal multiplexing of the shifter; wherein the shifter type is obtained according to the shift number list by using a shifter type configuration algorithm; Acquire a data multiplexing design by combining internal and external data for data multiplexing in parallel, and acquire a computing unit structure design based on the shifter type configuration and the data multiplexing design; The method of establishing a design space according to the data flow description parameters and obtaining a target data flow by searching for an optimal design point in the design space comprises: Acquire multiple feasible data flows according to the data flow description parameters, and establish a design space based on the multiple feasible data flows; Evaluate each feasible data flow in the design space, and obtain the feasible data flow with the best evaluation result as the target data flow; The process of evaluating each feasible data flow in the design space and obtaining the feasible data flow with the best evaluation result as the target data flow includes: Pruning the design space using redundant pruning and greedy strategies to obtain a pruned design space; A backtracking pruning search algorithm is used to evaluate each feasible data flow in the pruned design space, and the feasible data flow with the best evaluation result is obtained as the target data flow; wherein, the backtracking pruning search algorithm specifically refers to selecting IR primitives for the current order layer by layer from the bottom up during evaluation and search, using the current optimal total data access volume as a heuristic. If the evaluation of the layer exceeds the heuristic threshold, it is directly pruned and backtracked, and subsequent outer layer primitives are no longer searched.

2. The method according to claim 1, characterized in that The storage structure design includes a global storage structure design using multi-level cache and a local storage structure design using separate storage.

3. The method according to claim 1, characterized in that According to the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters, including: Performing software and hardware modeling according to the bit-level combinable hardware structure to obtain software description parameters and hardware description parameters; The software description parameters include tensor operation loop body characteristic parameters, including loop range, stride and tensor index relationship matrix; the hardware description parameters include multi-level hardware structure parameters, topology description and operand index relationship matrix, and the topology description includes array arrangement index and data sharing area; The scheduling process modeling is performed according to the software description parameters and the hardware description parameters, an intermediate expression primitive format is obtained, and data flow description parameters are obtained according to the intermediate expression primitive format.

4. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method for designing a bit-level combinable accelerator according to any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the method for designing a bit-level combinable accelerator according to any one of claims 1 to 3 when the computer program is executed.

6. A computer program product, characterized in that The invention comprises a computer program, which implements the method for designing a bit-level combinable accelerator according to any one of claims 1 to 3 when executed by a processor.

Images