Design method and device of bit-level combinable accelerator, medium and product

Abstract

The invention relates to the technical field of computers, and discloses a design method and device of a bit-level combinable accelerator, a medium and a product. The method comprises the following steps: carrying out bit-level combinable hardware modeling on an accelerator to be designed to obtain a bit-level combinable hardware structure; according to the bit-level combinable hardware structure, performing bit-level data stream modeling by fusing software and hardware parameterization description to obtain data stream description parameters; and establishing a design space according to the data stream description parameters, and performing optimal design point search on the design space to obtain a target data stream. According to the scheme of the embodiment, through bit-level combinable hardware modeling, bit-level data flow modeling and hardware scheduling design searching, bit-level calculation support can be added for hardware design and data scheduling design of the accelerator, quantitative evaluation and automatic guidance can be provided for the hardware and scheduling design, and the hardware design and the data scheduling design are optimized. Hardware design and scheduling optimization of an automatic bit-level combinable accelerator can be realized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a design method, device, medium and product of a bit-level combinable accelerator. Background Art

[0002] Neural networks are widely used in image classification, recommendation systems, object detection, and natural language understanding. Behind the remarkable achievements of neural networks are complex and deep network models, massive parameters, and a large amount of computation. The number of parameters in large models can reach hundreds of millions, and the increasing amount of computation has prolonged model training and inference time, increased resource consumption, and increased costs.

[0003] In order to improve the running speed of neural networks and save costs at the same time, model quantization has become a relatively effective solution. Model quantization refers to the use of low-bit-width fixed-point numbers or integers to replace high-bit-width floating-point numbers as network parameters and input data. On the one hand, the data bit width becomes smaller, the amount of data and the amount of calculation are reduced accordingly, the performance and power consumption are significantly improved, and the low bit width is conducive to reducing the complexity of the hardware. On the other hand, the decrease in data precision will lead to a decrease in the accuracy of the model. When quantizing, it is necessary to consider the trade-off between the decrease in accuracy and the benefits. Thanks to the tolerance of neural networks to data accuracy, the data bit width can be greatly reduced if the accuracy loss does not exceed a certain threshold. After model quantization, the ideal minimum bit width of different network models is different, and even the bit width required by different parts and layers of the same network model may be different, which leads to tasks with uncertain bit width or multiple bit width requirements.

[0004] In order to efficiently process neural networks of different bit widths, hardware support acceleration and software scheduling optimization have become key research topics, namely, the design of efficient hardware accelerators that support multiple data bit widths and the scheduling optimization of neural networks on this hardware. Existing fixed high-bitwidth machines obviously cannot take advantage of low-bitwidth and variable-bitwidth calculations. In most devices, low-precision data still needs to be expanded to the machine bit width before calculation. In graphics processors, there are multiple computing cores with different precision sizes that can make full use of low-bitwidth data, but designing a separate type of computing core for each precision will result in a waste of resources, extremely low hardware utilization, and high cost. Driven by the above problems, there is a need for dedicated hardware devices that support multiple data bit widths at the same time, and some hardware accelerator design work has been done.

[0005] In order to achieve a single structure supporting calculations of various bit widths, hardware usually adopts a method similar to "large number multiplication", that is, using a combination of "small number" operations to implement "large number" operations. High-bitwidth data is split into multiple groups of low-bitwidth data or even single-bit data, and then multi-stage calculations are performed to form multi-bit calculations. Hardware that supports this type of calculation is called a bit-level combinable architecture. The combinable architecture can adapt well to scenarios with different bit width requirements. When high precision is required, low-precision calculations can be combined into high-precision calculations. When low precision is required, low-precision components can be used directly for calculations, and hardware resources will not be wasted. Compared with providing separate computing components for data of different bit widths, the bit-level combinable architecture is characterized by the ability to perform multiplication and addition calculations of different numbers and sizes on the same device. The hardware can be dynamically configured as multiplication and addition calculation units of different bit widths during calculations, which meets different bit width requirements while improving component utilization and throughput.

[0006] Although the composable architecture has additional overhead in the microstructure, this does not prevent data bit width from being a consideration dimension in hardware design. Some accelerator designs have been able to gain benefits from the combined structure. In addition to hardware support, the data scheduling process of neural network applications on hardware affects the final operating efficiency. Scheduling, also known as data flow, describes when data is transmitted to each level of storage and computing units, and describes the size and flow of data. In actual deployment, scheduling is reflected in the reordering of instructions or data. Obviously, the same application has many different scheduling methods, and the data scheduling process of the application needs to be optimized during design. Furthermore, when optimizing scheduling, the characteristics of the underlying hardware should be considered, and the underlying hardware resources should be used reasonably. At present, there has been work in the field to evaluate and analyze the execution effect of neural networks on hardware, as well as methods and frameworks for optimizing scheduling, but there are important problems in past work:

[0007] First, some hardware and scheduling designs lack support for bit-level computing, that is, they lack support for computing with multiple bit widths. In order to accelerate neural network inference calculations as much as possible and reduce costs, bit-level computing has become an important design consideration. Low bit-width computing will bring direct performance improvements and power consumption reductions, and low bit widths are conducive to reducing the complexity of hardware. Neural network quantization methods have brought practical needs for bit-level computing, and neural networks of different bit widths have become more common. In addition to neural networks, tensor computing in a broader sense also has the need for computing with multiple bit widths.

[0008] Secondly, traditional neural network accelerators that support multiple precisions are based on empirical qualitative design. The hardware structure has many variants and trade-offs. The component type, quantity, topological interconnection between components, and storage structure are difficult to determine, and feasible designs are usually tried based on experience. After considering the needs of bit-level calculations and multiple bit widths, the design process becomes more complicated. In the past, accelerator core designs were manually designed according to application characteristics, which was time-consuming, labor-intensive, and inefficient. The rapid development of neural network applications has also put forward requirements for the update and iteration speed of the underlying hardware. Due to the long time required for hardware verification, synthesis, simulation, and simulation, some design trade-offs are difficult to be quickly evaluated.

[0009] Finally, even if the hardware structure is determined, the scheduling process needs to be designed and optimized for different applications. In the past, the scheduling of accelerator design work was manually designed and lacked scheduling optimization. New schedules need to be manually designed for new applications. The low efficiency of hardware design and scheduling optimization seriously hinders the update and iteration of software and hardware. Although data flow modeling evaluation and design automation work have alleviated this problem, these works lack bit-level computing support and cannot be applied to the scenario of bit-level composable architecture. Summary of the invention

[0010] The present invention provides a design method, device, medium and product for a bit-level combinable accelerator, which can add bit-level computing support to the hardware design and data scheduling design of the accelerator, provide quantitative evaluation and automated guidance for hardware and scheduling design, and realize automatic hardware design and scheduling optimization of the bit-level combinable accelerator.

[0011] According to one aspect of the present invention, a method for designing a bit-level combinable accelerator is provided, comprising:

[0012] Perform bit-level composable hardware modeling on the accelerator to be designed to obtain the bit-level composable hardware structure;

[0013] According to the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters;

[0014] A design space is established according to the data flow description parameters, and a target data flow is obtained by searching for an optimal design point in the design space.

[0015] According to another aspect of the present invention, there is provided a device for designing a bit-level combinable accelerator, comprising:

[0016] A hardware modeling module is used to perform bit-level composable hardware modeling on the accelerator to be designed and obtain a bit-level composable hardware structure;

[0017] A data flow modeling module is used to perform bit-level data flow modeling according to the bit-level combinable hardware structure by fusing software and hardware parameterized descriptions to obtain data flow description parameters;

[0018] The design point search module is used to establish a design space according to the data flow description parameters, and obtain the target data flow by searching the design space for the best design point.

[0019] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for designing a bit-level combinable accelerator described in any embodiment of the present invention.

[0023] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program is used to enable a processor to implement the method for designing a bit-level combinable accelerator according to any embodiment of the present invention when executed.

[0024] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for designing a bit-level combinable accelerator according to any embodiment of the present invention is implemented.

[0025] The technical solution of the embodiment of the present invention performs bit-level combinable hardware modeling on the accelerator to be designed to obtain the bit-level combinable hardware structure; based on the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters; a design space is established according to the data flow description parameters, and the target data flow is obtained by searching the design space for the best design point; by performing bit-level combinable hardware modeling, bit-level data flow modeling and hardware scheduling design search, bit-level computing support can be added to the hardware design and data scheduling design of the accelerator, quantitative evaluation and automated guidance can be provided for hardware and scheduling design, and automatic hardware design and scheduling optimization of the bit-level combinable accelerator can be realized.

[0026] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 is a flow chart of a method for designing a bit-level combinable accelerator provided in accordance with a first embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of a bit-level combinable hardware structure provided according to Embodiment 1 of the present invention;

[0030] Figure 3 is a calculation flow chart of a bit grouping and vectorization method provided according to Embodiment 1 of the present invention;

[0031] Figure 4 is a calculation flow chart of a vector dot product or multiplication-addition operation provided according to Embodiment 1 of the present invention;

[0032] Figure 5 is a schematic diagram of a storage structure design provided according to Embodiment 1 of the present invention;

[0033] Figure 6 is a schematic diagram of a bit width configuration algorithm provided according to Embodiment 1 of the present invention;

[0034] Figure 7 is a schematic diagram of a shifter type configuration algorithm provided according to Embodiment 1 of the present invention;

[0035] Figure 8 is a schematic diagram of a process of shifter multiplexing provided according to Embodiment 1 of the present invention;

[0036] Fig. 9 is a schematic diagram of a data multiplexing design solution provided according to Embodiment 1 of the present invention;

[0037] Fig.10 is a schematic diagram of a hardware parameter expression provided according to Embodiment 1 of the present invention;

[0038] Fig.11 is a schematic diagram of an array arrangement index and a data sharing area provided in Embodiment 1 of the present invention;

[0039] Fig.12 is a schematic diagram of an operand index relationship matrix provided according to Embodiment 1 of the present invention;

[0040] Fig.13 is a schematic diagram of hardware description parameters provided according to Embodiment 1 of the present invention;

[0041] Fig.14 is a schematic diagram of parameterized description of a classic accelerator topology provided according to Embodiment 1 of the present invention;

[0042] Fig.15 This is a diagram showing an example of a data flow provided according to Embodiment 1 of the present invention;

[0043] Fig.16 is a flowchart of a hardware and scheduling design search method provided according to Embodiment 1 of the present invention;

[0044] Fig.17 is a schematic diagram of a backtracking pruning search algorithm provided according to Embodiment 1 of the present invention;

[0045] Fig.18 is a schematic diagram of heterosexual analysis verification results provided according to the first embodiment of the present invention;

[0046] Fig.19 This is a schematic diagram of data flow for optimal scheduling provided in Embodiment 1 of the present invention;

[0047] Fig. 20 is a schematic diagram of reliability verification results provided according to the first embodiment of the present invention;

[0048] Fig.21 is a schematic diagram of neural network structure parameters provided according to the first embodiment of the present invention;

[0049] Fig. 22 is a schematic diagram of evaluation results of design search verification provided according to Embodiment 1 of the present invention;

[0050] Fig.23 is a schematic diagram of the structure of a device for designing a bit-level combinable accelerator according to a second embodiment of the present invention;

[0051] Fig.24 It is a schematic diagram of the structure of an electronic device for implementing the method for designing a bit-level combinable accelerator according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] It should be noted that the terms "first", "second", "target", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0054] Embodiment 1

[0055] Figure 1 A flowchart of a method for designing a bit-level combinable accelerator is provided for the first embodiment of the present invention. This embodiment is applicable to the case of hardware design and scheduling optimization of a bit-level combinable accelerator. The method can be executed by a design device for a bit-level combinable accelerator. The design device for a bit-level combinable accelerator can be implemented in the form of hardware and / or software. Typically, the design device for a bit-level combinable accelerator can be configured in an electronic device, such as a computer device or a server. Figure 1 As shown, the method includes:

[0056] S110 , performing bit-level composable hardware modeling on the accelerator to be designed to obtain a bit-level composable hardware structure.

[0057] In this embodiment, by performing bit-level combinable hardware modeling, bit-level combinable hardware structure abstraction can be achieved, and multiplication and addition operations of data with different bit widths can be supported while maintaining hardware resource utilization. The hardware structure abstraction can include computing component design, shifter optimization using component reuse, and bandwidth and complexity optimization using data reuse.

[0058] Specifically, the bit-level composable computing unit modeling and storage structure modeling can be performed on the accelerator to be designed in sequence to obtain a comprehensive composable design, component structure design, bandwidth design, and storage structure design. Then, based on the above design, multi-layer bit-level spatial accelerator modeling can be performed to give a bit-level composable hardware structure abstraction, thereby obtaining a bit-level composable hardware structure. For example, the bit-level composable hardware structure can be as follows Figure 2As shown, a basic bit-level composable accelerator can be composed of a computing array, on-chip interconnection, and on-chip storage. The computing array is a PE (Processing Element) array, and PE is a bit-level composable structure. The on-chip interconnection can be a memory access control module or a combination of data feeding methods. For the relevant parameters of the basic structure, the storage includes size and maximum bandwidth, the interconnection includes topology and broadband delay, and the computing array includes the number of PEs, register size, vector length and width, etc.

[0059] S120 , according to the bit-level combinable hardware structure, perform bit-level data flow modeling by fusing software and hardware parameterized descriptions to obtain data flow description parameters.

[0060] Specifically, based on the bit-level combinable hardware structure, the tensor computing application and the bit-level combinable hardware can be parameterized to obtain software description parameters and hardware description parameters; then, by fusing the software description parameters, hardware description parameters and bit-level calculations, the execution process of the application on the hardware can be parameterized to obtain data flow description parameters, for example, the process and parameter items required to describe the data flow.

[0061] S130: Establish a design space according to the data flow description parameters, and obtain a target data flow by searching for an optimal design point in the design space.

[0062] In this embodiment, the hardware and scheduling design search can be divided into two parts: design space construction and space search. Specifically, all feasible data flows can be automatically generated based on the data flow description parameters to construct the design space. For example, different feasible data flows can be generated by assigning different parameter values ​​to each parameter in the data flow description parameters. Among them, when constructing the design space, the software description parameters can be used as the main input, and the hardware description parameters other than the number of multipliers can be used as optional inputs in the form of constraints. Then, these feasible data flows can be evaluated, for example, based on indicators such as hardware utilization efficiency and delay, to search for the best design point in the design space, so as to find the feasible data flow with the best evaluation result as the final target data flow.

[0063] It can be understood that based on the determined target data flow, the hardware topology and scale of the accelerator and the data scheduling process can be determined, thereby realizing the automatic design of the accelerator.

[0064] The technical solution of the embodiment of the present invention performs bit-level combinable hardware modeling on the accelerator to be designed to obtain the bit-level combinable hardware structure; based on the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters; a design space is established according to the data flow description parameters, and the target data flow is obtained by searching the design space for the best design point; by performing bit-level combinable hardware modeling, bit-level data flow modeling and hardware scheduling design search, bit-level computing support can be added to the hardware design and data scheduling design of the accelerator, quantitative evaluation and automated guidance can be provided for hardware and scheduling design, and automatic hardware design and scheduling optimization of the bit-level combinable accelerator can be realized.

[0065] In an optional implementation of the present embodiment, bit-level combinable hardware modeling is performed on the accelerator to be designed to obtain the bit-level combinable hardware structure, which may include: based on data bit grouping and vectorization methods, bit-level combinable computing unit modeling is performed to obtain the computing unit structure design, and storage structure modeling is performed to obtain the storage structure design; according to the computing unit structure design and the storage structure design, the bit-level combinable hardware structure is obtained.

[0066] In this embodiment, for the data bit grouping and vectorization method, the bit grouping and vectorization process can be given by mathematical symbols to provide a theoretical basis for bit-level calculation and modeling of combination structures. For example, the process of the bit grouping and vectorization method can be as follows Figure 3 As shown in the figure, G represents the bit grouping size, that is, the bit width of the data after grouping. After bit grouping, it can continue to participate in external vector multiplication and addition calculations. At this time, the data has an extra dimension. The new dimension brings a new calculation method and introduces a new shift operation when performing multiplication. After data vectorization, the calculation process of the classic vector dot product or multiplication and addition operation can be as follows Figure 4 As shown. By transforming the inner and outer layers of the summation sign, another calculation method can be obtained, namely

[0067] Secondly, based on the data bit grouping and vectorization methods, bit-level composable computing unit modeling can be performed. Typically, a bottom-up approach can be used to build a composable computing unit design paradigm, so that the computing unit PE supports a variety of different bit widths while maintaining hardware resource utilization.

[0068] It should be noted that the design goal of the bit-level combinable computing unit is to allow the computing unit PE to support a variety of different bit widths, and the constraint is to keep the hardware fully loaded as much as possible. In order for PE to support low-bit-width data operations, the bottom layer must be a low-bit-width computing unit, and operations with a bit width of G can only be combined into operations with a bit width that is an integer multiple of G. By configuring the shifter, multiplication and addition operations of different vector lengths and different bit widths can be achieved. In this embodiment, the number of multipliers and the bit grouping size can be determined according to the initial structure of the accelerator to be designed, and the required shifter configuration can be automatically calculated based on the number of multipliers and the bit grouping size, thereby realizing the modeling of the bit-level combinable computing unit and obtaining the computing unit structure design.

[0069] In this embodiment, while modeling the bit-level combinable computing unit, the storage structure modeling can also be performed based on the preset storage body structure, and the storage structure is described using parameters to obtain the storage structure design. Optionally, the storage structure design can include a global storage structure design using multi-level cache and a local storage structure design using separate storage. This embodiment proposes the idea of ​​multi-level cache design and separate storage, which can utilize data access locality and data reuse to reduce repeated and inefficient external memory accesses. At the same time, through the double buffer design, memory access and calculation can be pipelined to increase hardware execution efficiency. The accelerator in this embodiment can adopt a storage structure design of multi-level cache, separate storage, and a double buffer pipeline design, which includes registers, local storage around the computing unit, on-chip global storage, and external storage from bottom to top.

[0070] In a specific example, the storage structure design can be as follows Figure 5 As shown in the figure, the left side of the figure shows multi-level cache and separate storage. There is a global storage body on the hardware chip, which is used to interact with external memory and send data to local storage. Local storage uses separate storage, and each type of data corresponds to a separate storage body. The lower layer of local storage is the computing unit, which stores data to the computing unit register, performs calculations, and then writes it back. The memory access hotspots of the multi-level cache, that is, the locations where data is read and written intensively, are concentrated on the more efficient storage at the bottom layer. The memory access becomes less efficient as it goes to the outer layer. The arrows in the figure represent the memory access bus or a specific data transmission path.

[0071] Secondly, the right side of the figure shows a double buffer "ping-pong" mechanism. The storage is doubled, and two storages form the same function. When the memory access task is in progress, the two storages perform reading and writing respectively, so that there is no structural conflict in the data interaction between the inner and outer layers, and the inner and outer layers can be pipelined for memory access. The inner layer can be a computing unit, so the memory access and computing can be pipelined.

[0072] Optionally, based on data bit grouping and vectorization methods, bit-level combinatorial computing unit modeling is performed to obtain a computing unit structural design, which may include: obtaining the number of multipliers and the bit grouping size, and calculating a combined bit width configuration based on the number of multipliers and the bit grouping size through a bit width configuration algorithm; obtaining a shifter type configuration based on the combined bit width configuration by adopting spatial multiplexing and temporal multiplexing of shifters; obtaining a data multiplexing design by combining internal and external data parallelism for data multiplexing, and obtaining the computing unit structural design based on the shifter type configuration and the data multiplexing design.

[0073] In this embodiment, the bit width configuration algorithm can be as follows: Figure 6 As shown, the input is the number of multipliers and the bit grouping size, and the output is the combined bit width configuration, including the number of shifts, the number of required shifters, etc. It can be understood that the number and type of shifters determine the achievable bit width configuration. In order to reduce hardware complexity, the shifter usually cannot support an arbitrary number of shifts, but supports several specific types of shifts, such as 2-bit and 4-bit left shifts. In order to obtain a more efficient computing component, the present embodiment proposes a shifter multiplexing structure.

[0074] In order to reduce the number of shifters as much as possible, use fixed-width shifts as much as possible, and try to reuse shifters in operations to reduce the complexity of calculation components. The shifter type configuration algorithm can be as follows Figure 7 As shown in the figure, the input of the algorithm is the shift list shiftlist, and the output is the shifter type list. The shift list gives the number of left shifts performed by all multiplications in order to achieve a specific bit width size calculation. If the list is regarded as a matrix, the number of rows in the matrix is ​​equal to the number of bit width configurations, and the number of columns is equal to the number of multipliers.

[0075] First, the algorithm sorts the data in each row of the matrix, and then permutes the rows to obtain an ordered matrix. Each row and column of the ordered matrix is ​​an ordered sequence, and each column of the matrix represents the total number of shifts that a shifter needs to support. Next, the matrix is ​​grouped and packed by column, and the packing process merges consecutive adjacent equal columns into one column. Equal columns mean that the configurations of these shifters are exactly the same, and one shifter is sufficient. Multiple multiplications can share one shifter, and after multiplication, addition can be performed first and then shifting. The order of operations of multiplication, shifting, and then addition becomes multiplication and addition first and then shifting. This optimization realizes component reuse by sharing one shifter with several components, which is also called shifter spatial reuse. Spatial reuse reduces the number of shifters and reduces hardware complexity. Then, the packed matrix is ​​subtracted in reverse column order. The subtraction process is used to extract the same part between two different types of shifters and reuse this part, which is also called shifter time reuse. The purpose of time multiplexing is to reduce the complexity of the shifter and try to convert multiple shifts into a fixed number of shifts. For example, the process of shifter multiplexing can be as follows: Figure 8 shown.

[0076] In addition to the difference in vector length and bit width, different bit width configurations also have another important difference in bandwidth requirements. In order to alleviate the problem of bandwidth imbalance of different bit width configurations, this embodiment proposes a data multiplexing hardware design method. The core idea is to combine internal and external data parallelism, incorporate data multiplexing into hardware design, increase the number of operations performed on a single data access, and thus alleviate bandwidth pressure. For example, the data multiplexing design solution can be as follows Fig. 9 shown.

[0077] In another optional implementation of the present embodiment, according to the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters, which may include: performing software and hardware modeling according to the bit-level combinable hardware structure to obtain software description parameters and hardware description parameters; wherein the software description parameters include tensor operation loop body characteristic parameters, the tensor operation loop body characteristic parameters include loop range, stride and tensor index relationship matrix, the hardware description parameters include multi-level hardware structure parameters, topology description and operand index relationship matrix, the topology description includes array arrangement index and data sharing area; scheduling process modeling is performed according to the software description parameters and the hardware description parameters to obtain an intermediate expression primitive format, and data flow description parameters are obtained according to the intermediate expression primitive format.

[0078] In this embodiment, bit-level data flow modeling that integrates software and hardware features and bit-level calculations is proposed to describe the execution process of the application on the hardware and realize quantitative evaluation of the operation process of different applications and hardware structures. Typically, bit-level data flow modeling can include stages such as mapping intermediate expression primitives, complete data flow description, and index matching. Among them, bit-level data flow modeling is based on software description parameters and hardware description parameters, and is oriented to tensor computing applications and multi-layer array-type bit-level combinable hardware structures, reflecting when and where tensor data participates in the calculation.

[0079] Specifically, bit-level data flow modeling can include two stages: software and hardware modeling and scheduling process modeling. Among them, software and hardware modeling mainly parameterizes tensor computing applications and bit-level composable hardware, including loop body operation characteristics of neural network operators and general tensor computing applications in software description, and parameterization of multi-level hardware structures, topological descriptions, and operand index relationship matrices in hardware description.

[0080] When describing the tensor operation loop body parameterized, the loop body is determined by two parameters: loop range and stride, and the calculation statement is determined by the tensor index relationship matrix. The tensor index matrix is ​​a relationship matrix that describes the correspondence between tensors and indices, and the indexes correspond to calculation statements one by one. When the index has an access relationship to the tensor, that is, when the tensor subscript contains the index, the corresponding position of the matrix is ​​1, which means that the calculation statement corresponding to the index needs to be used. If the tensor subscript does not contain the index, the corresponding position of the matrix is ​​0.

[0081] Secondly, when describing hardware abstraction parameterized, the main part of the hardware description can use nested definitions. For example, the hardware parameter expression can be as follows Fig.10 As shown, L1, L2 to L3 are nested layer by layer. Typically, each level of the hardware is described separately, and each layer needs to give a parameter description of the basic operation unit, storage, and topology. The parameter description of the basic operation unit includes the number of units, operation type, bit width size, and bit width configuration type, etc. The parameter description of the storage includes size, number of banks, number of ports, word size, read and write bandwidth, read and write latency, etc.

[0082] The topological interconnection is relatively complex and cannot be defined with simple numbers. This embodiment models the topological interconnection of the tensor operation accelerator. In order to parameterize the topology, this embodiment proposes the concepts of array arrangement index and data sharing area. The array arrangement index refers to a level of hardware in which the basic operation units or PEs are usually arranged in a two-dimensional plane array. Two indexes are specified in the array to indicate the row and column arrangement direction, and all units are divided according to the row and column indexes. The data sharing area refers to a group of several operation units, and all operation units in a group share the same data, such as sharing an input operand or being reduced to the same output.

[0083] For example, the array arrangement index and data sharing area can be shown as follows: Fig.11 As shown in the figure, different operands can have different shared areas. Multiple operand shared areas can be combined to obtain various topological structures, and finally a complete modeling of the topology of a layer of hardware structure can be achieved. This topological structure can be described by the relationship vectors of all operands. All vectors are combined into a relationship matrix, which is the definition of the hardware operand index relationship matrix. For example, the operand index relationship matrix can be as follows Fig.12 By combining the parameterized description of each hardware layer’s computing unit, storage, and topology, the complete hardware description parameters can be obtained as follows: Fig.13 Typically, the parameterized description of the classic accelerator topology of Dadianno, Stripes and BitFusion can be as follows Fig.14 shown.

[0084] Furthermore, after completing the software and hardware modeling, the intermediate representation (IR) format can be determined according to the software and hardware description parameters, and the data flow description parameters are composed of the software description parameters, hardware description parameters and IR format. IR can describe the characteristics of cyclic calculations in the time and space dimensions and map data to hardware components. Each IR format is unified and defined by four fields: mapping type, bit-level calculation, index and range. For example, IR can be expressed as representation = [MappingType, Bitwise, Index, Range & Stride], MappingType∈{Spatial, Temporal}, Bitwise∈{0,1}.

[0085] IR primitives can be used to describe the spatiotemporal characteristics of data transmission and operation processes. Multiple IR primitives can be nested from bottom to top to represent more complex mapping situations. The first two items of the IR primitive determine the IR type. Depending on whether the mapping type is spatial or temporal and whether it is a bit-level description, the IR primitive can be subdivided into four statements, represented by Tmap, Smap, bTmap, and bSmap, respectively. Therefore, the primitive can also be expressed as three items: type, index, and size. For example, a complete data flow example can be as follows Fig.15 shown.

[0086] In another optional implementation of the present embodiment, a design space is established according to the data flow description parameters, and a target data flow is obtained by searching the design space for an optimal design point, which may include: obtaining multiple feasible data flows according to the data flow description parameters, and establishing a design space based on the multiple feasible data flows; evaluating each feasible data flow in the design space, and obtaining a feasible data flow with the best evaluation result as the target data flow.

[0087] In this embodiment, the process of the hardware and scheduling design search method can be as follows: Fig.16 As shown, the parameter values ​​corresponding to the software and hardware description parameters are used as algorithm inputs. The software description parameters include the tensor index relationship matrix of the tensor operation loop body, the range, stride and input data bit width of each level of loop, and the hardware description parameters include the number of components, bit width, component arrangement scale, storage size and bandwidth, and the operand index relationship matrix representing the topological interconnection. Among them, the software description parameters are the main inputs, and the hardware description parameters, except for the size and number of multipliers, are optional inputs in the form of constraints.

[0088] The whole process includes two parts: design space construction and space search. The design space construction process starts from the software description, first adding a bit-level description and extending the operation to the bit level. This process mainly expands the software's relationship matrix and adds two layers of bit-level loops in the loop body. Next, select the index for spatial dimension mapping from the loop body to generate the data flow Smap primitive. At the same time, the same data flow relationship matrix is ​​generated based on the software relationship matrix, and the index is not matched first. The mapping process is followed by the layered slicing of the loop to determine which indexes are Smap and which are Tmap. The corresponding mappings of the bit-level loop are bTmap and bSmap. If the hardware description parameter value restricts the topology at this time, it is necessary to determine whether the mapped Smap meets the hardware topology restrictions based on the index matching. Data flows that do not meet the constraints are illegal and will be skipped.

[0089] If the hardware parameters do not restrict the topology, or the array supports arbitrary interconnection, there is no data sharing area, and the relationship values ​​are all 1, then the previously generated data flow is legal by default, and the sub-relationship matrix corresponding to the Smap will be used as a reference for the hardware topology. Finally, the nesting order of the IR is transformed to obtain a feasible data flow. After the design space is constructed, it is necessary to search for the best design point, which can be summarized as a sampling, evaluation, and arbitration process. The space construction process is the sampling process that generates a feasible data flow based on the input, and then the sampling results are evaluated and arbitration is performed. Arbitration includes two aspects. The first is to make a second legality judgment based on the evaluation results. The bandwidth and storage requirements in the evaluation results need to meet the hardware parameter restrictions. The second is to determine the subsequent search strategy, whether to further sample or determine the termination of the search, and output the final result.

[0090] The output of the algorithm is the data flow with the best evaluation result, which is represented by nested IR primitives and relationship matrices. The IR and sub-relationship matrix corresponding to Smap and bSmap determine the hardware topology and scale. The evaluation result of the data flow provides bandwidth and storage requirements. The sub-relationship matrix gives a reference topology for the hardware topology mode, and the actual hardware topology cannot be stricter than this topology. The scope of Smap determines the number and arrangement of hardware components. The actual bandwidth and storage size of the hardware design cannot be lower than the required value of the data flow evaluation. The entire data flow gives a detailed data scheduling process. As an intermediate expression, the data flow is a link in the automated design and tool chain. The actual hardware structure and scheduling can be further refined according to the data flow description.

[0091] Secondly, in the case of no restrictions, starting from the software description, space mapping and hierarchical slicing are performed to automatically generate possible topological structures and data flows. In the case of hardware constraints, the legality of the hardware description parameter values ​​is checked to determine the feasible data flow and search for the optimal data broadcasting order.

[0092] Optionally, evaluating each feasible data flow in the design space and obtaining the feasible data flow with the best evaluation result as the target data flow may include: pruning the design space using redundant pruning and greedy strategies to obtain the pruned design space; using a backtracking pruning search algorithm to evaluate each feasible data flow in the pruned design space, and obtaining the feasible data flow with the best evaluation result as the target data flow.

[0093] In this embodiment, redundant pruning and greedy strategies can be used to prune the design space to improve search efficiency. In addition, a backtracking pruning search algorithm is proposed to quickly optimize the nesting order, make legality judgments and search strategy decisions based on the evaluation results, and output the final results.

[0094] Specifically, by limiting the data flow primitives, the design space is pruned to improve the search efficiency. For example, the relative order between spatial mapping and temporal primitives can be ignored, and the order between spatial primitives can be ignored. It should be noted that spatial and temporal mappings do not affect each other. Data flow modeling can concentrate spatial loops, change the order, and transfer them to the innermost layer. The relative order between spatial primitives only affects the hardware arrangement and does not affect the final analysis results. There is no impact between spatial and temporal primitives, which is determined by the data flow definition.

[0095] In addition to redundant space pruning, some greedy strategies can also be designed to further reduce the design space. Specifically, first limit the stratification to two layers, and adopt a non-essential non-stratification strategy for the outermost third layer. Cache stratification is only required when there is data reuse in the outer primitives that cannot be utilized. Therefore, there is no need to search for all layered sharding situations. Only when the outer primitives have data reuse that cannot be utilized should further stratification be considered. Otherwise, one layer of sharding is used by default, and the data flow is limited to two layers. Secondly, the shard size is also limited. When single-layer sharding is performed, the shard size is limited to a power of 2.

[0096] It is understandable that a large part of the design space is contributed by the order. In the data flow evaluation process, the data volume is analyzed layer by layer. Based on this process, this embodiment proposes a backtracking pruning search algorithm that is consistent with the evaluation process to quickly optimize the nesting order. The backtracking pruning search algorithm can be as follows Fig.17 When evaluating and searching, IR primitives are selected layer by layer from the bottom up for the current order, with the current optimal total data access amount as the heuristic. If the evaluation of the layer has exceeded the heuristic threshold, it can be directly pruned and backtracked, and subsequent outer primitives do not need to be searched again.

[0097] The technical solution of this embodiment provides hardware design parameterization results for bit-level combinable computing arrays and automatically designs data scheduling for tensor operations and neural network applications with different bit widths, thereby achieving the goal of adding bit-level computing support to hardware design and data scheduling design, and providing quantitative evaluation and automated guidance for hardware and scheduling design.

[0098] In this embodiment, the following verification experiments are designed to verify the effectiveness of the technical solution of this embodiment. All experiments are run on the Ubuntu 18.04.1 operating system, kernel version 5.4.0-122-generic. Data flow evaluation, operation simulation, and design search experiments are implemented in Python language, Python version 3.9.7. The hardware function verification and simulation experiments use the Verilog language, the tool is Vivado 2021.2, and the auxiliary FPGA platform is Zynq UltraScale + ZCU104 Board. Among them, the difference analysis verifies the effectiveness of the design search and analyzes the differences caused by different scheduling strategies. In the reliability verification part, the bit-level evaluation and RTL simulation results are compared to verify the reliability of the evaluation results. In the design search verification part, the search results of the technical solution of this embodiment are compared with the existing manual design solution.

[0099] In the difference analysis verification, the differences between different data streams are analyzed to verify the effect of data stream design. The two-dimensional convolution tensor operation is used as the test program, the convolution scale is (48,256,3,3,27,27), and the batch size is 4. The hardware description is a single-layer slicing, the component width is 16 bits, and the two-dimensional array size is 16×16. There are no restrictions on storage and bandwidth. The array topology is limited to [i,j]=[[0,1,1],[1,0,1]], which is similar to DaDiannao. O is shared in the i direction, and one O is obtained for every 16 PEs. I is shared in the j direction, and 16 PEs share the same I. W is not shared and is unicast to each PE.

[0100] Since only the OC and IC sizes exceed 16, only these two loops need to be sharded when mapped to spatial IR. The range of other loops does not exceed 16, and direct mapping does not require sharding. The shard size should be sufficient to fill the hardware as much as possible, so the size is limited to 16. After mapping, a legality check is performed based on the topology restrictions. In addition, further restrictions are made on the data flow, and the restrictions are weighted stationary (WS). In the data flow IR, one of the three indexes x, b, and y must be located in the nested inner layer. The heterogeneous analysis verification results can be shown as follows Fig.18 As shown. Among them, Figure (a) gives the delay results and the average utilization results of PE components, which are sorted by utilization. The horizontal axis is different data streams, which are distinguished by numbers. The vertical axis is delay, and the delay unit is the number of cycles (Mcycles). The smaller the delay, the better the performance. The auxiliary vertical axis is utilization. The higher the utilization, the better the performance. The utilization is calculated based on the amount of data and the number of multiplication operations. Without considering data reuse, the number of accesses to each tensor is the multiplication of each index range. Each access performs a multiplication, and the theoretical total number of multiplications can be obtained. The component utilization is the PE multiplier utilization, which represents the efficiency of hardware resources. It is obtained by dividing the theoretical total number of multiplications by the number of multiplications at runtime. The hardware may not be full at runtime, and there are some empty operations, so the number of operations at runtime exceeds the total number of multiplications. The ratio of the two is the utilization. Utilization and latency are inversely correlated.

[0101] Figure (b) shows the data reuse rate of each specific tensor of these data flows. The reuse rate refers to the ratio of the amount of reused data to the total amount of data. In a certain batch of operations, if there is data reuse, there is no need to access memory, and the data is already in the register of the computing unit. This amount of data is the reuse amount. The higher the data reuse rate, the higher the memory access efficiency and the higher the overall efficiency of the data flow. Among various data flows, the optimal data flow reached 1412208 cycles, or 1.4Mcycles, with a PE utilization rate of 89%, and the hardware as a whole ran at a high load. An optimally scheduled data flow can be as follows Fig.19As shown, the relationship matrix is ​​omitted. In this data flow, spatial mapping and slicing are performed with indexes oc and ic, that is, the number of input and output channels. In the nested order, x and y are in the inner layer, followed by b, kx, and ky in the outer layer, that is, priority is given to accessing feature map rows and columns, then batch data, and finally weight values. This data flow allows the weight value W to participate in as many operations as possible before being swapped out, and W has a large amount of time reuse. On this basis, O is reused as much as possible.

[0102] The worst data flow requires 115Mcycles, and the component utilization is only 0.01087. An example is the spatial mapping of indexes oc and kx, with the nesting order from the inside to the outside being oc0, kx0, b, ky, x, ic, and oc1. The kx range is small, with a size of only 3, which means that 13 / 16 of the multipliers are always idle, and the hardware utilization is low. In addition, it can be seen from the nesting order that the data flow has little data reuse and a low reuse rate. First, the b index causes frequent swapping in and out of I and O, without reuse, and only a small amount of W is reused. Then the ky index swaps out W again, and W is not reused. The scheduler is always waiting for new data to be swapped in. When calculating the delay, the overall delay is high because a larger delay is taken as the final delay of a round of calculation. Although the effect is poor, this data flow is still a legal data flow on the specific hardware in this example, and the two-dimensional convolution can be performed correctly.

[0103] In summary, in a simple two-dimensional convolution and a specific hardware structure example, there are many different data flow designs, and the differences are large. It is difficult to estimate the differences between data flows based on human intuition and experience, so the design process of modeling, quantitative evaluation, and search for data flows is necessary and effective.

[0104] Secondly, in the reliability verification part, previous related evaluation work does not support bit-level combinability. Therefore, for bit-level calculation, this embodiment uses RTL to implement hardware calculation core design for combinable hardware abstraction, and performs functional verification and cycle-level simulation on FPGA (Field Programmable Gate Array). The hardware design language is Verilog, which implements 2-bit multipliers, shifters, registers and data exchange paths. 4-bit and 8-bit multiplication and addition calculations are implemented based on 16 two-bit multipliers, where each multiplier is followed by a configurable shifter, and the data bit width type is changed by configuring the number of shifts. FPGA-assisted design simulation and comprehensive verification are used. The vivado pre-simulation tool is used to perform cycle-level simulation of the application scheduling process, including 2b×2b, 2b×4b, 4b×4b, 4b×8b and 8b×8b matrix multiplication. The simulated scheduling process is generated from the data stream IR, the IR index corresponds to the loop index, the Smap part corresponds to the parallel data broadcast, and the nested order gives the data broadcast order. For an array of size 16×16, select the general matrix to multiply the three indexes in the range of [16×256×16]. The simulation frequency is 200MHz, that is, 5ns / cycle. The reliability verification results can be shown as follows Fig. 20 shown.

[0105] As can be seen from the figure, the evaluation of the computing core delay by the technical solution of this embodiment is close to the RTL (register-transfer level) cycle-level simulation results, with an accuracy rate of more than 93% and an average accuracy rate of 97%. The main source of error is the pipeline filling and emptying time under pulsation. This embodiment does not analyze pulsation control, so this part of the delay produces a large error. The main errors evaluated in this embodiment come from two aspects, pipeline filling and irregular boundaries. As the scale of calculation increases, the proportion of filling becomes smaller and smaller, and the accuracy will gradually improve.

[0106] Finally, in the design search verification, BitFusion is a typical bit-parallel combinable deep neural network accelerator, and its combined structure is still widely applicable. The bottom layer is a two-bit multiplier, 16 multipliers form a single PE, and each PE can be configured as a multiplier-adder of different bit widths. 16×16 PEs form a complete computing core. I rows share pulsation propagation, O columns share pulsation propagation, and W has no sharing. For two-dimensional convolution, the original BitFusion uses two methods, using im2col to convert the convolution into a matrix multiplication, or using a channel-wise data flow. This data flow expands the channel dimension of the output feature map to both directions of the array.

[0107] Use Fig.21The neural network shown in the figure is used for design search, where LeNet, Cifar, VGG7, and SVHN are 2b×2b data. LSTM, AlexNet, and ResNet are 4b×4b data. The evaluation results are statistically analyzed in the simulator, and the performance is calculated by dividing the data access volume by the bandwidth to get the number of cycles. The number of cycles is divided by the computing core operating frequency to get the maximum bandwidth limit of a single separated storage of 192bit / cycle. The frequency is set to 500MHz (megahertz), which is consistent with BitFusion. There is no restriction on storage for the time being. It is assumed that all data is cached on the chip, and the minimum storage requirement is taken. The storage power consumption is obtained by multiplying the total data access volume by the dynamic power consumption of data reading and writing. The dynamic power consumption is obtained by evaluating the storage with a network traffic monitoring graphic analysis tool. The computing core power consumption is obtained from the array database. The final evaluation results of the design search verification are shown in Fig. 22 As shown in the figure, the performance is improved by 1.07 times on average, the power consumption is improved by 1.1 times on average, and the overall energy efficiency, that is, the performance-to-power ratio, is improved by 1.17 times. This proves that the scheduling strategy automatically obtained by the data flow design search method of this embodiment is as efficient as the most advanced manual scheduling design.

[0108] Embodiment 2

[0109] Fig.23 This is a schematic diagram of the structure of a device for designing a bit-level combinable accelerator provided in Embodiment 2 of the present invention. Fig.23 As shown, the device includes: a hardware modeling module 210, a data flow modeling module 220 and a design point search module 230; wherein,

[0110] The hardware modeling module 210 is used to perform bit-level combinable hardware modeling on the accelerator to be designed and obtain a bit-level combinable hardware structure;

[0111] A data flow modeling module 220 is used to perform bit-level data flow modeling according to the bit-level combinable hardware structure by fusing software and hardware parameterized descriptions to obtain data flow description parameters;

[0112] The design point search module 230 is used to establish a design space according to the data flow description parameters, and obtain a target data flow by searching for an optimal design point in the design space.

[0113] The technical solution of the embodiment of the present invention performs bit-level combinable hardware modeling on the accelerator to be designed to obtain the bit-level combinable hardware structure; based on the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters; a design space is established according to the data flow description parameters, and the target data flow is obtained by searching the design space for the best design point; by performing bit-level combinable hardware modeling, bit-level data flow modeling and hardware scheduling design search, bit-level computing support can be added to the hardware design and data scheduling design of the accelerator, quantitative evaluation and automated guidance can be provided for hardware and scheduling design, and automatic hardware design and scheduling optimization of the bit-level combinable accelerator can be realized.

[0114] Optionally, a hardware modeling module 210 is specifically used to perform bit-level combinable computing unit modeling based on data bit grouping and vectorization methods to obtain a computing unit structure design, and to perform storage structure modeling to obtain a storage structure design;

[0115] According to the computing unit structure design and the storage structure design, a bit-level combinable hardware structure is obtained.

[0116] Optionally, the hardware modeling module 210 is specifically used to obtain the number of multipliers and the bit grouping size, and calculate the combined bit width configuration according to the number of multipliers and the bit grouping size through a bit width configuration algorithm;

[0117] According to the combined bit width configuration, obtaining a shifter type configuration by using spatial multiplexing and temporal multiplexing of the shifter;

[0118] The data multiplexing design is obtained by combining internal and external data for parallel data multiplexing, and the computing unit structure design is obtained according to the shifter type configuration and the data multiplexing design.

[0119] Optionally, the storage structure design includes a global storage structure design using multi-level cache and a local storage structure design using separate storage.

[0120] Optionally, a data flow modeling module 220 is specifically used to perform software and hardware modeling according to the bit-level combinable hardware structure to obtain software description parameters and hardware description parameters;

[0121] The software description parameters include tensor operation loop body characteristic parameters, the tensor operation loop body characteristic parameters include loop range, stride and tensor index relationship matrix, the hardware description parameters include multi-level hardware structure parameters, topology description and operand index relationship matrix, the topology description includes array arrangement index and data sharing area;

[0122] The scheduling process modeling is performed according to the software description parameters and the hardware description parameters, an intermediate expression primitive format is obtained, and a data flow description parameter is obtained according to the intermediate expression primitive format.

[0123] Optionally, a design point search module 230 is specifically used to obtain multiple feasible data flows according to the data flow description parameters, and establish a design space based on the multiple feasible data flows;

[0124] Each feasible data flow in the design space is evaluated, and the feasible data flow with the best evaluation result is obtained as the target data flow.

[0125] Optionally, a design point search module 230 is specifically used to prune the design space using redundant pruning and greedy strategies to obtain a pruned design space;

[0126] The backtracking pruning search algorithm is used to evaluate each feasible data flow in the pruned design space, and the feasible data flow with the best evaluation result is obtained as the target data flow.

[0127] The device for designing a bit-level combinable accelerator provided in an embodiment of the present invention can execute the method for designing a bit-level combinable accelerator provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.

[0128] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0129] Embodiment 3

[0130] Fig.24 A schematic diagram of the structure of an electronic device 30 that can be used to implement an embodiment of the present invention is shown. The electronic device 30 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0131] like Fig.24As shown, the electronic device 30 includes at least one processor 31, and a memory connected to the at least one processor 31, such as a read-only memory (ROM) 32, a random access memory (RAM) 33, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 31 can perform various appropriate actions and processes according to the computer program stored in the read-only memory 32 or the computer program loaded from the storage unit 38 to the random access memory 33. In the RAM 33, various programs and data required for the operation of the electronic device 30 can also be stored. The processor 31, the ROM 32 and the RAM 33 are connected to each other through a bus 34. The input / output (I / O) interface 35 is also connected to the bus 34.

[0132] A number of components in the electronic device 30 are connected to the I / O interface 35, including: an input unit 36, such as a keyboard, a mouse, etc.; an output unit 37, such as various types of displays, speakers, etc.; a storage unit 38, such as a disk, an optical disk, etc.; and a communication unit 39, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 39 allows the electronic device 30 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0133] The processor 31 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 31 include, but are not limited to, a central processing unit, a graphics processing unit, various special artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any appropriate processors, controllers, microcontrollers, etc. The processor 31 performs the various methods and processes described above, such as the design method of a bit-level combinable accelerator.

[0134] In some embodiments, the design method of the bit-level combinable accelerator may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 30 via the ROM 32 and / or the communication unit 39. When the computer program is loaded into the RAM 33 and executed by the processor 31, one or more steps of the design method of the bit-level combinable accelerator described above may be performed. Alternatively, in other embodiments, the processor 31 may be configured to execute the design method of the bit-level combinable accelerator in any other appropriate manner (e.g., by means of firmware).

[0135] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays, application specific integrated circuits, application specific standard products, systems on a chip, load programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0136] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0137] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device 30 having: a display device (e.g., a cathode ray tube or a liquid crystal display) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device 30. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0139] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area networks, wide area networks, blockchain networks, and the Internet.

[0140] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other. The server may be a cloud server.

[0141] This embodiment may also include a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method for designing a bit-level combinable accelerator provided by any embodiment of the present invention.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0143] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A design method for a bit-level combinatorial accelerator, characterized in that: include: Perform bit-level composable hardware modeling on the accelerator to be designed to obtain the bit-level composable hardware structure; According to the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters; A design space is established according to the data flow description parameters, and a target data flow is obtained by searching for an optimal design point in the design space.

2. The method according to claim 1, characterized in that Perform bit-level composable hardware modeling on the accelerator to be designed and obtain the bit-level composable hardware structure, including: Based on data bit grouping and vectorization methods, bit-level combinable computing unit modeling is performed to obtain the computing unit structure design, and storage structure modeling is performed to obtain the storage structure design; According to the computing unit structure design and the storage structure design, a bit-level combinable hardware structure is obtained.

3. The method according to claim 2, characterized in that Based on data bit grouping and vectorization methods, bit-level combinable computing unit modeling is performed to obtain the computing unit structure design, including: Obtaining the number of multipliers and the bit group size, and calculating a combined bit width configuration based on the number of multipliers and the bit group size using a bit width configuration algorithm; According to the combined bit width configuration, obtaining a shifter type configuration by adopting spatial multiplexing and temporal multiplexing of the shifter; By combining internal and external data for parallel data multiplexing, a data multiplexing design is obtained, and based on the shifter type configuration and the data multiplexing design, the computing unit structure design is obtained.

4. The method according to claim 2, characterized in that The storage structure design includes a global storage structure design using multi-level cache and a local storage structure design using separate storage.

5. The method according to claim 1, wherein According to the bit-level combinable hardware structure, bit-level data flow modeling is performed by fusing software and hardware parameterized descriptions to obtain data flow description parameters, including: Performing software and hardware modeling according to the bit-level combinable hardware structure to obtain software description parameters and hardware description parameters; The software description parameters include tensor operation loop body characteristic parameters, including loop range, stride and tensor index relationship matrix; the hardware description parameters include multi-level hardware structure parameters, topology description and operand index relationship matrix, and the topology description includes array arrangement index and data sharing area; The scheduling process modeling is performed according to the software description parameters and the hardware description parameters, an intermediate expression primitive format is obtained, and data flow description parameters are obtained according to the intermediate expression primitive format.

6. The method according to claim 1, wherein Establishing a design space according to the data flow description parameters and obtaining a target data flow by searching for an optimal design point in the design space includes: Acquire multiple feasible data flows according to the data flow description parameters, and establish a design space based on the multiple feasible data flows; Each feasible data flow in the design space is evaluated, and the feasible data flow with the best evaluation result is obtained as the target data flow.

7. The method according to claim 6, characterized in that Evaluating each feasible data flow in the design space and obtaining the feasible data flow with the best evaluation result as the target data flow, including: Pruning the design space using redundant pruning and greedy strategies to obtain a pruned design space; A backtracking pruning search algorithm is used to evaluate each feasible data flow in the pruned design space, and the feasible data flow with the best evaluation result is obtained as the target data flow.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the method for designing a bit-level combinable accelerator according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the method for designing a bit-level combinable accelerator according to any one of claims 1 to 7 when the computer program is executed.

10. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the method for designing a bit-level combinable accelerator according to any one of claims 1 to 7.

Images