A systolic array hardware implementation method and device supporting multiple different precision operations

Abstract

The application provides a systolic array hardware implementation method and device supporting multiple different precision operations, and belongs to the technical field of high-performance microprocessor design. The method comprises the following steps: S1: establishing an association table associated with the stage of a neural network and an operation configuration; S2: obtaining the stage of the neural network, and obtaining the operation configuration corresponding to the stage from the association table; S3: if the operation configuration is a mixed precision floating point operation, then a 16*16bit+32bit fixed point / floating point operation model is established based on a systolic array, and the mixed precision floating point operation is performed based on the 16*16bit+32bit fixed point / floating point operation model; if the operation configuration is a fixed point shaping operation, then an 8*8bit+16bit fixed point operation model is established based on a systolic array, and the fixed point shaping operation is performed based on the 8*8bit+16bit fixed point operation model. The application fully excavates the space for reducing precision, improves processing throughput, improves the operation performance of the systolic array, and reduces unnecessary hardware overhead.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of high-performance microprocessor design, in particular to a systolic array hardware implementation method and device supporting multiple different precision operations. BACKGROUND

[0002] Deep Neutral Networks (DNN) has become the basis of many modern artificial intelligence applications, especially in the fields of speech recognition, image recognition and natural language processing, and has shown breakthrough results, making the number of applications increase explosively. However, although the accuracy of deep neural networks has surpassed humans in many fields, the super-high accuracy is at the cost of super-high density computing, and the hardware overhead of different precision types of computing is very different. Deep neural networks generally include training and inference stages, and the operations they need to complete in the computing process are different, and the precision of the data is also different. Therefore, the precision requirements of data for different stages of deep neural network computing are different, and it is of great significance to design different operation units to reduce the overhead. SUMMARY

[0003] In order to solve the above problems, the embodiments of the present application provide a systolic array hardware implementation method and device supporting multiple different precision operations.

[0004] In a first aspect, the embodiments of the present application provide a systolic array hardware implementation method supporting multiple different precision operations, including the following steps:

[0005] S1: establishing an association table associated with the stage and operation configuration of the neural network;

[0006] S2: obtaining the stage of the neural network, and obtaining the operation configuration corresponding to the stage from the association table;

[0007] S3: if the operation configuration is a mixed precision floating point operation, establishing a 16*16bit+32bit fixed point / floating point operation model based on the systolic array, and performing the mixed precision floating point operation based on the 16*16bit+32bit fixed point / floating point operation model; if the operation configuration is a fixed point shaping operation, establishing an 8*8bit+16bit fixed point operation model based on the systolic array, and performing the fixed point shaping operation based on the 8*8bit+16bit fixed point operation model.

[0008] Preferably, establishing a 16*16bit+32bit fixed point / floating point operation model based on the systolic array, and performing the mixed precision floating point operation based on the 16*16bit+32bit fixed point / floating point operation model specifically includes:

[0009] a first westward data processor with a data bit width of M*2B and a precision of 16 bits is established to calculate M channels of the input feature map;

[0010] a first northward data processor with a data bit width of N*2B and a precision of 16 bits is established to load N convolution kernels;

[0011] a first systolic array with a size of M*N is established;

[0012] a first southward data processor is established;

[0013] data output by the first westward data processor and data output by the first northward data processor are respectively input into the first systolic array to perform mixed-precision floating-point operation, the first systolic array inputs the operation result into the first southward data processor, and the first southward data processor obtains a first output feature image with N channels and a data bit width of N*4B.

[0014] Preferably, an 8*8bit+16bit fixed-point operation model is established based on the systolic array, and the fixed-point shaping operation based on the 8*8bit+16bit fixed-point operation model specifically includes:

[0015] a second westward data processor with a data bit width of M*2B and a precision of 8 bits is established to calculate 2M channels of the input feature map;

[0016] a first northward data processor with a data bit width of N*2B and a precision of 8 bits is established to load 2N convolution kernels;

[0017] a second systolic array with a size of M*2N is established;

[0018] a second southward data processor is established;

[0019] data output by the second westward data processor and data output by the second northward data processor are respectively input into the second systolic array to perform fixed-point shaping operation, the second systolic array inputs the operation result into the second southward data processor, and the second southward data processor obtains a second output feature image with 2N channels and a data bit width of N*2*2B.

[0020] Preferably, the second systolic array includes a data replication control unit, when the data replication control unit is set to 0, the second systolic array replicates the first M channels of the input image to form a first intermediate image with M*2B channels, when the data replication control unit is set to 1, the second systolic array replicates the last M channels of the input image to form a second intermediate image with M*2B channels, and the second systolic array performs convolution calculation on the first intermediate image and the second intermediate image to obtain an output image with 2M channels.

[0021] In a second aspect, the embodiments of the present application provide a hardware implementation device of systolic array supporting multiple different precision operations, comprising

[0022] An association table establishing module is configured to establish an association table associated with stages of a neural network and operation configurations;

[0023] A matching module is configured to obtain a stage of the neural network, and obtain an operation configuration corresponding to the stage from the association table;

[0024] A model calculation module is configured to, if the operation configuration is a mixed precision floating point operation, establish a 16*16bit+32bit fixed point / floating point operation model based on the systolic array, and perform the mixed precision floating point operation based on the 16*16bit+32bit fixed point / floating point operation model, if the operation configuration is a fixed point shaping operation, establish an 8*8bit+16bit fixed point operation model based on the systolic array, and perform the fixed point shaping operation based on the 8*8bit+16bit fixed point operation model.

[0025] Preferably, the 16*16bit+32bit fixed point / floating point operation model comprises:

[0026] a first westward data processor with a data bit width of M*2B and a precision of 16bit, configured to perform calculation on M channels of an input feature graph,

[0027] a first northward data processor with a data bit width of N*2B and a precision of 16bit, configured to load N convolution kernels,

[0028] a first systolic array with a size of M*N,

[0029] a first southward data processor,

[0030] The first systolic array performs the mixed precision floating point operation when the first westward data processor and the first northward data processor respectively input data into the first systolic array, and the first southward data processor obtains a first output feature image with N channels and a data bit width of N*4B when the first systolic array inputs an operation result into the first southward data processor.

[0031] Preferably, the 8*8bit+16bit fixed point operation model comprises

[0032] a second westward data processor with a data bit width of M*2B and a precision of 8bit, configured to perform calculation on 2M channels of an input feature graph,

[0033] a first northward data processor with a data bit width of N*2B and a precision of 8bit, configured to load 2N convolution kernels,

[0034] a second systolic array with a size of M*2N;

[0035] a second southward data processor;

[0036] The second systolic array performs fixed-point shaping operation when the second westward data processor and the second northward data processor input data into the second systolic array, respectively, and the second southward data processor obtains a second output feature image with 2N channels and a data bit width of N*2*2B when the second systolic array inputs operation results into the second southward data processor.

[0037] Preferably, the second systolic array comprises a data replication control unit, when the data replication control unit is set to 0, the second systolic array replicates the first M channels of the input image to form a first intermediate image with M*2B channels, when the data replication control unit is set to 1, the second systolic array replicates the last M channels of the input image to form a second intermediate image with M*2B channels, and the second systolic array performs convolution calculation on the first intermediate image and the second intermediate image to obtain an output image with 2M channels.

[0038] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method provided in the first aspect or any possible implementation manner of the first aspect when executing the computer program.

[0039] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the method provided in the first aspect or any possible implementation manner of the first aspect when executed by a processor.

[0040] The present application has the following beneficial effects: according to the association table, the operation configuration corresponding to the stage of the neural network is obtained, if the operation configuration is mixed precision floating point operation, a 16*16bit+32bit fixed point / floating point operation model is established based on the systolic array, the mixed precision floating point operation is performed based on the 16*16bit+32bit fixed point / floating point operation model, if the operation configuration is fixed-point shaping operation, an 8*8bit+16bit fixed point operation model is established based on the systolic array, the fixed-point shaping operation is performed based on the 8*8bit+16bit fixed point operation model, different precision calculation modes are realized on the systolic array, on the basis of guaranteeing the data precision requirement of the deep neural network algorithm under different layers, different scenes and different stages, the space of reducing precision is fully mined, the processing throughput is improved, the operation performance of the systolic array is improved, and unnecessary hardware overhead is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described in the following description only constitute some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without any creative effort.

[0042] Figure 1 A flowchart of a hardware implementation method of a systolic array supporting multiple different precision operations provided by an embodiment of the present application;

[0043] Figure 2 A structural diagram of a hardware implementation device of a systolic array supporting multiple different precision operations provided by an embodiment of the present application;

[0044] Figure 3 A structural diagram of an electronic device provided by an embodiment of the present application;

[0045] Figure 4 A principle diagram of a 16*16+32bit operation mode systolic array working condition in a hardware implementation method of a systolic array supporting multiple different precision operations provided by an embodiment of the present application;

[0046] Figure 5 A principle diagram of an 8*8+16bit operation mode systolic array working condition in a hardware implementation method of a systolic array supporting multiple different precision operations provided by an embodiment of the present application;

[0047] Figure 6 A principle diagram of a neural network model convolution calculation process in a hardware implementation method of a systolic array supporting multiple different precision operations provided by an embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application.

[0049] In the following description, the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance. The following description provides multiple embodiments of the present application, and different embodiments can be replaced or combined, so the present application can also be considered to include all possible combinations of the same and / or different embodiments described. Therefore, if one embodiment includes features A, B, C, and another embodiment includes features B, D, the present application should also be considered to include one or more embodiments of all other possible combinations of A, B, C, and D, although the embodiment may not be explicitly described in the following.

[0050] The following description provides examples, and is not limiting of the scope, applicability, or examples set forth in the claims. Changes can be made in the function and arrangement of elements discussed without departing from the scope of the application. Various examples can omit, substitute, or add various procedures or components as appropriate. For instance, the methods described can be performed in an order different than described, and various steps can be added, omitted, or combined. Also, features described with respect to some examples can be combined in other examples.

[0051] See Figure 1 . Figure 1 is a flowchart of a hardware implementation method of a systolic array supporting multiple different precision operations provided by an embodiment of the application. In the embodiment of the application, the method comprises the following steps:

[0052] S1: establishing an association table associated with the stage of the neural network and the operation configuration;

[0053] S2: obtaining the stage of the neural network, and obtaining the operation configuration corresponding to the stage from the association table;

[0054] S3: if the operation configuration is a mixed precision floating point operation, establishing a 16*16bit+32bit fixed point / floating point operation model based on the systolic array, and performing the mixed precision floating point operation based on the 16*16bit+32bit fixed point / floating point operation model, if the operation configuration is a fixed point shaping operation, establishing an 8*8bit+16bit fixed point operation model based on the systolic array, and performing the fixed point shaping operation based on the 8*8bit+16bit fixed point operation model.

[0055] In the embodiment of the application, see Figure 6 , Figure 6The schematic diagram of the calculation process of the neural network model convolution in the hardware implementation method of the systolic array supporting multiple different precision operations provided by the embodiment of the application, the deep neural network generally includes two stages of training (Training) and inference (inference), and the operations required to be completed in the calculation process are different, and the precision of the data is also different, so different precisions can be used for processing. In the training stage of the deep learning network model, a pre-labeled data set is used, the deep learning network is supervised and trained using the back propagation algorithm, the parameters of the network are updated through the error, and the iteration is continuously performed until the network converges to the expected error range. In the training process, the gradient needs to be multiplied several times, and the normalization layer needs to be accumulated several times, and the data precision requirement is high. At present, the industry generally adopts the method of mixed precision floating point operation for training, that is, the multiplication operation adopts half precision, and the addition operation adopts single precision, which can not only ensure that the gradient does not disappear or explode in the process of backward gradient propagation, but also can reduce the unnecessary hardware overhead caused by pure single precision calculation, and the training precision is almost not reduced. In the inference stage of the deep learning network model, the trained neural network is used to infer the input data, and this process does not involve precision-sensitive calculation such as gradient update, so the precision requirement is low. The industry usually uses 16bit or 8bit fixed-point shaping operation to realize inference, which will not affect the inference result.

[0056] In summary, on the basis of considering the hardware overhead, in order to meet the calculation precision requirements of different stages of the deep neural network, a configurable operation unit can be designed, which can be configured into different working modes according to specific calculation requirements, such as training configured into mixed precision floating point operation and inference configured into fixed-point shaping operation. The precision requirements of each stage of the deep neural network are met, and the hardware overhead is saved, and the performance is improved

[0057] In an implementation manner, a 16*16bit+32bit fixed point / floating point operation model is established based on the systolic array, and the mixed precision floating point operation based on the 16*16bit+32bit fixed point / floating point operation model specifically includes:

[0058] A first westward data processor with a data bit width of M*2B and a precision of 16bit is set up to calculate M channels of input feature graphs;

[0059] A first northward data processor with a data bit width of N*2B and a precision of 16bit is set up to load N convolution kernels;

[0060] A first systolic array with a size of M*N is set up;

[0061] A first southward data processor is set up;

[0062] The data output by the first west data processor and the data output by the first north data processor are input into the first systolic array respectively for mixed-precision floating-point operation, the first systolic array inputs the operation result into the first south data processor, and the first south data processor obtains the first output feature image with N channels and a data bit width of N*4B.

[0063] In the embodiment of the present application, B represents 1 Byte, 1 Byte = 8 bits. The systolic array of the present application supports multiple operation modes, such as 8*8 bit + 16 bit fixed point, 16*16 bit + 32 bit fixed point / floating point. According to the deep neural network convolution calculation process, as shown in the figure, when 8*8 + 16 bit and 16*16 + 32 bit calculations are performed, the operation mode of the systolic array is different, and the following will specifically describe the operation modes of the systolic array. Figure 3

[0064] The size of the systolic array is M*N, data is loaded from the north and west ports, and the operation result is output from the south of the systolic array. The present application refers to the north data, west data and south data as north data, west data and south data respectively. The bit width of the north data and the west data is 2B. For a deep convolutional neural network, the north data is usually a convolution kernel, and the systolic array loads one convolution kernel per column; the west data is usually an input feature image, each row corresponds to one channel; and the south data is usually an output feature image, each column corresponds to one channel.

[0065] Referring to Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the working condition of the 16*16 + 32 bit operation mode of the systolic array in the hardware implementation method of the systolic array supporting multiple different precision operations provided in the embodiment of the present application. During deep neural network training, the precision of the north data output by the first north data processor and the precision of the west data output by the first west data processor are both 16 bits, and the training is performed in the 16*16 + 32 bit floating point calculation mode. During the working process of the first systolic array, the first west data processor can simultaneously process the calculation of M channels of input feature images, and the data bit width is M*2B. The first north data processor can load N convolution kernels, each convolution kernel loads M channels, and the data bit width is N*2B. The first south data processor obtains the result of N channels of output feature images, and the data bit width is N*4B. All of them are in the form of continuous memory data transmission from the storage component.

[0066] In one implementation manner, an 8*8 bit + 16 bit fixed point operation model is established based on the systolic array, and the fixed point shaping operation based on the 8*8 bit + 16 bit fixed point operation model specifically includes:

[0067] ​a second westward data processor with a data bit width of M*2B and a precision of 8 bits is set up to calculate 2M channels of the input feature map;

[0068] a first northward data processor with a data bit width of N*2B and a precision of 8 bits is set up to load 2N convolution kernels;

[0069] a second systolic array with a size of M*2N is set up;

[0070] a second southward data processor is set up;

[0071] data output by the second westward data processor and data output by the second northward data processor are respectively input into the second systolic array to perform fixed-point shaping operation, the second systolic array inputs the operation result into the second southward data processor, and the second southward data processor obtains a second output feature image with 2N channels and a data bit width of N*2*2B.

[0072] In an implementation manner, the second systolic array comprises a data replication control unit, when the data replication control unit is set to 0, the second systolic array replicates the first M channels of the input image to form a first intermediate image with M*2B channels, when the data replication control unit is set to 1, the second systolic array replicates the last M channels of the input image to form a second intermediate image with M*2B channels, and the second systolic array performs convolution calculation on the first intermediate image and the second intermediate image to obtain an output image with 2M channels.

[0073] In the embodiments of the present application, referring to Figure 5 , Figure 5 FIG. 1 is a schematic diagram of the working condition of an 8*8+16 bit operation mode in a hardware implementation method of a systolic array supporting multiple different precision operations provided by the embodiments of the present application, one multiplication-addition unit supporting 16*16+32 bit can simultaneously perform two 8*8+16 bit operations, therefore, when the second systolic array works in the 8*8+16 bit calculation mode, the performance is twice that of the 16*16+32 bit calculation mode, which is equivalent to expanding the original M*N systolic array into a 2M*N systolic array.

[0074] For the second systolic array of 2M*N, the bit width of the northward data and the westward data loaded each time is still the same as that of the 16*16+32bit calculation mode, but the data volume is doubled. The second systolic array northward can load 2*N convolution kernels, each of which has M channels. According to the principle of convolution calculation, the channel number must be the same when the input feature image is operated with the convolution kernel, and therefore the westward data output by the second westward data processor only needs M data. Since the second systolic array has 2 MACs in each CELL at this time, the M data needs to be copied and expanded to 2M, so that the second systolic array can obtain the calculation result of 2N channels each time.

[0075] Since the westward data granularity accessed by the second systolic array is M*2B, when the hardware is implemented, a data copy control signal is set. When the second systolic array works in 8*8+16bit, the first M channels of the input image are copied to become a first intermediate image with M*2B channels when the bit is configured as 0, and the last M channels of the input image are copied to become a second intermediate image with M*2B channels when the bit is configured as 1. Two convolution calculations can obtain an output image with 2M channels. When working in 16*16+32bit, ignore the signal and do not copy the data.

[0076] The systolic array supports 16*16+32bit and 8*8+16bit different precision type calculation, and realizes data copy function through a 1-bit marking signal. The performance of the algorithm in the 8*8+16bit working mode is twice that of the 16*16+32bit.

[0077] The embodiments of the present application will be described below with reference to the accompanying drawings. Figure 2 The embodiments of the present application will be described below with reference to the accompanying drawings. Figure 2 The embodiments of the present application will be described below with reference to the accompanying drawings. Figure 1 The embodiments of the present application will be described below with reference to the accompanying drawings. Figure 1 The embodiments of the present application will be described below with reference to the accompanying drawings.

[0078] Please refer to Figure 2 , Figure 2 is a structure diagram of a systolic array hardware implementation device provided by an embodiment of the present application. As shown in Figure 2 , the device comprises

[0079] The association table establishing module 201 is used to establish an association table related to the stage and operation configuration of the neural network;

[0080] The matching module 202 is configured to obtain a stage of a neural network, and obtain an operation configuration corresponding to the stage from an association table.

[0081] The model calculation module 203 is configured to, if the operation configuration is a mixed precision floating point operation, establish a 16*16bit+32bit fixed point / floating point operation model based on a systolic array, and perform the mixed precision floating point operation based on the 16*16bit+32bit fixed point / floating point operation model, or if the operation configuration is a fixed point shaping operation, establish an 8*8bit+16bit fixed point operation model based on the systolic array, and perform the fixed point shaping operation based on the 8*8bit+16bit fixed point operation model.

[0082] In an implementation, the 16*16bit+32bit fixed point / floating point operation model comprises:

[0083] a first westward data processor with a data bit width of M*2B and a precision of 16bit, configured to perform calculation on M channels of an input feature map,

[0084] a first northward data processor with a data bit width of N*2B and a precision of 16bit, configured to load N convolution kernels,

[0085] a first systolic array with a size of M*N,

[0086] a first southward data processor,

[0087] The first systolic array performs the mixed precision floating point operation when the first westward data processor and the first northward data processor respectively input data into the first systolic array, and the first southward data processor obtains a first output feature image with N channels and a data bit width of N*4B when the first systolic array inputs an operation result into the first southward data processor.

[0088] In an implementation, the 8*8bit+16bit fixed point operation model comprises

[0089] a second westward data processor with a data bit width of M*2B and a precision of 8bit, configured to perform calculation on 2M channels of an input feature map,

[0090] a first northward data processor with a data bit width of N*2B and a precision of 8bit, configured to load 2N convolution kernels,

[0091] a second systolic array with a size of M*2N,

[0092] a second southward data processor,

[0093] The second pulsatile array performs fixed-point shaping operation when the second westward data processor and the second northward data processor input data into the second pulsatile array respectively, and the second southward data processor obtains a second output feature image with 2N channels and a data bit width of N*2*2B when the second pulsatile array inputs an operation result into the second southward data processor.

[0094] In an implementation manner, the second pulsatile array comprises a data replication control unit, when the data replication control unit is set to 0, the second pulsatile array replicates the first M channels of the input image to form a first intermediate image with M*2B channels, when the data replication control unit is set to 1, the second pulsatile array replicates the last M channels of the input image to form a second intermediate image with M*2B channels, and the second pulsatile array performs convolution calculation on the first intermediate image and the second intermediate image to obtain an output image with 2M channels.

[0095] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented by means of software and / or hardware. The "unit" and "module" in the specification refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, and the hardware may, for example, be a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), and the like.

[0096] The various processing units and / or modules of the embodiments of the present application can be implemented by means of analog circuits that implement the functions of the embodiments of the present application, or can be implemented by means of software that implements the functions of the embodiments of the present application.

[0097] Referring to Figure 3 , a structural schematic diagram of an electronic device related to the embodiments of the present application is shown, which can be used to implement the method in the embodiments shown in Figure 1 . As shown in Figure 3 , the electronic device 300 can include at least one central processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0098] The communication bus 302 is used to realize the connection and communication between the components.

[0099] The user interface 303 can include a display screen (Display) and a camera (Camera), and the optional user interface 303 can further include a standard wired interface and a wireless interface.

[0100] The network interface 304 can optionally include a standard wired interface, a wireless interface (e.g., a WI-FI interface).

[0101] The central processor 301 can include one or more processing cores. The central processor 301 connects various parts within the electronic device 300 through various interfaces and lines, and performs various functions of the terminal 300 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. The central processor 301 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The central processor 301 can be integrated with one or a combination of a central central processing unit (CPU), a graphics processing unit (GPU), and a modem. The CPU is mainly responsible for processing an operating system, a user interface, and an application program. The GPU is responsible for rendering and drawing content to be displayed on a display screen. The modem is responsible for processing wireless communication. It can be understood that the modem can also not be integrated into the central processor 301, but can be implemented by a separate chip.

[0102] The memory 305 can include a random access memory (RAM) and a read-only memory (ROM). The memory 305 can include a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 can include a program storage area and a data storage area. The program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can store data related to the above-mentioned various method embodiments, etc. The memory 305 can optionally be at least one storage device located away from the central processor 301. As shown, the memory 305 as a computer storage medium can include an operating system, a network communication module, a user interface module, and program instructions. Figure 3 As shown, the memory 305 as a computer storage medium can include an operating system, a network communication module, a user interface module, and program instructions.

[0103] In Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an interface for user input, and obtain data input by the user; and the central processing unit 301 can be used to call an application program stored in the memory 305 supporting a plurality of different precision operations, and specifically perform the following operations:

[0104] S1: establishing an association table associated with the stage of the neural network and the operation configuration;

[0105] S2: obtaining the stage of the neural network, and obtaining the operation configuration corresponding to the stage from the association table;

[0106] S3: if the operation configuration is a mixed precision floating point operation, a 16*16bit+32bit fixed point / floating point operation model is established based on the systolic array, and the mixed precision floating point operation is performed based on the 16*16bit+32bit fixed point / floating point operation model; if the operation configuration is a fixed point shaping operation, an 8*8bit+16bit fixed point operation model is established based on the systolic array, and the fixed point shaping operation is performed based on the 8*8bit+16bit fixed point operation model.

[0107] The application also provides a computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps of the above method. The computer readable storage medium can include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0108] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a series of action combinations, but those skilled in the art should know that the application is not limited by the action order described, because according to the application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the application.

[0109] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the division of the apparatus embodiments described above is merely illustrative, and the division of units can be different, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some intervening general purpose or special purpose hardware devices, or can be in electrical or other forms.

[0111] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0112] In addition, the functional units in each embodiment of the present application can be integrated into a processing unit, or each unit can be physically present alone, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0113] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable memory. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or all or part of the technical solutions can be embodied in the form of a software product, which is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory includes: a U disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0114] Those of ordinary skill in the art can understand that all or part of the steps of the various methods of the above embodiments can be completed by a program instructing relevant hardware, and the program can be stored in a computer readable memory, which can include a flash disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, etc.

[0115] The above merely show example embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Those skilled in the art will readily conceive of other implementations of the present disclosure upon considering the specification and practicing the disclosure herein. The present application is intended to cover any variations, uses, or adaptive changes of the present disclosure following the general principles of the present disclosure and including common knowledge or conventional technical means in the art not described in the present disclosure. The specification and examples are merely considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A systolic array hardware implementation method supporting multiple different precision operations, characterized by, The method comprises the following steps: S1: establishing an association table associated with the stage of the neural network and the operation configuration; S2: obtaining the stage of the neural network, and obtaining the operation configuration corresponding to the stage from the association table; S3: if the operation configuration is a mixed precision floating point operation, establishing a 16*16bit+32bit fixed point / floating point operation model based on the systolic array, and performing the mixed precision floating point operation based on the 16*16bit+32bit fixed point / floating point operation model; if the operation configuration is a fixed point shaping operation, establishing an 8*8bit+16bit fixed point operation model based on the systolic array, and performing the fixed point shaping operation based on the 8*8bit+16bit fixed point operation model; The 16*16bit+32bit fixed point / floating point operation model based on the systolic array comprises: a first westward data processor with a data bit width of M*2B and a precision of 16bit, used for calculating M channels of an input feature image; a first northward data processor with a data bit width of N*2B and a precision of 16bit, used for loading N convolution kernels; a first systolic array with a size of M*N; a first southward data processor; the data output by the first westward data processor and the data output by the first northward data processor are input into the first systolic array for mixed precision floating point operation, the first systolic array inputs the operation result into the first southward data processor, and the first southward data processor obtains a first output feature image with N channels and a data bit width of N*4B; The 8*8bit+16bit fixed point operation model based on the systolic array comprises: a second westward data processor with a data bit width of M*2B and a precision of 8bit, used for calculating 2M channels of an input feature image; a first northward data processor with a data bit width of N*2B and a precision of 8bit, used for loading 2N convolution kernels; a second systolic array with a size of M*2N; a second southward data processor; the data output by the second westward data processor and the data output by the second northward data processor are input into the second systolic array for fixed point shaping operation, the second systolic array inputs the operation result into the second southward data processor, and the second southward data processor obtains a second output feature image with 2N channels and a data bit width of N*2*2B.

2. A systolic array hardware implementation method of supporting multiple different precision operations as claimed in claim 1, wherein, The second systolic array comprises a data replication control unit, when the data replication control unit is set to 0, the second systolic array replicates the first M channels of an input image to form a first intermediate image with M*2B channels, when the data replication control unit is set to 1, the second systolic array replicates the last M channels of an input image to form a second intermediate image with M*2B channels, and the second systolic array performs convolution calculation on the first intermediate image and the second intermediate image to obtain an output image with 2M channels.

3. A hardware implementation device for a pulsating array that supports multiple operations with different precisions, characterized in that: The association table establishment module is configured to establish an association table between stages of the neural network and operation configurations; The matching module is configured to obtain a stage of the neural network, and obtain an operation configuration corresponding to the stage from the association table; The model calculation module is configured to, if the operation configuration is a mixed-precision floating-point operation, establish a 16*16bit+32bit fixed-point / floating-point operation model based on a systolic array, and perform the mixed-precision floating-point operation based on the 16*16bit+32bit fixed-point / floating-point operation model, or if the operation configuration is a fixed-point shaping operation, establish an 8*8bit+16bit fixed-point operation model based on a systolic array, and perform the fixed-point shaping operation based on the 8*8bit+16bit fixed-point operation model. The 16*16bit+32bit fixed-point / floating-point operation model comprises: a first westward data processor with a data bit width of M*2B and a precision of 16bit, configured to perform calculation on M channels of an input feature image, a first northward data processor with a data bit width of N*2B and a precision of 16bit, configured to load N convolution kernels, a first systolic array with a size of M*N, a first southward data processor, The first systolic array performs the mixed-precision floating-point operation when the first westward data processor and the first northward data processor respectively input data into the first systolic array, and the first southward data processor obtains a first output feature image with N channels and a data bit width of N*4B when the first systolic array inputs an operation result into the first southward data processor. The 8*8bit+16bit fixed-point operation model comprises a second westward data processor with a data bit width of M*2B and a precision of 8bit, configured to perform calculation on 2M channels of an input feature image, a first northward data processor with a data bit width of N*2B and a precision of 8bit, configured to load 2N convolution kernels, a second systolic array with a size of M*2N, a second southward data processor, The second systolic array performs the fixed-point shaping operation when the second westward data processor and the second northward data processor respectively input data into the second systolic array, and the second southward data processor obtains a second output feature image with 2N channels and a data bit width of N*2*2B when the second systolic array inputs an operation result into the second southward data processor.

4. A systolic array hardware implementation device supporting multiple different precision operations as recited in claim 3, wherein, The second systolic array comprises a data replication control unit, when the data replication control unit is set to 0, the second systolic array replicates the first M channels of an input image to form a first intermediate image with M*2B channels, when the data replication control unit is set to 1, the second systolic array replicates the last M channels of an input image to form a second intermediate image with M*2B channels, and the second systolic array performs convolution calculation on the first intermediate image and the second intermediate image to obtain an output image with 2M channels.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1-2 when executing the computer program.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by the processor, implements the steps of the method of any one of claims 1-2.

Images