A convolution operation device and a convolution neural network system

Abstract

The present invention provides a neural network unit, comprising a static random access memory unit, a forward readout isolation branch, and a reverse readout isolation branch; the static random access memory unit comprises a first transmission transistor and a second transmission transistor electrically connected in series, and two first and second inverters oppositely interlocked and connected in parallel between the first and second transmission transistors; the forward readout isolation branch is connected between the first transmission transistor and the two oppositely interlocked inverters, and is used to convert an externally input digital voltage into an analog current output according to a control signal stored in the static random access memory unit; the reverse readout isolation branch is connected between the second transmission transistor and the two oppositely interlocked inverters, and is used to convert an externally input digital voltage into an analog current output according to a control signal stored in the static random access memory unit.

Description

Technical Field

[0001] The present invention relates to the field of integrated circuit design, and in particular to a neural network unit, a convolution operation module and a convolutional neural network. Background Art

[0002] With the development of the big data era, artificial intelligence has become a very important academic field, and neural network dedicated chips are important hardware tools for computing systems to efficiently complete neural network calculations. The traditional computing architecture adopts the von Neumann system with separated computing and storage. Under the trend of big data, the memory bandwidth and memory power consumption in the von Neumann architecture have begun to dominate the computing bandwidth and energy. A large part of the power consumption is spent on data transfer between memory and computing units. In-memory computing dominated by memory greatly reduces the huge time and power consumption overhead caused by data transfer through the combination of neural network algorithms and storage hardware architecture.

[0003] The existing convolutional neural network storage and computing integrated memory (CIM SRAM) uses the traditional SRAM word line and bit line architecture, so when inputting data, the data received by each row of word lines is the same set of input data. Although different convolution kernels can be used for multiplication and addition operations when performing convolution operations on the same set of data, based on the current simple convolutional neural network algorithm, for the same set of convolution window data, at most 16-20 sets of convolution kernels are used. Therefore, for the SRAM array, only 16-20 columns of bit lines are used for operations, which wastes resources for the remaining array parts. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a neural network unit, a convolution operation module and a convolutional neural network, which improve the resource utilization of array units and reduce power consumption.

[0005] In order to solve the above problems, the present invention provides a neural network unit, including a static random access memory unit, a forward readout isolation branch, and a reverse readout isolation branch; the static random access memory unit includes a first transmission transistor and a second transmission transistor electrically connected in series, and two opposing interlocked first and second inverters connected in parallel between the first and second transmission transistors, the forward readout isolation branch is connected between the first transmission transistor and the two opposing interlocked inverters, and is used to convert an externally input digital voltage into an analog current output according to a control signal stored in the static random access memory unit; the reverse readout isolation branch is connected between the second transmission transistor and the two opposing interlocked inverters, and is used to convert an externally input digital voltage into an analog current output according to a control signal stored in the static random access memory unit.

[0006] Optionally, the forward read isolation branch includes a first readout transistor and a second readout transistor connected in series; the gate of the first readout transistor is connected between the first transfer transistor and two opposing interlocked inverters, serving as the control end of the forward read isolation branch, and the drain / source of the first readout transistor is connected to an operating voltage; the gate of the second readout transistor serves as an external input end, and the source / drain of the second readout transistor serves as an output end of the forward readout isolation branch.

[0007] Optionally, the reverse read isolation branch includes a third readout transistor and a fourth readout transistor connected in series; the gate of the third readout transistor is connected between the second transfer transistor and two opposing interlocked inverters, serving as the control end of the reverse read isolation branch, and the drain / source of the third readout transistor is grounded; the gate of the fourth readout transistor serves as an external input end, and the source / drain of the fourth readout transistor serves as an output end of the reverse readout isolation branch.

[0008] The present invention also provides a convolution operation module, which includes a convolution operator block and a pulse frequency quantization unit; the convolution operator block includes an array composed of N×N neural network units mentioned above, N is a positive integer, wherein: the first and second transmission transistors of the neural network unit are connected to the write line of the array, the drain / source of the first transmission transistor and the source / drain of the second transmission transistor are respectively connected to the bit line and the reverse bit line, the external input end of the forward isolation branch is connected to the row word line, the external input end of the reverse isolation branch is connected to the column word line, the output end of the forward isolation branch is connected to the row current output, and the output end of the reverse isolation branch is connected to the column current output; the pulse frequency quantization unit is connected to the output end of the convolution operator block, converts the current output in the convolution operator block into a pulse signal for pulse frequency counting, and then converts it into a multi-bit digital output.

[0009] Optionally, the pulse frequency quantization unit includes an amplifier, a comparator, and a counter connected in series: the amplifier is connected in parallel with an integrating capacitor and in series with the comparator, serving as the input of the pulse frequency quantization unit; the output of the comparator is further connected to both ends of the integrating capacitor via a discharge control switch, and the discharge control switch is used to discharge the integrating capacitor when the comparator outputs a high level; the integrating capacitor is charged by the analog current input and amplified by the amplifier, causing a pulse output of the comparator, thereby converting the current value into a pulse frequency; the counter is connected in series with the comparator, counts the number of pulses and forms a multi-bit digital output.

[0010] The present invention also provides a convolutional neural network, comprising an array consisting of N×N convolution operation modules as described above, where N is a positive integer.

[0011] In the present invention, each word line receives two groups of different data and performs convolution operations respectively. More groups of convolution kernels can be used for the data of the same group of convolution windows, thus avoiding waste of resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Attached Figure 1 Shown is a schematic diagram of a circuit structure of a neural network unit provided in a specific embodiment of the present invention.

[0013] Attached Figure 2 The figure is a schematic diagram of the circuit structure of a convolution operation sub-block in a convolution operation module according to a specific embodiment of the present invention.

[0014] Attached Figure 3 The figure is a schematic diagram of the circuit structure of a pulse quantization unit in a convolution operation module according to a specific embodiment of the present invention. DETAILED DESCRIPTION

[0015] The specific implementation modes of the static random access memory unit, the convolution operation module and the convolutional neural network provided by the present invention are described in detail below with reference to the accompanying drawings.

[0016] In the following specific implementations, the connection methods of the source and drain of the transistor are selectively described. It should be noted that in other specific implementations, the source and drain of the transistor can be equivalently replaced without affecting the actual electrical function.

[0017] Attached Figure 1 The figure shows a circuit structure diagram of a neural network unit provided by a specific embodiment of the present invention, including a static random access memory cell M1, a forward readout isolation branch B1, and a reverse readout isolation branch B2. The static random access memory cell M1 is a 6T structure, including a first transmission transistor T1 and a second transmission transistor T2 electrically connected in series, and two opposed interlocked first inverters D1 and second inverters D2 connected in parallel between the first transmission transistor T1 and the second transmission transistor T2.

[0018] In this specific embodiment, the forward readout isolation branch B1 includes a first readout transistor R1 and a second readout transistor R2 connected in series; the gate of the first readout transistor R1 is connected between the first transmission transistor T1 and two opposite interlocked inverters, as the control end of the forward readout isolation branch, and the drain of the first readout transistor R1 is connected to the working voltage VDD; the gate of the second transistor R2 is used as the external input end, and the source of the second readout transistor R2 is used as the output end of the forward readout isolation branch. The forward readout isolation branch B1 is connected between the first transmission transistor and the two opposite interlocked inverters, and is used to convert an external input digital voltage into an analog current output according to the control signal stored in the static random access memory cell M1.

[0019] In this specific embodiment, the reverse read isolation branch B2 includes a third read transistor R3 and a fourth read transistor R4 connected in series. The gate of the third read transistor R3 is connected between the second transmission transistor T2 and two opposing interlocked inverters, serving as the control end of the reverse read isolation branch, and the drain of the third read transistor R3 is grounded. The gate of the fourth transmission transistor R4 serves as an external input end, and the source of the fourth read transistor R4 serves as the output end of the reverse read isolation branch. The reverse read isolation branch is connected between the second transmission transistor T2 and two opposing interlocked inverters, and is used to convert an external input digital voltage into an analog current output according to the control signal stored in the static random access memory cell M1.

[0020] Next, a specific implementation of the convolution operation module of the present invention is given. The module includes a convolution operation sub-block and a pulse frequency quantization unit. Figure 2 The figure shows a schematic diagram of the circuit structure of a convolution operator block in a convolution operation module described in a specific embodiment of the present invention. The convolution operator block includes an array composed of N×N neural network units described above. In this specific embodiment, 3×3 neural network units C11~C33 are taken as an example for description. Among them, the first transmission transistor T1 and the second transmission transistor T2 of the neural network unit are connected to the write word line WWL of the array, the drain of the first transmission transistor T1 and the source of the second transmission transistor are respectively connected to the bit line BL and the reverse bit line BLB, the external input end of the forward isolation branch is connected to the row word line LWL, the external input end of the reverse isolation branch is connected to the column word line CWL, the output end of the forward isolation branch is connected to the row current output Lo, and the output end of the reverse isolation branch is connected to the column current output Co. The array can realize the convolution operation of the 3×3 convolution kernel on the 4×8 input matrix in three cycles.

[0021] Attached Figure 3 FIG. 1 is a schematic diagram of the circuit structure of a pulse quantization unit in a convolution operation module according to a specific embodiment of the present invention. The pulse frequency quantization unit is connected to the output end of the convolution operation sub-block, converts the current output in the convolution operation sub-block into a pulse signal for pulse frequency counting, and then converts it into a multi-bit digital output. As a specific embodiment of the above, as shown in FIG. Figure 2As shown, the pulse frequency quantization unit includes a series-connected amplifier 21, a comparator 22, and a counter 23. The amplifier 21 is connected in parallel with an integral capacitor C and in series with the comparator 22 as the input end of the pulse frequency quantization unit. The output end of the comparator 22 is further connected to the two ends of the integral capacitor C through a discharge control switch K, and the discharge control switch K is used to discharge the integral capacitor C when the comparator 22 outputs a high level 1; the integral capacitor C is charged by the analog current input and amplified by the amplifier 21, causing the pulse output of the comparator 22, thereby converting the current value into a pulse frequency equivalently. Specifically, the flip output of the comparator 22 is used as the discharge signal of the integral capacitor C. Whenever the integral capacitor C is charged by the analog current input to the flip voltage of the comparator 22, the output of the comparator 22 is flipped from 0 to 1. When the output of the comparator 22 is 1, the integral capacitor C is discharged, and the comparator 22 outputs 1 to 0, and then the comparator 22 forms a pulse output. The larger the analog input current, the faster the frequency of the pulse formation. The counter 23 is connected in series with the comparator 22 to count the number of pulses and form a multi-bit digital output.

[0022] The above attached Figure 2 and attached Figure 3 The N×N array of convolution operation modules including convolution operation sub-blocks and pulse frequency quantization units constitutes a convolutional neural network, which can perform accelerated calculations of the convolutional neural network.

[0023] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A convolution operation device, characterized in that: The device includes a convolution operator block and a pulse frequency quantization unit; The convolution operator block includes an array of N×N neural network units, where N is a positive integer. The neural network unit includes a static random access memory unit, a forward readout isolation branch, and a reverse readout isolation branch; The static random access memory cell includes a first transmission transistor and a second transmission transistor electrically connected in series, and two opposed interlocked first and second inverters connected in parallel between the first and second transmission transistors. The forward readout isolation branch is connected between the first transmission transistor and two opposing interlocked inverters, and is used to convert an external input digital voltage into an analog current output according to a control signal stored in the static random access memory cell; The reverse readout isolation branch is connected between the second transmission transistor and the two opposing interlocked inverters, and is used to convert an external input digital voltage into an analog current output according to a control signal stored in the static random access memory cell; in: The first and second transfer transistors of the neuron network unit are connected to the write word line of the array, the drain / source of the first transfer transistor and the source / drain of the second transfer transistor are connected to the bit line and the reverse bit line respectively, the external input end of the forward isolation branch is connected to the row word line, the external input end of the reverse isolation branch is connected to the column word line, the output end of the forward isolation branch is connected to the row current output, and the output end of the reverse isolation branch is connected to the column current output; The pulse frequency quantization unit is connected to the output end of the convolution operator block, converts the current output in the convolution operator block into a pulse signal for pulse frequency counting, and then converts it into a multi-bit digital output.

2. The convolution operation device according to claim 1, characterized in that: The pulse frequency quantization unit includes an amplifier, a comparator, and a counter connected in series: The amplifier is connected in parallel with an integrating capacitor and in series with a comparator, serving as an input terminal of a pulse frequency quantization unit; The output end of the comparator is further connected to the two ends of the integral capacitor through a discharge control switch, and the discharge control switch is used to discharge the integral capacitor when the comparator outputs a high level; the integral capacitor is charged by the analog current input and amplified by the amplifier, causing a pulse output of the comparator, thereby converting the current value into a pulse frequency equivalently; The counter is connected in series with the comparator to count the number of pulses and form a multi-bit digital output.

3. The convolution operation device according to claim 1, characterized in that: The forward readout isolation branch includes a first readout transistor and a second readout transistor connected in series; the gate of the first readout transistor is connected between the first transfer transistor and two opposing interlocked inverters, serving as the control end of the forward readout isolation branch, and the drain / source of the first readout transistor is connected to the operating voltage; the gate of the second readout transistor serves as the external input end, and the source / drain of the second readout transistor serves as the output end of the forward readout isolation branch.

4. The convolution operation device according to claim 1, characterized in that: The reverse readout isolation branch includes a third readout transistor and a fourth readout transistor connected in series; the gate of the third readout transistor is connected between the second transfer transistor and two opposing interlocked inverters, serving as the control end of the reverse readout isolation branch, and the drain / source of the third readout transistor is grounded; the gate of the fourth readout transistor serves as an external input end, and the source / drain of the fourth readout transistor serves as the output end of the reverse readout isolation branch.

5. A convolutional neural network system, comprising an array of N×N convolution operation devices as described in claim 1, where N is a positive integer.

Images