Linear multiplication and addition circuit for image processing, implementation method of linear multiplication and addition circuit and chip

Abstract

The invention discloses a linear multiplication and addition circuit for image processing, an implementation method thereof and a chip. The linear multiplication and addition circuit comprises a digital processing module, a register, a pulse generator, a linear neuron pulse circuit and a counter which are connected in sequence, the digital processing module is used for converting the image gray absolute value into multi-bit data; the register is used for arranging and caching the multi-bit data according to digits; the pulse generator is used for converting the multi-bit data into high and low level signals; the linear neuron pulse circuit is used for converting the high and low levels into pulses with a certain frequency; and the counter is used for calculating the pulse output number of the linear neuron circuit. The linear multiplication and addition circuit is adopted, large-scale circuits such as a multiplier and an adder array are avoided, high-speed linear operation of image gray scale information can be achieved, and power consumption of the circuit is greatly reduced.

Description

Technical Field

[0001] The invention relates to a linear multiplication-addition circuit for image processing and a realization method and a chip thereof, and belongs to the field of integrated circuits. Background Art

[0002] The convolution multiplication and addition computing chips of traditional convolutional neural networks, such as GPU, FPGA, NPU, etc., need to use a combination of digital modules such as multipliers and adders to implement linear operations such as convolution and full connection, which greatly limits the speed of the circuit and increases power consumption.

[0003] In order to solve the storage wall problem caused by the separation of computing and storage units in the traditional von Neumann architecture, including time extension and high energy consumption, the integrated storage and computing structure came into being. The integrated storage and computing architecture integrates the computing unit with the storage unit, or moves the computing function to the inside of the memory, reducing the frequent transfer of data between the computing unit and the storage unit. This greatly reduces the time and energy consumption of data transmission. Since data can be processed directly inside the memory, the integrated storage and computing architecture can support high-throughput data processing, which is a significant advantage for big data and artificial intelligence applications. At present, common storage and computing integration implementation methods include: Flash-based storage and computing integration, RRAM-based, SRAM-based storage and computing integration, etc., but there are still problems with redundant data and computing. In addition, the traditional linear multiplication and addition array chip needs to specify the size of the array according to the usage scenario at the beginning of the chip design, and the connection between chips is directly completed through analog layout design or digital back-end design, which increases the loss of analog layout and digital back-end. Summary of the invention

[0004] In view of the above problems existing in the prior art, the present invention proposes a linear multiplication and addition circuit for image processing and its implementation method and chip, which can realize linear multiplication and addition operations of image grayscale information and avoid large-scale circuits such as multipliers and adder arrays.

[0005] The technical solution adopted by the circuit of the present invention is as follows:

[0006] A linear multiplication and addition circuit for image processing comprises a digital processing module, a register, a pulse generator, a linear neuron pulse circuit and a counter connected in sequence; the digital processing module is used to convert the absolute value of the image grayscale into multi-bit data; the register is used to sort and cache the multi-bit data according to the number of bits; the pulse generator is used to convert the multi-bit data into high and low level signals; the linear neuron pulse circuit is used to convert the high and low levels into pulses of a certain frequency; the counter is used to count the number of pulse outputs of the linear neuron circuit.

[0007] The present invention also provides a method for implementing a linear multiplication and addition circuit for image processing, the method comprising: the digital processing module performs preprocessing operations on the grayscale information of the image, and then transmits the preprocessed grayscale information to a register and converts it into grayscale information data; the grayscale information data is then converted into a square wave required by a neuron through a pulse generator, and then input into a linear neuron pulse circuit; the linear neuron pulse circuit outputs pulses with a certain frequency, and the number of output pulses is counted by a counter.

[0008] The present invention also provides a chip of a linear multiplication-addition circuit for image processing, the chip integrating the linear multiplication-addition circuit, the chip comprising a plurality of input ports and a plurality of output ports, which are respectively arranged at the four sides and corners of the chip packaging shell, for receiving off-chip data and outputting on-chip data; wherein the processing result of the linear multiplication-addition circuit is transmitted on-chip, and the signal input to the linear neuron pulse circuit from outside the chip is input through the input ports at the four sides and corners of the chip.

[0009] The beneficial effects of the present invention are:

[0010] (1) Compared with the traditional convolution multiplication and addition computing chip, the present invention adopts a linear multiplication and addition circuit, which can realize high-speed linear operation of image grayscale information and greatly reduce the power consumption of the circuit.

[0011] (2) The present invention adopts an integrated storage and computing structure based on SRAM. Storing data in SRAM and performing calculations on the chip can greatly reduce the amount of redundant data and the amount of calculation, thereby reducing storage and computing pressure.

[0012] (3) By forming an array of the circuits of the present invention and simultaneously processing the image grayscale information within a matrix range and subsequent operations, the eigenvalues ​​of the grayscale information of the image within a certain matrix range can be extracted, and the convolution calculation of the grayscale matrix is ​​actually completed.

[0013] (4) The chip module design and connection method proposed in the present invention can completely isolate a single chip. In the scenario of array application, the array can be combined by plug-in connection according to the actual array scale requirements, which not only reduces the loss of analog layout and digital back-end, but also increases the flexibility of array scale. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0015] Figure 1is a block diagram of a linear multiplication-addition circuit of the present invention;

[0016] Figure 2 is a block diagram of a digital logic device used in the present invention;

[0017] Figure 3 is a schematic diagram of a packaged single chip according to Embodiment 1 of the present invention;

[0018] Figure 4 It is a schematic diagram of the chip array combination of Example 2 of the present invention. DETAILED DESCRIPTION

[0019] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0020] Example 1

[0021] like Figure 1 , Figure 2 As shown, the present embodiment is a linear multiplication and addition circuit for image processing, comprising a digital processing module, a register, a pulse generator, a linear neuron pulse circuit and a counter connected in sequence. Among them, the digital processing module is used to convert the absolute value of the image grayscale into multi-bit data; the register is used to sort and cache the multi-bit data according to the number of bits; the pulse generator is used to convert the multi-bit data into high and low level signals; the linear neuron pulse circuit is used to convert the high and low levels into pulses of a certain frequency; the counter is used to count the number of pulse outputs of the linear neuron circuit for subsequent data processing.

[0022] When the linear multiplication and addition circuit is working, the grayscale information of the image is preprocessed in the digital processing module, and then transmitted to the register and converted into grayscale information data. The data stream is converted into the square wave input required by the neuron through the pulse generator. The grayscale information square wave is input to the linear neuron pulse circuit. The output of the pulse circuit is a pulse with a specific frequency per unit time. The pulse frequency is related to the number of high-level inputs in the same time.

[0023] The digital processing module includes an erasable memory SRAM and a digital preprocessing unit. The erasable memory SRAM is used to store grayscale data, and the digital preprocessing unit is used to convert grayscale values ​​into multi-bit data. The linear neuron pulse circuit can adopt the linear neuron pulse circuit disclosed in patent CN118228787A, but the pulse width of the circuit output is not stable and is not easy to control. This embodiment improves the circuit structure of the patent by adding a buffer Buffer (such as Figure 3The pulse width can be controlled by voltage, which is convenient for the counter to count. The buffer inputs the pulse into the counter, which counts the number of pulses per unit time and shifts the data bit according to the current timing.

[0024] The digital processing module needs to store the image grayscale information in a matrix, and also process multiple data in parallel during subsequent processing. When processing different image information, the data stored in the memory SRAM can be erased. The digital processing module converts the grayscale value of the image into grayscale information data, and performs preprocessing according to the needs of the subsequent circuit. The preprocessing includes scaling or matrix multiplication operations. The grayscale information data is preprocessed by the digital processing module and transmitted to the register for sorting. The register outputs the grayscale information data to the pulse generator, converts the grayscale information data into a square wave with specific grayscale information, and outputs it to the linear neuron pulse circuit. When processing image information, the circuit generally works in an array form, and multiple identical circuits process the image grayscale matrix information in parallel. In addition to receiving data processed by the circuit in this chip, the linear neuron pulse circuit also receives data from other chips in the same array, with a total of up to nine data. Multiple grayscale information square waves are simultaneously transmitted to the linear neuron pulse circuit. The linear neuron pulse circuit generates a pulse output according to the number of high-level inputs at the same time, and the frequency of the pulse output is linearly positively correlated with the number of high-level inputs. The linear neuron pulse circuit generates pulse output by charging and discharging capacitors. When a high level is input, the capacitor starts to charge. When the threshold is reached, the charge is discharged to generate a pulse. The more high levels are input, the higher the charge and discharge frequency, and the higher the corresponding pulse frequency.

[0025] In addition, digital logic devices such as digital processing modules and counters are not on the same process layer as analog modules such as registers, pulse generators, and linear neuron pulse circuits. The digital logic devices are connected to the analog modules through through holes.

[0026] The above linear multiplication and addition circuits are integrated on a chip, and the input and output ports are specified through a specific layout design method, such as Figure 3As shown, In0~In3, In5~In8, Out0~Out3, Out5~Out8 are respectively placed around and at the corners of the package shell to receive off-chip data and output on-chip data. Since the linear neuron pulse circuit of this embodiment receives nine signals, in addition to transmitting the data processed by the on-chip linear multiplication and addition circuit on the chip, it is also necessary to receive eight additional input data from outside the chip. Among them, the processing results of this chip are transmitted on the chip. The other eight signals are input through the input ports on the four sides and corners of the chip. Similarly, if used in an array, the output results of the pulse generator in this chip also need to be transmitted to other chips in the same array. The output method is similar to the input of the linear neuron pulse circuit, and both are output through the output ports on the four sides and corners of the chip. The layout is performed on the chip, and the input pins of the linear neuron pulse circuit are dragged to the input ports In0~In3, In5~In8 with a metal layer, and In4 is the on-chip data. The output of the pulse generator is simultaneously transmitted to the on-chip linear neuron pulse circuit input and eight output ports Out0~Out3, Out5~Out8. Connect the external input and output directly to the chip. The analog layout can realize the above input and output through on-chip wiring, and the single chip is completely and independently packaged. In order to facilitate the interconnection between chip arrays, the four corners of the chip after packaging are designed to be concave.

[0027] Example 2

[0028] like Figure 4 In the case of array use, multiple independently packaged chips can be directly connected through the input and output ports at the concave corners, and wires are used between the chips. The connection between multiple chips does not require additional analog layout design and digital back-end loss. In this way, the array can flexibly adjust the array size according to actual needs. When processing the grayscale information of images of different sizes, the computing pressure of the subsequent digital network can be reduced. In addition, the multi-layer design of digital modules and analog modules reduces the area of ​​a single chip.

Claims

1. A linear multiplication and addition circuit for image processing, characterized in that: The linear multiplication and addition circuit comprises a digital processing module, a register, a pulse generator, a linear neuron pulse circuit and a counter which are connected in sequence; the digital processing module is used to convert the image grayscale absolute value into multi-bit data; the register is used to sort and cache the multi-bit data according to the number of bits; The pulse generator is used to convert multi-bit data into high and low level signals; the linear neuron pulse circuit is used to convert high and low levels into pulses of a certain frequency; and the counter is used to count the number of pulse outputs of the linear neuron circuit.

2. A linear multiplication and addition circuit for image processing according to claim 1, characterized in that: The digital processing module includes an erasable memory SRAM and a digital preprocessing unit. The erasable memory SRAM is used to store grayscale data, and the digital preprocessing unit is used to convert grayscale values ​​into multi-bit data.

3. The linear multiplication and addition circuit for image processing according to claim 1, characterized in that: The output end of the linear neuron pulse circuit is connected to a buffer, the buffer output is connected to the counter input, and the counter counts the number of pulses output by the buffer.

4. A method for implementing a linear multiplication-addition circuit for image processing according to claim 1, characterized in that: The method includes: the digital processing module performs preprocessing operations on the grayscale information of the image, and then transmits the preprocessed grayscale information to the register and converts it into grayscale information data; the grayscale information data is then converted into a square wave required by the neuron through a pulse generator, and then input into a linear neuron pulse circuit; the linear neuron pulse circuit outputs pulses with a certain frequency, and the number of output pulses is counted by a counter.

5. The implementation method according to claim 4, characterized in that: The digital processing module stores the image grayscale information in a matrix and processes multiple channels of data in parallel.

6. The implementation method according to claim 4, characterized in that: The pre-processing operations of the digital processing module include scaling or matrix multiplication operations.

7. The implementation method according to claim 4, characterized in that: The linear neuron pulse circuit receives square waves of grayscale information of up to nine channels at the same time. The linear neuron pulse circuit is effective at high level and generates pulse output according to the number of high level inputs at the same time. The frequency of the pulse output is linearly positively correlated with the number of high level inputs.

8. A chip using a linear multiplication and addition circuit for image processing as claimed in claim 1, characterized in that: The chip integrates the linear multiplication and addition circuit, and includes multiple input ports and multiple output ports, which are respectively arranged around and at the corners of the chip packaging shell, and are used to receive off-chip data and output on-chip data; wherein, the processing results of the linear multiplication and addition circuit are transmitted on the chip, and the signals input to the linear neuron pulse circuit from outside the chip are input through the input ports around and at the corners of the chip.

9. The chip according to claim 8, characterized in that: The four corners of the chip are designed to be concave.

10. The chip according to claim 9, characterized in that: The chips in multiple independent packages are directly connected through input and output ports at the concave corners, and wires are used to connect the chips.

Images