Linear multiply-add circuit for image processing and implementation method and chip thereof

Abstract

The application discloses a linear multiplication-addition circuit for image processing and an implementation method and a chip thereof. The linear multiplication-addition circuit comprises a digital processing module, a register, a pulse generator, a linear neuron pulse circuit and a counter which are connected in sequence. The digital processing module is used for converting the absolute value of image gray scale into multi-bit data. The register is used for arranging and buffering the multi-bit data by bit number. The pulse generator is used for converting the multi-bit data into high-low level signals. The linear neuron pulse circuit is used for converting the high-low level signals into pulses with a certain frequency. The counter is used for counting the number of pulse outputs of the linear neuron circuit. The linear multiplication-addition circuit is adopted, so that large-scale circuits such as a multiplier and an adder array are avoided, high-speed linear operation of image gray scale information can be realized, and the power consumption of the circuit is greatly reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to a linear multiplication and addition circuit for image processing and its implementation method and chip, belonging to the field of integrated circuits. BACKGROUND

[0002] The convolution multiplication and addition calculation chip of the traditional convolutional neural network, such as GPU, FPGA, NPU, etc., needs to use digital modules such as multipliers and adders to realize linear operations such as convolution and full connection, which greatly limits the speed of the circuit and increases the power consumption.

[0003] In order to solve the problem of memory wall caused by the separation of calculation and storage units in the traditional Von Neumann architecture, including long delay, high energy consumption, etc. The memory and calculation integrated structure emerges as the times require. The memory and calculation integrated architecture integrates the calculation unit and the storage unit together, or moves the calculation function to the inside of the memory, reducing the frequent transfer of data between the calculation unit and the storage unit. This greatly reduces the time and energy consumption of data transmission. Since data can be processed directly within the memory, the memory and calculation integrated architecture can support high-throughput data processing, which is a significant advantage for big data and artificial intelligence applications. The common memory and calculation integrated implementation methods at present include: memory and calculation integrated based on Flash, memory and calculation integrated based on RRAM, SRAM, etc., but there is still a problem of redundant data volume and calculation volume. In addition, the traditional linear multiplication and addition array chip needs to specify the size of the array according to the use scene at the beginning of chip design, and the connection between chips is directly completed through analog layout design or digital back-end design, which increases the loss of analog layout and digital back-end. SUMMARY

[0004] In view of the problems existing in the prior art, the present application provides a linear multiplication and addition circuit for image processing and its implementation method and chip, which can realize linear multiplication and addition operation of image gray scale information and avoid large-scale circuits such as multiplier and adder array.

[0005] The technical scheme adopted by the circuit of the present application is as follows:

[0006] A linear multiplication and addition circuit for image processing, comprising a digital processing module, a register, a pulse generator, a linear neuron pulse circuit and a counter connected in sequence; the digital processing module is used for converting the absolute value of image gray scale into multi-bit data; the register is used for sorting and buffering the multi-bit data by bit number; the pulse generator is used for converting the multi-bit data into high-low level signals; the linear neuron pulse circuit is used for converting the high-low level signals into pulses of a certain frequency; and the counter is used for counting the number of pulse outputs of the linear neuron circuit.

[0007] The application further provides an implementation method of a linear multiply-add circuit for image processing, which comprises: the digital processing module performs a preprocessing operation on the gray information of an image, and then transmits the preprocessed gray information to a register to be converted into gray information data; the gray information data is converted into a square wave required by a neuron by a pulse generator, and then input to a linear neuron pulse circuit; the linear neuron pulse circuit outputs pulses with a certain frequency, and the number of the output pulses is counted by a counter.

[0008] The application further provides a chip of a linear multiply-add circuit for image processing, which is integrated with the linear multiply-add circuit, and comprises a plurality of input ports and a plurality of output ports, which are respectively arranged at the four sides and corners of a chip package shell to receive off-chip data and output on-chip data; wherein the processing results of the linear multiply-add circuit are transmitted on-chip, and the signals of the linear neuron pulse circuit are input through the input ports at the four sides and corners of the chip.

[0009] The application has the following beneficial effects:

[0010] (1) Compared with a traditional convolution multiply-add calculation chip, the linear multiply-add circuit can realize high-speed linear operation of image gray information, and greatly reduces the power consumption of the circuit.

[0011] (2) The data storage and calculation integrated structure based on SRAM is adopted, the data is stored in the SRAM, and the calculation is performed on-chip, which can greatly reduce the amount of redundant data and calculation, and reduce the storage and calculation pressure.

[0012] (3) The circuit is composed of an array and simultaneously processes the gray information of the image in a matrix range and subsequent operations, that is, the eigenvalues of the gray information of the image in a certain matrix range are extracted, and the convolution calculation of the gray matrix is actually completed.

[0013] (4) The chip module design and connection mode can completely separate a single chip. In the array application scenario, the array can be combined by plug-in according to the actual array size, which reduces the loss of analog layout and digital back end, and increases the flexibility of the array size. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description only some embodiments of the application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0015] Figure 1is a block diagram of the linear multiply-add circuit of the present application;

[0016] Figure 2 is a block diagram of the digital logic device used in the present application;

[0017] Figure 3 is a schematic diagram of the packaged single chip of embodiment 1 of the present application;

[0018] Figure 4 is a schematic diagram of the chip array combination of embodiment 2 of the present application. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0020] Embodiment 1

[0021] As shown in Figure 1 , Figure 2 , the linear multiply-add circuit for image processing of the present embodiment includes, in sequence, a digital processing module, a register, a pulse generator, a linear neuron pulse circuit and a counter. The digital processing module is used to convert the absolute value of the image gray scale into multi-bit data; the register is used to sort and buffer the multi-bit data by bit number; the pulse generator is used to convert the multi-bit data into high-low level signals; the linear neuron pulse circuit is used to convert the high-low level into pulses of a certain frequency; and the counter is used to count the number of pulse outputs of the linear neuron circuit, facilitating subsequent data processing.

[0022] When the linear multiply-add circuit is working, the gray scale information of the image is preprocessed in the digital processing module and then transmitted to the register to be converted into gray scale information data. The data stream is converted into a square wave input required by the neuron through the pulse generator. The gray scale information square wave input is given to the linear neuron pulse circuit, and the output of the pulse circuit is a pulse with a specific frequency in a unit of time, and the pulse frequency is related to the number of high levels input in the same time.

[0023] The digital processing module includes an erasable memory SRAM and a digital preprocessing unit. The erasable memory SRAM is used to store gray scale data, and the digital preprocessing unit is used to convert the gray scale value into multi-bit data. The linear neuron pulse circuit can use the linear neuron pulse circuit disclosed in the patent CN118228787A, but the pulse width of the circuit output is not stable and is not convenient to control. The circuit structure of the patent is improved in the present embodiment, and a buffer Buffer is added at the output end of the circuit (as shown in Figure 3The pulse width can be controlled by voltage, and the counter can count. The buffer inputs the pulse into the counter, and the counter calculates the number of pulses in a unit time, and shifts according to the number of data bits specified by the current timing.

[0024] The digital processing module needs to store the image gray scale information in a matrix, and the subsequent processing is also parallel processing of multiple data. When processing different image information, the data stored in the internal memory SRAM can be erased. The digital processing module converts the gray scale value of the image into gray scale information data, and makes preprocessing according to the needs of the subsequent circuit. The preprocessing includes scaling or matrix multiplication operation. The gray scale information data is transmitted to the register after being preprocessed by the digital processing module. The register outputs the gray scale information data to the pulse generator, converts the gray scale information data into a square wave with specific gray scale information, and outputs it to the linear neuron pulse circuit. When processing image information, the circuit is generally in array form. Multiple identical circuits process image gray scale matrix information in parallel. The linear neuron pulse circuit receives data from other chip circuits in the same array in addition to the data processed by the internal circuit. There are up to nine channels of data. Multiple gray scale information square waves are transmitted to the linear neuron pulse circuit at the same time. The linear neuron pulse circuit generates pulse output according to the number of high-level inputs at the same time. The frequency of the pulse output is positively correlated with the number of input high levels. The linear neuron pulse circuit generates pulse output through the charging and discharging of the capacitor. When the input high level is input, the capacitor starts to charge, and after reaching the threshold, the charge is discharged to generate a pulse. The more the number of input high levels, the higher the charging and discharging frequency, and the higher the corresponding pulse frequency.

[0025] In addition, the digital logic devices such as digital processing module and counter are not in the same process layer as the analog modules such as register, pulse generator and linear neuron pulse circuit. The digital logic devices are connected with the analog modules through through holes.

[0026] The above linear multiplication and addition circuit is integrated on a chip, and the input and output ports are specified by a specific layout design method, such as Figure 3As shown, In0-In3, In5-In8, Out0-Out3, Out5-Out8 are respectively arranged at the four corners of the package shell, for receiving off-chip data and outputting on-chip data. Since the linear neuron pulse circuit in the embodiment receives nine signals, in addition to the data processed by the on-chip linear multiply-add circuit, eight additional input data need to be received from off-chip. Among them, the on-chip transmits the processing results of the chip. The other eight signals are input through the input ports at the four corners of the chip. Similarly, if used in an array, the output results of the pulse generator in the chip also need to be transmitted to other chips in the array. The output method is similar to the input of the linear neuron pulse circuit, that is, the output is performed through the output ports at the four corners of the chip. In the layout in the chip, the input pins of the linear neuron pulse circuit are connected to the input ports In0-In3, In5-In8 by metal layers, and In4 is the on-chip data. The output of the pulse generator is transmitted to the input of the linear neuron pulse circuit in the chip and the eight output ports Out0-Out3, Out5-Out8. The off-chip input and output are directly connected to the on-chip. The above input and output methods can be realized by analog layout through on-chip wiring, and a single chip can be completely and independently packaged. In order to facilitate the interconnection between the chip arrays, the four corners of the packaged chip are designed to be concave.

[0027] Embodiment 2

[0028] As Figure 4 In the array use case, multiple independently packaged chips can directly use wires to connect between chips through the input and output ports at the concave corners. The connection between multiple chips does not require additional analog layout design and digital backend consumption. In this way, the array can be flexibly adjusted in size according to actual needs. When processing gray scale information of different sizes of images, the operation pressure of the subsequent digital network can be reduced. In addition, the multi-layer design of the digital module and the analog module reduces the area of a single chip.

Claims

1. A method for implementing a linear multiply-accumulate circuit for image processing, characterized in that, Its linear multiply-accumulate circuit includes a digital processing module, a register, a pulse generator, a linear neuron pulse circuit, and a counter connected in sequence; the digital processing module is used to convert the absolute value of the image grayscale into multi-bit data; the register is used to organize and cache the multi-bit data by bit. The pulse generator is used to convert multi-bit data into high and low level signals; the linear neuron pulse circuit is used to convert high and low levels into pulses of a certain frequency. The counter is used to calculate the number of pulse outputs of the linear neuron circuit; The output of the linear neuron pulse circuit is connected to a buffer, and the output of the buffer is connected to the input of the counter. The counter calculates the number of pulses output by the buffer. The implementation method includes: the digital processing module performs preprocessing operations on the grayscale information of the image, including scaling or matrix multiplication; then the preprocessed grayscale information is transmitted to a register and converted into grayscale data; the grayscale data is then converted into square waves required by the neuron by a pulse generator and input to the linear neuron pulse circuit; the linear neuron pulse circuit outputs pulses with a certain frequency, and the number of output pulses is counted by a counter; wherein, the linear neuron pulse circuit simultaneously receives square waves of grayscale information from up to nine channels, the linear neuron pulse circuit is active high level, and generates pulse output according to the number of high-level inputs at the same time, and the frequency of the pulse output is linearly positively correlated with the number of high-level inputs.

2. The implementation method according to claim 1, characterized in that, The digital processing module includes an erasable memory (SRAM) and a digital preprocessing unit. The SRAM is used to store grayscale data, and the digital preprocessing unit is used to convert grayscale values ​​into multi-bit data.

3. The implementation method according to claim 1, characterized in that, The digital processing module stores the image grayscale information in a matrix and processes multiple data streams in parallel.

4. A chip for a linear multiply-accumulate circuit for image processing, characterized in that, The chip integrates the linear multiply-accumulate circuit as described in claim 1. The chip includes multiple input ports and multiple output ports, which are respectively located around the perimeter and corners of the chip package shell for receiving external data and outputting internal data. The processing results of the linear multiply-accumulate circuit are transmitted internally, and the signals of the linear neuron pulse circuit input externally are input through the input ports around the perimeter and corners of the chip.

5. The chip according to claim 4, characterized in that, The four corners of the chip are designed to be concave.

6. The chip according to claim 5, characterized in that, Multiple independently packaged chips are directly connected to each other using wires through input / output ports located at recessed corners.

Images