Convolution calculation unit and sensor internal processing circuit

Abstract

The invention relates to a convolution calculation unit and a sensor internal processing circuit, and the convolution calculation unit comprises a Gilbert multiplication circuit which is used for carrying out the voltage multiplication of a differential voltage of a processed image pixel value and a differential voltage of a weight of a convolution kernel; and the switched capacitor circuit is used for carrying out weighted calculation on the voltage multiplication result. The output bandwidth of the Gilbert multiplier is optimized on the basis of the Gilbert multiplier, the stability and the accuracy of the system are improved, the power consumption of the system can be further reduced by adopting the switched capacitor circuit to carry out weighting operation on the signal, the work and the turn-off of the system can be controlled by utilizing a time signal, and the power consumption of the system is reduced. And the system hardly consumes energy in a static state.

Description

Technical Field

[0001] The present invention relates to the technical field of circuit design, and in particular to a convolution calculation unit and a processing circuit in a sensor. Background Art

[0002] System latency, power consumption, and privacy issues are three major constraints hindering the further development and wider application of traditional machine vision systems and related technologies. In recent years, research on low-power, energy-efficient CMOS image sensors (CIS) has advanced and is widely used in edge devices. Unlike traditional vision systems, which have independent perception, memory, and processing architectures, in-sensor visual computing technology integrates sensing, storage, and computation into a sensor-storage-computing architecture. This architecture directly processes collected analog signals at the pixel level, significantly improving image processing speed and significantly reducing power consumption. A growing number of researchers are exploring in-sensor processing technologies to achieve low-power, real-time image processing. Several sensor circuit architectures based on in-sensor processing have emerged. Vision sensors extract key information from raw analog signals and pre-process images by simulating the functions of network layers in convolutional neural networks. In image processing methods using in-sensor processing circuits, circuit structure and complexity are key factors affecting power consumption and efficiency. While in-sensor processing based on static random access memory (SRAM) offers high processing speeds, its circuit design faces significant area overhead and power consumption. Furthermore, using SRAM for in-memory computing requires the circuit to be constantly charged, making it unsuitable for real-time image processing in edge devices. In addition, the method of using analog multiply-accumulate operations that share charge with the capacitor array will lead to problems such as gain loss, low weight resolution and limited accuracy, which is not conducive to subsequent signal processing. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a convolution calculation unit and a processing circuit in a sensor, which can save computing resources, reduce energy consumption, and improve the real-time performance of the entire system.

[0004] The technical solution adopted by the present invention to solve the technical problem is to provide a convolution calculation unit, including:

[0005] a Gilbert multiplication circuit for performing voltage multiplication on a differential voltage of a processed image pixel value and a differential voltage of a convolution kernel weight;

[0006] The switched capacitor circuit is used to perform weighted calculation on the result of multiplying the voltages.

[0007] The Gilbert multiplication circuit includes a first MOS transistor, a second MOS transistor, a third MOS transistor, a fourth MOS transistor, a fifth MOS transistor, a sixth MOS transistor, a seventh MOS transistor, an eighth MOS transistor and a ninth MOS transistor;

[0008] The gate terminal of the first MOS transistor is connected to the first voltage signal terminal corresponding to the processed image pixel value, the source terminal is connected to the drain terminal of the ninth MOS transistor, and the drain terminal is connected to the source terminal of the third MOS transistor;

[0009] The gate terminal of the second MOS transistor is connected to the second voltage signal terminal corresponding to the processed image pixel value, the source terminal is connected to the drain terminal of the ninth MOS transistor, and the drain terminal is connected to the source terminal of the fifth MOS transistor;

[0010] The gate terminal of the third MOS transistor is connected to the first voltage signal terminal corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the first MOS transistor, and the drain terminal is connected to the source terminal of the seventh MOS transistor;

[0011] The gate terminal of the fourth MOS transistor is connected to the second voltage signal terminal corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the first MOS transistor, and the drain terminal is connected to the drain terminal of the sixth MOS transistor;

[0012] The gate terminal of the fifth MOS transistor is connected to the second voltage signal terminal corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the second MOS transistor, and the drain terminal is connected to the drain terminal of the third MOS transistor;

[0013] The gate terminal of the sixth MOS transistor is connected to the first voltage signal terminal corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the second MOS transistor, and the drain terminal is connected to the drain terminal of the eighth MOS transistor;

[0014] The gate terminal of the seventh MOS transistor is connected to the gate terminal of the eighth MOS transistor, the drain terminal of the seventh MOS transistor is connected to the drain terminal of the eighth MOS transistor, and the source terminal of the seventh MOS transistor and the source terminal of the eighth MOS transistor serve as the output terminal of the Gilbert multiplication circuit; the gate terminal of the seventh MOS transistor is also connected to the drain terminal of the seventh MOS transistor;

[0015] The gate end of the ninth MOS transistor is connected to the control signal end, and the source end is grounded.

[0016] The switched capacitor circuit includes an operational amplifier and a complementary CMOS switch circuit connected to each other.

[0017] The operational amplifier adopts a folded cascode structure, and the folded cascode amplifier adopts PMOS as an input transistor.

[0018] The technical solution adopted by the present invention to solve the technical problem is to provide a sensor processing circuit, including:

[0019] A voltage conversion circuit, configured to convert image data into a first voltage signal;

[0020] A voltage simulation circuit, used for simulating the convolution kernel weight into a second voltage signal;

[0021] a plurality of convolution calculation units according to any one of claims 1 to 4, the convolution calculation units being configured to perform a multiplication and accumulation calculation on the first voltage signal and the second voltage signal to obtain a convolution result of the convolution kernel and the image data;

[0022] The data storage matrix is ​​used to store the convolution result.

[0023] Beneficial effects

[0024] Due to the adoption of the above-mentioned technical solution, the present invention has the following advantages and positive effects compared with the prior art: the convolution calculation unit of the present invention is composed of a Gilbert multiplier and a switched capacitor filter (SCF) coupled together, and uses a voltage signal to simulate the weight of the convolution kernel and the pixel value of the image to realize the convolution calculation. This method of using all hardware to realize the convolution calculation saves a lot of computing resources, greatly reduces energy consumption, and improves the real-time performance of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a general framework diagram of the processing circuit in the sensor according to an embodiment of the present invention;

[0026] Figure 2 Schematic diagram of the structure of a convolution calculation unit in an embodiment of the present invention;

[0027] Figure 3 is a schematic structural diagram of a Gilbert multiplication circuit according to an embodiment of the present invention;

[0028] Figure 4 is a graph showing output characteristic curves of the Gilbert multiplication circuit under different AC input signals in an embodiment of the present invention;

[0029] Figure 5 1 is a comparison diagram of the results of single-ended input and double-ended input of the switched capacitor circuit in an embodiment of the present invention;

[0030] Figure 6 This is a processing circuit layout in the sensor according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the appended claims of the application.

[0032] Embodiments of the present invention relate to an in-sensor processing circuit for convolution calculations, such as Figure 1 As shown, including:

[0033] A voltage conversion circuit, configured to convert image data into a first voltage signal;

[0034] A voltage simulation circuit, used for simulating the convolution kernel weight into a second voltage signal;

[0035] a plurality of convolution calculation units, each of which is configured to perform a multiplication and accumulation calculation on the first voltage signal and the second voltage signal to obtain a convolution result of the convolution kernel and the image data;

[0036] The data storage matrix is ​​used to store the convolution result.

[0037] Taking a 3*3 convolution kernel as an example, after image data is input to the device, it is converted into a first voltage signal by a voltage conversion circuit. The convolution kernel weight is then simulated into a second voltage signal by a voltage simulation circuit. The first and second voltage signals are then input into the convolution calculation unit for multiplication and accumulation, resulting in the convolution result of the convolution kernel and the image data. The data storage matrix stores the convolution result. This all-hardware approach to convolution calculations saves a significant amount of computing resources, significantly reduces energy consumption, and improves the real-time performance of the entire system.

[0038] like Figure 2 As shown, the convolution calculation unit in this embodiment includes:

[0039] a Gilbert multiplication circuit for performing voltage multiplication on a differential voltage of a processed image pixel value and a differential voltage of a convolution kernel weight;

[0040] The switched capacitor circuit is used to perform weighted calculation on the result of multiplying the voltages.

[0041] The differential voltage of the processed image pixel value and the differential voltage of the convolution kernel weight are input as two pairs of differential signals by the two differential input terminals of the Gilbert multiplier respectively. The results of the multiplication of the two sets of differential voltages are input into the switched capacitor circuit for weighted calculation. The output of each convolution calculation unit is the result of the convolution of the 3*3 convolution kernel and the image.

[0042] like Figure 3 As shown, the Gilbert multiplication circuit in this embodiment includes a first MOS transistor Q1, a second MOS transistor Q2, a third MOS transistor Q3, a fourth MOS transistor Q4, a fifth MOS transistor Q5, a sixth MOS transistor Q6, a seventh MOS transistor Q7, an eighth MOS transistor Q8 and a ninth MOS transistor Q9.

[0043] The gate terminal of the first MOS transistor Q1 is connected to the first voltage signal terminal V3 corresponding to the processed image pixel value, the source terminal is connected to the drain terminal of the ninth MOS transistor Q9, and the drain terminal is connected to the source terminal of the third MOS transistor Q3;

[0044] The gate terminal of the second MOS transistor Q2 is connected to the second voltage signal terminal V4 corresponding to the processed image pixel value, the source terminal is connected to the drain terminal of the ninth MOS transistor Q9, and the drain terminal is connected to the source terminal of the fifth MOS transistor Q5;

[0045] The gate terminal of the third MOS transistor Q3 is connected to the first voltage signal terminal VIN corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the first MOS transistor Q1, and the drain terminal is connected to the source terminal of the seventh MOS transistor Q7;

[0046] The gate terminal of the fourth MOS transistor Q4 is connected to the second voltage signal terminal VIP corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the first MOS transistor Q1, and the drain terminal is connected to the drain terminal of the sixth MOS transistor Q6;

[0047] The gate terminal of the fifth MOS transistor Q5 is connected to the second voltage signal terminal VIP corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the second MOS transistor Q2, and the drain terminal is connected to the drain terminal of the third MOS transistor Q3;

[0048] The gate terminal of the sixth MOS transistor Q6 is connected to the first voltage signal terminal VIN corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the second MOS transistor Q2, and the drain terminal is connected to the drain terminal of the eighth MOS transistor Q8;

[0049] The gate terminal of the seventh MOS transistor Q7 is connected to the gate terminal of the eighth MOS transistor Q8, the drain terminal of the seventh MOS transistor Q7 is connected to the drain terminal of the eighth MOS transistor Q8, the source terminal of the seventh MOS transistor Q7 and the source terminal of the eighth MOS transistor Q8 serve as the output terminal of the Gilbert multiplication circuit; the gate terminal of the seventh MOS transistor Q7 is also connected to the drain terminal of the seventh MOS transistor Q7;

[0050] The gate terminal of the ninth MOS transistor Q9 is connected to the control signal terminal V5, and the source terminal is grounded.

[0051] The Gilbert multiplication circuit in this embodiment adopts a Gilbert mixing structure, which includes: three differential pairs, namely the first MOS transistor Q1 - the second MOS transistor Q2, the third MOS transistor Q3 - the fourth MOS transistor Q4, the fifth MOS transistor Q5 - the sixth MOS transistor Q6, and two diode-connected device loads, namely the seventh MOS transistor Q7 - the eighth MOS transistor Q8 and a current source, namely the ninth MOS transistor Q9. This Gilbert multiplication circuit has two differential inputs (i.e., V3 - V4 and VIN - VIP) and an output. The current value provided by the ninth MOS transistor Q9 serving as the current source is determined by the bias voltage V5 at its gate terminal, and this bias voltage also indirectly affects the overall gain of the circuit. At the same time, this bias voltage also controls the on or off of the Gilbert multiplication circuit, improving the resource utilization rate of the multiplier. For symmetry reasons, the three differential pairs and the two loads have the same size. The seventh MOS transistor Q7 and the eighth MOS transistor Q8 are connected in a gate-drain manner as loads because when Vgs - Vth < Vds, the MOS transistor operates in the saturation region, and the gate-drain connection can meet this condition, thus ensuring that the two MOS transistors always operate in the saturation region. When the seventh MOS transistor Q7 and the eighth MOS transistor Q8 are NMOS transistors, it can also increase the output impedance of the entire circuit, further improving the gain of the circuit, which plays an important role in enhancing the performance and stability of the circuit. VIN - VIP and V3 - V4 are two pairs of differential input voltages, the gate terminal voltage V5 of the ninth MOS transistor Q9 is the control signal, and the two differential output terminals are respectively the source terminals of the seventh MOS transistor Q7 and the eighth MOS transistor Q8. The final output voltage expression is V o = V Ooffset + K * (VIN - VIP) * (V3 - V4). Where, V Ooffset is the single-ended output DC level, V o is the single-ended output voltage, and K is the gain coefficient of the multiplier. Keeping the voltage difference between V3 - V4 at 0.2V unchanged, AC signals of 20mV and 40mV are respectively input to VIN - VIP of the Gilbert multiplication circuit to test the performance of the Gilbert multiplication circuit, and the AC output curve of the Gilbert multiplication circuit is obtained as shown in Figure 4 shown. The output voltage is halved, which well realizes the performance of multiplication and has a relatively large output bandwidth.

[0052] It can be seen that the convolution calculation unit of this embodiment optimizes the output bandwidth of the multiplier based on the Gilbert multiplication circuit, improving the stability and accuracy of the system.

[0053] The switched capacitor circuit in this embodiment includes an interconnected operational amplifier and a complementary CMOS switch circuit. The operational amplifier uses a folded cascode structure. The folded cascode structure has smaller input and output common-mode level restrictions and can provide a wider input common-mode range compared to a telescopic cascode. In this embodiment, the folded cascode amplifier uses PMOS as the input transistor, which can avoid the substrate bias effect and has stronger noise interference resistance. The input transistor plays the role of converting the input voltage into current.

[0054] The switched capacitor circuit is divided into two stages: sampling and amplification. In the sampling stage, the input signal is sampled and stored on the capacitor through the switch. In the amplification stage, the voltage sampled on the capacitor is transferred to the output and may be amplified by an operational amplifier. The switch uses a transmission gate circuit, which has the characteristics of flexibility and low power consumption. By controlling the opening and closing of the switch, the circuit connection mode is adjusted to change the working state of the circuit. The output of the Gilbert multiplication circuit is input into the switched capacitor circuit in single-ended and double-ended forms for testing. The test results are shown as follows: Figure 5 As shown, the final output of the switched capacitor circuit is a step signal and can achieve weighting of the output of the Gilbert multiplication circuit.

[0055] This embodiment uses a switched capacitor circuit to perform a weighted operation on the signal, which can further reduce the power consumption of the system, and can use the time signal to control the operation and shutdown of the system, so that the system consumes almost no energy when static.

[0056] Figure 6 This is a schematic diagram of the in-sensor processing circuitry of an embodiment of the present invention. Compared to other in-sensor processing circuits, the convolution calculation unit designed in this embodiment utilizes fewer circuit components, effectively implementing convolution functionality and reducing resource consumption between the storage and computing centers, making it suitable for more complex application scenarios.

Claims

1. A convolution calculation unit, characterized in that: include: a Gilbert multiplication circuit for performing voltage multiplication on a differential voltage of a processed image pixel value and a differential voltage of a convolution kernel weight; The switched capacitor circuit is used to perform weighted calculation on the result of multiplying the voltages.

2. The convolution calculation unit according to claim 1, characterized in that The Gilbert multiplication circuit includes a first MOS transistor, a second MOS transistor, a third MOS transistor, a fourth MOS transistor, a fifth MOS transistor, a sixth MOS transistor, a seventh MOS transistor, an eighth MOS transistor and a ninth MOS transistor; The gate terminal of the first MOS transistor is connected to the first voltage signal terminal corresponding to the processed image pixel value, the source terminal is connected to the drain terminal of the ninth MOS transistor, and the drain terminal is connected to the source terminal of the third MOS transistor; The gate terminal of the second MOS transistor is connected to the second voltage signal terminal corresponding to the processed image pixel value, the source terminal is connected to the drain terminal of the ninth MOS transistor, and the drain terminal is connected to the source terminal of the fifth MOS transistor; The gate terminal of the third MOS transistor is connected to the first voltage signal terminal corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the first MOS transistor, and the drain terminal is connected to the source terminal of the seventh MOS transistor; The gate terminal of the fourth MOS transistor is connected to the second voltage signal terminal corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the first MOS transistor, and the drain terminal is connected to the drain terminal of the sixth MOS transistor; The gate terminal of the fifth MOS transistor is connected to the second voltage signal terminal corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the second MOS transistor, and the drain terminal is connected to the drain terminal of the third MOS transistor; The gate terminal of the sixth MOS transistor is connected to the first voltage signal terminal corresponding to the weight of the convolution kernel, the source terminal is connected to the drain terminal of the second MOS transistor, and the drain terminal is connected to the drain terminal of the eighth MOS transistor; The gate terminal of the seventh MOS transistor is connected to the gate terminal of the eighth MOS transistor, the drain terminal of the seventh MOS transistor is connected to the drain terminal of the eighth MOS transistor, and the source terminal of the seventh MOS transistor and the source terminal of the eighth MOS transistor serve as the output terminal of the Gilbert multiplication circuit; the gate terminal of the seventh MOS transistor is also connected to the drain terminal of the seventh MOS transistor; The gate end of the ninth MOS transistor is connected to the control signal end, and the source end is grounded.

3. The convolution calculation unit according to claim 1, wherein The switched capacitor circuit includes an operational amplifier and a complementary CMOS switch circuit connected to each other.

4. The convolution calculation unit according to claim 3, characterized in that The operational amplifier adopts a folded cascode structure, and the folded cascode amplifier adopts PMOS as an input transistor.

5. A processing circuit in a sensor, characterized in that: include: A voltage conversion circuit, configured to convert image data into a first voltage signal; A voltage simulation circuit, used for simulating the convolution kernel weight into a second voltage signal; a plurality of convolution calculation units according to any one of claims 1 to 4, the convolution calculation units being configured to perform a multiplication and accumulation calculation on the first voltage signal and the second voltage signal to obtain a convolution result of the convolution kernel and the image data; The data storage matrix is ​​used to store the convolution result.

Images