CMOS image sensing integrated circuit based on 5t, 8t, 14t hybrid pixel calculation unit

Abstract

The present disclosure provides a CMOS image sensing integrated circuit based on a 5T, 8T, 14T hybrid pixel calculation unit, comprising: a photoelectric sensing array, a weight voltage generation module, a readout circuit and a control module; the photoelectric sensing array comprises a plurality of groups of hybrid pixel calculation units and a plurality of row selection switches, the types of pixel calculation units in the hybrid pixel calculation units include 5T, 8T and 14T, the photoelectric sensing array performs convolution multiplication and accumulation operation on the weight voltage of the convolution kernel through the pixel calculation unit, and outputs current; the weight voltage generation module is used for generating the weight voltage corresponding to the convolution kernel; the readout circuit is used for subtraction operation, current-voltage conversion and correlated double sampling; and the control module is used for generating a control signal. Through the present disclosure, the multiplication and accumulation operation in the sensing and convolution algorithm is automatically realized on the chip, parallel calculation and output are realized in the convolution kernel region, the weight does not need to be rewritten in the overlapping region of adjacent convolution kernels, the circuit power consumption is reduced, and the frame rate and operation efficiency are improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of integrated circuits, in particular, to a CMOS image sensing and computing integrated circuit based on a 5T, 8T and 14T hybrid pixel computing unit. BACKGROUND

[0002] A CMOS image sensor (CIS) is the core of a camera module, which can efficiently convert captured light signals into electrical signals, and plays an indispensable role in many industries such as automation, intelligent monitoring, medical diagnosis, etc. On the other hand, convolutional computation greatly simplifies the model structure with its characteristics of local connection, weight sharing and dimension reduction output, thereby effectively improving the recognition efficiency of object detection, face recognition, image compression and motion perception tasks, making convolutional computation a widely used core technology in the field of machine vision.

[0003] However, in the era of rapid development of artificial intelligence algorithms, big data, Internet of Things and other technologies, the problems and challenges of traditional CIS are increasingly prominent. First, in the traditional von Neumann architecture, the sensing and computing units are separated, and the unprocessed image data contains a lot of redundant information. Transmission of these data will lead to more energy consumption, further delay increase and additional hardware cost. Second, the readout mode of the pixels in the traditional CMOS image sensor has the problem of slow speed, which is not suitable for the high throughput requirement of artificial intelligence algorithms. In addition, directly transmitting raw image information through the network is not conducive to the protection of personal privacy. Therefore, the development of CIS needs to keep pace with the changes of the times, and the intelligent transformation of traditional CIS is the trend of the times. Currently, there is a lack of research on the collaborative design of "neural network algorithm-pixel-in-parallel convolutional architecture-analog domain computing circuit".

[0004] Intelligent CIS integrates convolutional operation directly into the CIS chip, which can complete image processing inside the image sensor, effectively reducing the data transmission requirement, improving the processing speed while reducing the power consumption. This efficient "sensing and computing integrated" machine vision system architecture has broad development potential and commercial prospects in applications such as autonomous driving, high-end security, virtual reality devices (such as eye tracking and real-time processing functions), and other smart appliances (such as drones, robotic vacuum cleaners and translation pens).

[0005] The CMOS image sensing and computing integrated circuit currently used to realize on-chip convolution calculation has problems of low weight precision, slow processing speed, and further optimization space for power consumption. Specifically, the existing related research needs to write weights in the overlapping area of convolution kernels multiple times in time-sharing mode, and cannot achieve parallel calculation and output in the overlapping area of adjacent convolution kernels, resulting in additional processing time and energy consumption. Therefore, it is necessary to optimize the optoelectronic sensing and computing array architecture to design a more efficient CMOS image sensing and computing integrated circuit for on-chip convolution calculation. SUMMARY

[0006] In view of the defects in the prior art, the purpose of the present disclosure is to provide a CMOS image sensing and computing integrated circuit based on a 5T, 8T, and 14T hybrid pixel computing unit.

[0007] To achieve the above-mentioned purpose, according to one aspect of the present disclosure, a CMOS image sensing and computing integrated circuit based on a 5T, 8T, and 14T hybrid pixel computing unit is provided, comprising: an optoelectronic sensing and computing array, a weight voltage generation module, a readout circuit, and a control module; wherein,

[0008] The optoelectronic sensing and computing array comprises a plurality of groups of hybrid pixel computing units and a plurality of row selection switches, the types of pixel computing units in the hybrid pixel computing units include 5T, 8T, and 14T, each group of hybrid pixel computing units comprises a first current bus and a second current bus, the optoelectronic sensing and computing array is used to convert light intensity signals into current signals, perform convolution multiplication and accumulation operations with weight voltages of convolution kernels through the pixel computing units, and output currents through the first current bus and the second current bus;

[0009] The weight voltage generation module comprises a plurality of groups of weight voltage generation circuits, the weight voltage generation circuits are used to generate weight voltages corresponding to the convolution kernels, and provide weight information for the global optoelectronic sensing and computing array;

[0010] The readout circuit is coupled with the first current bus and the second current bus of the optoelectronic sensing and computing array, and is used for subtraction operation, current-voltage conversion, and correlated double sampling;

[0011] The control module is coupled with the hybrid pixel computing units of the optoelectronic sensing and computing array and the row selection switches, and the control module is used to generate control signals.

[0012] Optionally, each hybrid pixel computing unit is arranged based on a convolution algorithm, 5T pixel computing units are arranged at the center positions of each convolution kernel, 8T pixel units are arranged at the left and right positions and the upper and lower positions of each convolution kernel, and 14T pixel computing units are arranged at the upper left position, the lower left position, the upper right position, and the lower right position of the center position of each convolution kernel.

[0013] Optionally, left and right positions or top and bottom positions of every two convolution kernels overlap with each other, and right bottom, left bottom, left top and right top positions of every four convolution kernels overlap with each other.

[0014] Optionally, the 8T pixel calculation unit is configured to perform 2 parallel multiplications and output, and the 14T pixel calculation unit is configured to perform 4 parallel multiplications and output.

[0015] Optionally, the 5T pixel calculation unit comprises MOS transistor M1, MOS transistor M2, MOS transistor M3, MOS transistor M4, MOS transistor M5 and photodiode PD1, a gate of the MOS transistor M1 is connected with a row line of a pixel reset signal, a drain of the MOS transistor M1 is connected with a power supply, a source of the MOS transistor M1 is connected with a drain of the MOS transistor M2, a gate of the MOS transistor M2 is connected with a row line of a charge transfer signal, a source of the MOS transistor M2 is connected with a cathode of the photodiode PD1, an anode of the photodiode PD1 is grounded, a gate of the MOS transistor M3 is connected with the drain of the MOS transistor M2, a drain of the MOS transistor M3 is connected with the power supply, a source of the MOS transistor M3 is connected with a drain of the MOS transistor M4 and a drain of the MOS transistor M5 respectively, a gate of the MOS transistor M4 is connected with a first positive weight voltage, a source of the MOS transistor M4 outputs a first current representing a positive value, a gate of the MOS transistor M5 is connected with a first negative weight voltage, and a source of the MOS transistor M5 outputs a first current representing a negative value.

[0016] Optionally, the 8T pixel calculation unit comprises MOS transistor M6, MOS transistor M7, MOS transistor M8, MOS transistor M9, MOS transistor M10, MOS transistor M11, MOS transistor M12, MOS transistor M13 and photodiode PD2, the gate of the MOS transistor M6 is connected with the row line of the pixel reset signal, the drain of the MOS transistor M6 is connected with the power supply, the source of the MOS transistor M6 is connected with the drain of the MOS transistor M7, the gate of the MOS transistor M7 is connected with the row line of the charge transfer signal, the source of the MOS transistor M7 is connected with the cathode of the photodiode PD2, the anode of the photodiode PD2 is grounded, the gate of the MOS transistor M8 is connected with the drain of the MOS transistor M7, the drain of the MOS transistor M8 is connected with the power supply, the source of the MOS transistor M8 is connected with the drain of the MOS transistor M9 and the drain of the MOS transistor M10 respectively, the gate of the MOS transistor M9 is connected with the second positive weight voltage, the source of the MOS transistor M9 outputs the second current representing the positive value, the gate of the MOS transistor M10 is connected with the second negative weight voltage, the source of the MOS transistor M10 outputs the second current representing the negative value, the gate of the MOS transistor M11 is connected with the drain of the MOS transistor M7, the drain of the MOS transistor M11 is connected with the power supply, the source of the MOS transistor M11 is connected with the drain of the MOS transistor M12 and the drain of the MOS transistor M13 respectively, the gate of the MOS transistor M12 is connected with the third positive weight voltage, the source of the MOS transistor M12 outputs the third current representing the positive value, the gate of the MOS transistor M13 is connected with the third negative weight voltage, and the source of the MOS transistor M13 outputs the third current representing the negative value.

[0017] Optionally, the 14T pixel calculation unit comprises a MOS transistor M14, a MOS transistor M15, a MOS transistor M16, a MOS transistor M17, a MOS transistor M18, a MOS transistor M19, a MOS transistor M20, a MOS transistor M21, a MOS transistor M22, a MOS transistor M23, a MOS transistor M24, a MOS transistor M25, a MOS transistor M26, a MOS transistor M27 and a photodiode PD3, a gate of the MOS transistor M14 is connected with a row line of the pixel reset signal, a drain of the MOS transistor M14 is connected with the power supply, a source of the MOS transistor M14 is connected with a drain of the MOS transistor M15, a gate of the MOS transistor M15 is connected with a row line of the charge transfer signal, a source of the MOS transistor M15 is connected with a cathode of the photodiode PD3, an anode of the photodiode PD3 is grounded, a gate of the MOS transistor M16 is connected with the drain of the MOS transistor M15, a drain of the MOS transistor M16 is connected with the power supply, a source of the MOS transistor M16 is connected with a drain of the MOS transistor M17 and a drain of the MOS transistor M18 respectively, a gate of the MOS transistor M17 is connected with the fourth positive weight voltage, a source of the MOS transistor M17 outputs the fourth current representing the positive value, a gate of the MOS transistor M18 is connected with the fourth negative weight voltage, a source of the MOS transistor M18 outputs the fourth current representing the negative value, a drain of the MOS transistor M19 is connected with the power supply, a gate of the MOS transistor M19 is connected with the drain of the MOS transistor M15, a source of the MOS transistor M19 is connected with a drain of the MOS transistor M20 and a drain of the MOS transistor M21 respectively, a gate of the MOS transistor M20 is connected with the fifth positive weight voltage, a source of the MOS transistor M20 outputs the fifth current representing the positive value, a gate of the MOS transistor M21 is connected with the fifth negative weight voltage, a source of the MOS transistor M21 outputs the fifth current representing the negative value, a drain of the MOS transistor M22 is connected with the power supply, a gate of the MOS transistor M22 is connected with the drain of the MOS transistor M15, a source of the MOS transistor M22 is connected with a drain of the MOS transistor M23 and a drain of the MOS transistor M24 respectively, a gate of the MOS transistor M23 is connected with the sixth positive weight voltage, a source of the MOS transistor M23 outputs the sixth current representing the positive value, a gate of the MOS transistor M24 is connected with the sixth negative weight voltage, a source of the MOS transistor M24 outputs the sixth current representing the negative value, a drain of the MOS transistor M25 is connected with the power supply, a gate of the MOS transistor M25 is connected with the drain of the MOS transistor M15, a source of the MOS transistor M25 is connected with a drain of the MOS transistor M26 and a drain of the MOS transistor M27 respectively, a gate of the MOS transistor M26 is connected with the seventh positive weight voltage, a source of the MOS transistor M26 outputs the seventh current representing the positive value,The gate of the MOS tube M27 is connected with the seventh negative weight voltage, and the source of the MOS tube M27 outputs the seventh current representing a negative value.

[0018] Optionally, the current output lines representing positive values corresponding to the same convolution kernel are respectively connected with the first current buses corresponding to the same convolution kernel, and the current output lines representing negative values corresponding to the same convolution kernel are respectively connected with the second current buses corresponding to the same convolution kernel, the first current buses output currents representing positive values in the results of the convolution multiply-accumulate operation, and the second current buses output currents representing negative values in the results of the convolution multiply-accumulate operation.

[0019] Optionally, the weight voltage generation circuit comprises a PMOS tube M28, a MOS tube M29, a PMOS tube M30, a PMOS tube M31, a PMOS tube M32, a PMOS tube M33, a MOS tube M34, a MOS tube M35, and an operational amplifier OP1, the source of the MOS tube M28 is connected with a power supply, the gate of the PMOS tube M28 is respectively connected with the gate and the drain of the MOS tube M29, the drain of the PMOS tube M28 is respectively connected with the gate and the drain of the MOS tube M29, the gate and the drain of the PMOS tube M28 and the gate and the drain of the MOS tube M29 are respectively connected with the positive input terminal of the operational amplifier OP1, the source of the MOS tube M29 is grounded, the negative input terminal of the operational amplifier OP1 is connected with the output terminal of the operational amplifier OP1, the source of the PMOS tube M30 is connected with the power supply, the gate of the PMOS tube M30 is connected with a first weight digital signal, the drain of the PMOS tube M30 is connected with the drain of the MOS tube M34, the gate of the MOS tube M34 is connected with the power supply, the source of the MOS tube M34 is connected with the drain of the MOS tube M35, the gate of the MOS tube M35 is connected with the drain of the MOS tube M34, the source of the MOS tube M35 is connected with the output terminal of the operational amplifier OP1, the source of the PMOS tube M31 is connected with the power supply, the gate of the PMOS tube M31 is connected with a second weight digital signal, the drain of the PMOS tube M31 is connected with the drain of the MOS tube M34, the source of the PMOS tube M32 is connected with the power supply, the gate of the PMOS tube M32 is connected with a third weight digital signal, the drain of the PMOS tube M32 is connected with the drain of the MOS tube M34, the source of the PMOS tube M33 is connected with the power supply, the gate of the PMOS tube M33 is grounded, the drain of the PMOS tube M33 is connected with the drain of the MOS tube M34, and the gate of the MOS tube M35 outputs a weight analog voltage.

[0020] Optionally, the readout circuit comprises an operational amplifier OP2, an operational amplifier OP3, an operational amplifier OP4, an operational amplifier OP5, a resistor R1, a resistor R2, a resistor R3, a resistor R4, a resistor R5, a resistor R6, a capacitor C1, a switch S1, a switch S2, a switch S3, a switch S4, a switch S5, a switch S6, a positive input end of the operational amplifier OP2 is grounded, a negative input end of the operational amplifier OP2 is connected with the first current bus, an input end of the resistor R1 is connected with the negative input end of the operational amplifier OP2, an output end of the resistor R1 is connected with an output end of the operational amplifier OP2, a positive input end of the operational amplifier OP3 is grounded, a negative input end of the operational amplifier OP3 is connected with the second current bus, an input end of the resistor R2 is connected with the negative input end of the operational amplifier OP3, an output end of the resistor R3 is connected with an output end of the operational amplifier OP3, one end of the switch S1 is connected with the output end of the operational amplifier OP2, the other end of the switch S1 is connected with an input end of the resistor R3, one end of the switch S2 is connected with the output end of the operational amplifier OP3, the other end of the switch S2 is connected with the input end of the resistor R3, one end of the switch S3 is connected with the output end of the operational amplifier OP2, the other end of the switch S3 is connected with an input end of the resistor R4, one end of the switch S4 is connected with the output end of the operational amplifier OP3, the other end of the switch S4 is connected with the input end of the resistor R4, the output end of the resistor R3 is connected with an input end of the resistor R5 and a positive input end of the operational amplifier OP4 respectively, the output end of the resistor R4 is connected with an input end of the resistor R6 and a negative input end of the operational amplifier OP4 respectively, the output end of the resistor R5 is grounded, an output end of the operational amplifier OP5 and an output end of the R6 are connected with one end of the switch S5 respectively, the other end of the switch S5 is connected with a negative input end of the operational amplifier OP5, a positive input end of the operational amplifier OP5 is grounded, an input end of the capacitor C1 is connected with the negative input end of the operational amplifier OP5, an output end of the capacitor C1 is connected with an output end of the operational amplifier OP5, one end of the switch S6 is connected with the negative input end of the operational amplifier OP5, the other end of the switch S6 is connected with the output end of the operational amplifier OP5, and the output end of the operational amplifier OP5 outputs a voltage.

[0021] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:

[0022] The CMOS image sensing and computing integrated circuit based on the 5T, 8T and 14T hybrid pixel computing units has the characteristics of high speed, high energy efficiency and high integration, and the weight voltage generation module is used to automatically assign weight voltages to the photoelectric sensing and computing array, without the need to re-write weights at the convolution kernel overlapping position, thereby reducing the dynamic loss of the circuit structure, and the 5T, 8T and 14T pixel computing units are used in the photoelectric sensing and computing array to form overlapping positions, so that parallel computing and output are realized at the convolution kernel overlapping position, and the readout circuit is combined to realize efficient whole-row parallel readout, thereby improving the frame rate and computing efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0023] Other features, objects and advantages of the present disclosure will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings:

[0024] Figure 1 FIG. 1 is a schematic diagram of a logic structure of a CMOS image sensing and computing integrated circuit based on 5T, 8T and 14T hybrid pixel computing units according to an example embodiment.

[0025] Figure 2 FIG. 2 is a schematic diagram of a circuit structure of a 5T pixel computing unit according to an example embodiment.

[0026] Figure 3 FIG. 3 is a schematic diagram of a circuit structure of an 8T pixel computing unit according to an example embodiment.

[0027] Figure 4 FIG. 4 is a schematic diagram of a circuit structure of a 14T pixel computing unit according to an example embodiment.

[0028] Figure 5 FIG. 5 is a schematic diagram of a circuit structure of a weight voltage generation circuit according to an example embodiment.

[0029] Figure 6 FIG. 6 is a schematic diagram of a circuit structure of a readout circuit according to an example embodiment.

[0030] Figure 7 FIG. 7 is a schematic diagram of voltage waveforms of a clock signal and a control signal of a control module according to an example embodiment.

[0031] Figure 8 FIG. 8 is a schematic diagram of a weight analog voltage output-digital weight relationship of a weight voltage generation circuit according to an example embodiment.

[0032] Figure 9A relationship between a first current representing a positive value output by a 5T pixel calculation unit according to an exemplary embodiment and a digital weight corresponding to a first positive weight voltage of a gate of a MOS tube M4 and a light power is shown. DETAILED DESCRIPTION

[0033] The present disclosure will be described in detail below with specific embodiments. The following embodiments will help those skilled in the art to further understand the present disclosure, but do not limit the present disclosure in any form. It should be pointed out that, for those skilled in the art, without departing from the concept of the present disclosure, a number of modifications and improvements can be made. These all belong to the protection scope of the present disclosure.

[0034] Figure 1 A CMOS image sensing and calculation integrated circuit logic structure schematic diagram based on a 5T, 8T, 14T hybrid pixel calculation unit according to an exemplary embodiment is shown.

[0035] As shown in Figure 1 , the present disclosure provides a CMOS image sensing and calculation integrated circuit based on a 5T, 8T, 14T hybrid pixel calculation unit, comprising: a photoelectric sensing and calculation array, a weight voltage generation module, a readout circuit and a control module.

[0036] The photoelectric sensing and calculation array comprises a plurality of groups of hybrid pixel calculation units and a plurality of row selection switches, the types of pixel calculation units in the hybrid pixel calculation units include 5T, 8T and 14T, each group of hybrid pixel calculation units comprises a first current bus and a second current bus, the photoelectric sensing and calculation array is used to convert a light intensity signal into a current signal, perform a convolution multiplication and accumulation operation through the weight voltage of the pixel calculation unit and the convolution kernel, and output the current through the first current bus and the second current bus.

[0037] Among them, the first current bus is used to output the current representing the positive value of the convolution multiplication and accumulation operation result, and the second current bus is used to output the current representing the negative value of the convolution multiplication and accumulation operation result.

[0038] The photoelectric sensing and calculation array comprises n×n pixel calculation units and a plurality of row selection switches Sel(j), wherein the row selection switch Sel(j) represents the row selection switch Sel of the jth row, each row selection switch controls the current representing the positive value and the current representing the negative value belonging to the same convolution kernel in the adjacent 3×3 pixel calculation units to flow into the first current bus and the second current bus respectively, and there are groups of row selection switches in the photoelectric sensing and calculation array, i.e. the photoelectric sensing and calculation array performs times of 3×3(stride=2) convolution.

[0039] The weight voltage generation module includes a plurality of groups of weight voltage generation circuits, which are configured to generate weight voltages corresponding to convolution kernels and provide weight information to the global photoelectric sensing array.

[0040] The weight voltage generation module includes 3*3*2 groups of weight voltage generation circuits, i.e., 18 groups of weight voltage generation circuits, which are configured to generate ±3bit weight voltages corresponding to 3*3 convolution kernels, and the result of subtraction of positive weight voltages (≥0V) and negative weight voltages (≥0V) is used to represent the real weight voltage corresponding to the convolution kernel.

[0041] The readout circuit is coupled to the first current bus and the second current bus of the photoelectric sensing array, and is configured to perform subtraction operation, current-voltage conversion and correlated double sampling (CDS).

[0042] The readout circuit is configured to receive current signals output by each group of mixed pixel calculation units in the photoelectric sensing array, wherein the readout circuit receives current representing positive values output by the first current bus through convolution multiplication and accumulation operation and current representing negative values output by the second current bus through convolution multiplication and accumulation operation.

[0043] The readout circuit performs subtraction operation on the received current representing positive values and the current representing negative values, obtains output current of the complete convolution multiplication and accumulation operation, and determines the real current output by the convolution multiplication and accumulation operation.

[0044] The readout circuit performs current-voltage conversion and correlated double sampling on the real current output by the convolution multiplication and accumulation operation, and outputs voltage.

[0045] The control module is coupled to the mixed pixel calculation unit and the row selection switch of the photoelectric sensing array, and the control module is configured to generate a control signal.

[0046] The control module includes a digital circuit, and the control module is configured to provide a reset signal and a charge transfer signal for each row of pixel calculation units of the photoelectric sensing array, and to provide a row selection signal for each row of row selection switches Sel(j) of the photoelectric sensing array, so as to ensure that sensing, operation and output of the photoelectric sensing array are automatically performed row by row.

[0047] The CMOS image sensing integrated circuit based on the 5T, 8T and 14T hybrid pixel calculation units has the characteristics of high speed, high energy efficiency and high integration, and the weight voltage generation module is used to automatically assign the weight voltage to the photoelectric sensing array, so that the weight does not need to be written again at the overlapping position of the convolution kernel, the dynamic loss of the circuit structure is reduced, the 5T, 8T and 14T pixel calculation units are used in the photoelectric sensing array, and the overlapping positions are formed, so that the parallel calculation and output at the overlapping position of the convolution kernel are realized, and the high-efficiency whole-row parallel readout is realized by combining the readout circuit, so that the frame rate and the operation efficiency are improved.

[0048] In a possible embodiment, the 5T, 8T and 14T pixel calculation units in each hybrid pixel calculation unit are arranged based on a 3*3 (stride=2) convolution algorithm.

[0049] As shown in Figure 1 In a possible embodiment, the 5T, 8T and 14T pixel calculation units in each hybrid pixel calculation unit are arranged based on a convolution algorithm, the 5T pixel calculation unit is arranged at the center position of each convolution kernel, the 8T pixel unit is arranged at the left and right positions and the upper and lower positions of each convolution kernel, and the 14T pixel calculation unit is arranged at the upper left position, the lower left position, the upper right position and the lower right position of the center position of each convolution kernel.

[0050] In the present disclosure, the left and right positions or the upper and lower positions of every two convolution kernels overlap with each other, and the lower right, lower left, upper left and upper right positions of every four convolution kernels overlap with each other.

[0051] As an example, the 5T pixel calculation unit is located at the center position of each convolution kernel, the 8T pixel calculation unit is located at the 4 positions of the left and right or upper and lower overlapping of two convolution kernels, and the 14T pixel calculation unit is located at the corner positions of the overlapping of four convolution kernels.

[0052] In a possible embodiment, the 8T pixel calculation unit is used to perform 2 parallel multiplications and output, and the 14T pixel calculation unit is used to perform 4 parallel multiplications and output.

[0053] Figure 2 FIG. 1 is a circuit structure schematic diagram of a 5T pixel calculation unit according to an example embodiment.

[0054] As shown in Figure 2 In a possible embodiment, the 5T pixel calculation unit includes MOS tubes M1, M2, M3, M4, M5 and a photodiode PD1.

[0055] The gate of the MOS transistor M1 is connected to the row line of the pixel reset signal, the drain of the MOS transistor M1 is connected to the power supply, the source of the MOS transistor M1 is connected to the drain of the MOS transistor M2, the gate of the MOS transistor M2 is connected to the row line of the charge transfer signal, the source of the MOS transistor M2 is connected to the cathode of the photodiode PD1, the anode of the photodiode PD1 is grounded, the gate of the MOS transistor M3 is connected to the drain of the MOS transistor M2, the drain of the MOS transistor M3 is connected to the power supply, the source of the MOS transistor M3 is connected to the drain of the MOS transistor M4 and the drain of the MOS transistor M5 respectively, the gate of the MOS transistor M4 is connected to the first positive weight voltage, the source of the MOS transistor M4 outputs a first current representing a positive value, the gate of the MOS transistor M5 is connected to the first negative weight voltage, and the source of the MOS transistor M5 outputs a first current representing a negative value.

[0056] The gate of MOS transistor M1 is connected to the row line of the k-th row pixel reset signal Vrst(k), and the gate of MOS transistor M2 is connected to the row line of the k-th row charge transfer signal Vtx(k). The gate of MOS transistor M4 is connected to the first positive weight voltage Wp1, and the source of computing transistor M4 outputs a positive first current Ip1. The gate of MOS transistor M5 is connected to the first negative weight voltage Wn1, and the source of computing transistor M5 outputs a negative first current In1.

[0057] In the present disclosure, in the circuit structure of the 5T pixel computing unit, the MOS transistor M1 is represented as the reset transistor M1, the MOS transistor M2 is represented as the charge transfer transistor M2, the MOS transistor M3 is represented as the source follower transistor M3, the MOS transistor M4 is represented as the computing transistor M4, and the MOS transistor M5 is represented as the computing transistor M5.

[0058] Figure 3 FIG. 4 is a schematic diagram showing the circuit structure of an 8T pixel calculation unit according to an exemplary embodiment.

[0059] like Figure 3 As shown, in a possible embodiment, the 8T pixel computing unit includes a MOS transistor M6, a MOS transistor M7, a MOS transistor M8, a MOS transistor M9, a MOS transistor M10, a MOS transistor M11, a MOS transistor M12, a MOS transistor M13 and a photodiode PD2.

[0060] The gate of the MOS transistor M6 is connected with the row line of the pixel reset signal, the drain of the MOS transistor M6 is connected with the power supply, the source of the MOS transistor M6 is connected with the drain of the MOS transistor M7, the gate of the MOS transistor M7 is connected with the row line of the charge transfer signal, the source of the MOS transistor M7 is connected with the cathode of the photodiode PD2, the anode of the photodiode PD2 is grounded, the gate of the MOS transistor M8 is connected with the drain of the MOS transistor M7, the drain of the MOS transistor M8 is connected with the power supply, the source of the MOS transistor M8 is connected with the drain of the MOS transistor M9 and the drain of the MOS transistor M10 respectively, the gate of the MOS transistor M9 is connected with the second positive weight voltage, the source of the MOS transistor M9 outputs the second current representing the positive value, the gate of the MOS transistor M10 is connected with the second negative weight voltage, the source of the MOS transistor M10 outputs the second current representing the negative value, the gate of the MOS transistor M11 is connected with the drain of the MOS transistor M7, the drain of the MOS transistor M11 is connected with the power supply, the source of the MOS transistor M11 is connected with the drain of the MOS transistor M12 and the drain of the MOS transistor M13 respectively, the gate of the MOS transistor M12 is connected with the third positive weight voltage, the source of the MOS transistor M12 outputs the third current representing the positive value, and the gate of the MOS transistor M13 is connected with the third negative weight voltage, the source of the MOS transistor M13 outputs the third current representing the negative value.

[0061] The gate of the MOS transistor M6 is connected with the row line of the pixel reset signal, the drain of the MOS transistor M6 is connected with the power supply, the source of the MOS transistor M6 is connected with the drain of the MOS transistor M7, the gate of the MOS transistor M7 is connected with the row line of the charge transfer signal, the source of the MOS transistor M7 is connected with the cathode of the photodiode PD2, the anode of the photodiode PD2 is grounded, the gate of the MOS transistor M8 is connected with the drain of the MOS transistor M7, the drain of the MOS transistor M8 is connected with the power supply, the source of the MOS transistor M8 is connected with the drain of the MOS transistor M9 and the drain of the MOS transistor M10 respectively, the gate of the MOS transistor M9 is connected with the second positive weight voltage, the source of the MOS transistor M9 outputs the second current representing the positive value, the gate of the MOS transistor M10 is connected with the second negative weight voltage, the source of the MOS transistor M10 outputs the second current representing the negative value, the gate of the MOS transistor M11 is connected with the drain of the MOS transistor M7, the drain of the MOS transistor M11 is connected with the power supply, the source of the MOS transistor M11 is connected with the drain of the MOS transistor M12 and the drain of the MOS transistor M13 respectively, the gate of the MOS transistor M12 is connected with the third positive weight voltage, the source of the MOS transistor M12 outputs the third current representing the positive value, and the gate of the MOS transistor M13 is connected with the third negative weight voltage, the source of the MOS transistor M13 outputs the third current representing the negative value.

[0062] In the present disclosure, in the circuit structure of the 8T pixel calculation unit, the MOS transistor M6 represents the reset transistor M6, the MOS transistor M7 represents the pixel transfer transistor M7, the MOS transistor M8 represents the source follower transistor M8, the MOS transistor M9 represents the calculation transistor M9, the MOS transistor M10 represents the calculation transistor M10, the MOS transistor M11 represents the source follower M11, the MOS transistor M12 represents the calculation transistor M12, and the MOS transistor M13 represents the calculation transistor M13.

[0063] Figure 4 FIG. 1 is a circuit structure schematic diagram of a 14T pixel calculation unit according to an exemplary embodiment.

[0064] AsFigure 4 As shown, in one possible embodiment, the 14T pixel computing unit includes MOS transistor M14, MOS transistor M15, MOS transistor M16, MOS transistor M17, MOS transistor M18, MOS transistor M19, MOS transistor M20, MOS transistor M21, MOS transistor M22, MOS transistor M23, MOS transistor M24, MOS transistor M25, MOS transistor M26, MOS transistor M27, and photodiode PD3.

[0065] The gate of MOS transistor M14 is connected to a row line of a pixel reset signal, the drain of MOS transistor M14 is connected to a power supply, the source of MOS transistor M14 is connected to the drain of MOS transistor M15, the gate of MOS transistor M15 is connected to a row line of a charge transfer signal, the source of MOS transistor M15 is connected to the cathode of photodiode PD3, the anode of photodiode PD3 is grounded, the gate of MOS transistor M16 is connected to the drain of MOS transistor M15, the drain of MOS transistor M16 is connected to a power supply, the source of MOS transistor M16 is connected to the drain of MOS transistor M17 and the drain of MOS transistor M18, the gate of MOS transistor M17 is connected to a fourth positive weight voltage, the source of MOS transistor M17 outputs a fourth current representing a positive value, the gate of MOS transistor M18 is connected to a fourth negative weight voltage, the source of MOS transistor M18 outputs a fourth current representing a negative value, the drain of MOS transistor M19 is connected to a power supply, the gate of MOS transistor M19 is connected to the drain of MOS transistor M15, the source of MOS transistor M19 is connected to the drain of MOS transistor M20 and the drain of MOS transistor M21, the gate of MOS transistor M20 is connected to a fifth positive weight voltage, the source of MOS transistor M20 outputs a fifth current representing a positive value, the gate of MOS transistor M21 is connected to a fifth negative weight voltage, the source of MOS transistor M21 outputs a fifth current representing a negative value, the drain of MOS transistor M22 is connected to a power supply, the gate of MOS transistor M22 is connected to the drain of MOS transistor M15, the source of MOS transistor M22 is connected to the drain of MOS transistor M23 and the drain of MOS transistor M24, the gate of MOS transistor M23 is connected to a sixth positive weight voltage, the source of MOS transistor M23 outputs a sixth current representing a positive value, the gate of MOS transistor M24 is connected to a sixth negative weight voltage, the source of MOS transistor M24 outputs a sixth current representing a negative value, the drain of MOS transistor M25 is connected to a power supply, the gate of MOS transistor M25 is connected to the drain of MOS transistor M15, the source of MOS transistor M25 is connected to the drain of MOS transistor M26 and the drain of MOS transistor M27, the gate of MOS transistor M26 is connected to a seventh positive weight voltage, the source of MOS transistor M26 outputs a seventh current representing a positive value, the gate of MOS transistor M27 is connected to a seventh negative weight voltage, the source of MOS transistor M27 outputs a seventh current representing a negative value.

[0066] The gate of the MOS transistor M14 is connected with the row line of the kth row pixel reset signal Vrst(k), and the gate of the MOS transistor M15 is connected with the row line of the kth row charge transfer signal Vtx(k). The gate of the MOS transistor M17 is connected with the fourth positive weight voltage Wp4, the source of the MOS transistor M17 outputs the fourth current Ip4 representing a positive value, the gate of the MOS transistor M18 is connected with the fourth negative weight voltage Wn4, and the source of the MOS transistor M18 outputs the fourth current In4 representing a negative value; the gate of the MOS transistor M20 is connected with the fifth positive weight voltage Wp5, the source of the MOS transistor M20 outputs the fifth current Ip5 representing a positive value, the gate of the MOS transistor M21 is connected with the fifth negative weight voltage Wn5, and the source of the MOS transistor M21 outputs the fifth current In5 representing a negative value; the gate of the MOS transistor M23 is connected with the sixth positive weight voltage Wp6, the source of the MOS transistor M23 outputs the sixth current Ip6 representing a positive value, the gate of the MOS transistor M24 is connected with the sixth negative weight voltage Wn6, and the source of the MOS transistor M24 outputs the sixth current In6 representing a negative value, the gate of the MOS transistor M26 is connected with the seventh positive weight voltage Wp7, the source of the MOS transistor M26 outputs the seventh current Ip7 representing a positive value, the gate of the MOS transistor M27 is connected with the seventh negative weight voltage Wn7, and the source of the MOS transistor M27 outputs the seventh current In7 representing a negative value.

[0067] In the present disclosure, in the circuit structure of the 14T pixel calculation unit, the MOS transistor M14 represents the reset transistor M14, the MOS transistor M15 represents the pixel transfer transistor M15, the MOS transistor M16 represents the source follower M16, the MOS transistor M17 represents the calculation transistor M17, the MOS transistor M18 represents the calculation transistor M18, the MOS transistor M19 represents the source follower M19, the MOS transistor M20 represents the calculation transistor M20, the MOS transistor M21 represents the calculation transistor M21, the MOS transistor M22 represents the source follower M22, the MOS transistor M23 represents the calculation transistor M23, the MOS transistor M24 represents the calculation transistor M24, the MOS transistor M25 represents the source follower M25, the MOS transistor M26 represents the calculation transistor M26, and the MOS transistor M27 represents the calculation transistor M27.

[0068] In a possible embodiment, the current output lines representing positive values corresponding to the same kernel are respectively connected with the first current buses corresponding to the same kernel, and the current output lines representing negative values corresponding to the same kernel are respectively connected with the second current buses corresponding to the same kernel. The first current buses output currents representing positive values in the results of the convolution multiplication and accumulation operation, and the second current buses output currents representing negative values in the results of the convolution multiplication and accumulation operation.

[0069] Each group of mixed pixel calculation units corresponding to one volume kernel includes a group of buses, wherein the group of buses includes a first current bus and a second current bus.

[0070] In the present disclosure, the 9 current output lines representing positive values belonging to the same volume kernel are respectively connected with the first current bus belonging to the same volume kernel, and the 9 current output lines representing negative values belonging to the same volume kernel are respectively connected with the second current bus belonging to the same volume kernel, and the connection of the current output lines and the current bus is controlled by the row selection switch Sel.

[0071] As an example, the source of MOS tube M4 of one 5T pixel calculation unit, the source of MOS tube M9 of two 8T pixel calculation units, the source of MOS tube M12 of two 8T pixel calculation units, the source of MOS tube M17 of one 14T pixel calculation unit, the source of MOS tube M20 of one 14T pixel calculation unit, the source of MOS tube M23 of one 14T pixel calculation unit, and the source of MOS tube M26 of one 14T pixel calculation unit corresponding to the same volume kernel are respectively connected with the first current bus as the current output lines representing positive values. The source of MOS tube M5 of one 5T pixel calculation unit, the source of MOS tube M10 of two 8T pixel calculation units, the source of MOS tube M13 of two 8T pixel calculation units, the source of MOS tube M18 of one 14T pixel calculation unit, the source of MOS tube M21 of one 14T pixel calculation unit, the source of MOS tube M24 of one 14T pixel calculation unit, and the source of MOS tube M27 of one 14T pixel calculation unit corresponding to the same volume kernel are respectively connected with the second current bus as the current output lines representing negative values. The positive current bus outputs the positive current of the convolution multiplication accumulation operation, and the negative current bus outputs the negative current of the convolution multiplication accumulation operation.

[0072] Among them, the source of MOS tube M9 and the source of MOS tube M12 of the same 8T pixel calculation unit correspond to different convolution kernels as current output lines representing positive values; the source of MOS tube M10 and the source of MOS tube M13 of the same 8T pixel calculation unit correspond to different convolution kernels as current output lines representing negative values. The source of MOS tube M17, the source of MOS tube M20, the source of MOS tube M23, and the source of MOS tube M26 of the same 14T pixel calculation unit correspond to different convolution kernels as current output lines representing positive values; the source of MOS tube M18, the source of MOS tube M21, the source of MOS tube M24, and the source of MOS tube M27 of the same 14T pixel calculation unit correspond to different convolution kernels as current output lines representing negative values.

[0073] As Figures 2 to 4As shown, in one possible embodiment, when the reset signal Vrst(k) and the charge transfer signal Vtx(k) are high, the parasitic capacitances of the photodiodes PD1, PD2, and PD3 are charged until the voltages of the photodiodes PD1, PD2, and PD3 approach the power supply voltage VDD, and the reset signal Vrst(k) and the charge transfer signal Vtx(k) are both set to low, and the charging of the parasitic capacitances of the photodiodes PD1, PD2, and PD3 is stopped.

[0074] The pixel calculation units of the photodiode array are exposed to a preset light intensity, and the current flowing through the photodiodes PD1, PD2, and PD3 is a photocurrent, which is proportional to the light intensity of the incident light. The parasitic capacitances inside the photodiodes PD1, PD2, and PD3 are discharged, and the voltages across the photodiodes PD1, PD2, and PD3 are reduced,

[0075] The reset signal Vrst(k) is kept low, and the charge transfer signal Vtx(k) is set to high to remove the electrons leaked from the cathodes of the photodiodes PD1, PD2, and PD3 to the charge transfer regions of the transfer tubes during the exposure. At this time, the voltage of the charge transfer region is Vr.

[0076] For example, the electrons leaked from the cathode of the photodiode PD1 to the drain of the MOS tube M2 through the MOS tube M2; the electrons leaked from the cathode of the photodiode PD2 to the drain of the MOS tube M7 through the MOS tube M7; and the electrons leaked from the cathode of the photodiode PD3 to the drain of the MOS tube M15 through the MOS tube M15.

[0077] The reset signal Vrst(k) and the charge transfer signal Vtx(k) are both set to low, and the voltage Vr of the charge transfer region is sampled, i.e., the voltages of the drains of the MOS tubes M2, M7, and M15 are sampled. The voltage Vr of the charge transfer region is output to the readout circuit through the source follower tube and the calculation tube, and the readout circuit performs a correlated double sampling (CDS) operation. The voltage Vr is the first sampling signal of the CDS operation of the readout circuit.

[0078] After the voltage Vr is sampled, the charge transfer phase is entered, the reset signal Vrst(k) is kept low, and the charge transfer signal Vtx(k) is set to high. The electrons are leaked from the cathodes of the photodiodes PD1, PD2, and PD3 to the charge transfer regions of the transfer tubes.

[0079] The reset signal Vrst(k) and the charge transfer signal Vtx(k) are both set to low, and the voltage Vr of the charge transfer region is sampled. Based on the action of the source follower M3, the source follower M8, the source follower M11, the source follower M16, the source follower M19, the source follower M22, and the source follower M25, the voltage of the source of the source follower M3, the source follower M8, the source follower M11, the source follower M16, the source follower M19, the source follower M22, and the source follower M25 changes following the voltage Vr of the charge transfer region, and the voltage of the source of each source follower is represented by the signal Vs. At this time, the computing tubes M4, M5, M9, M10, M12, M13, M17, M18, M20, M21, M23, M24, M26, and M27 work in the linear region, and the positive voltage Wp or the negative voltage Wn of the gate voltage comes from the weight analog voltage Wout of the output of the weight voltage generation circuit, and the difference between the gate voltages of the two computing tubes connected to the same source follower is taken as the weight, that is, (Wp-Wn).

[0080] The voltage formula of the MOS tube working in the linear region is:

[0081]

[0082]

[0083] (Ip-In)=2Kn[(Wp-Wn)Vs]→ΔI=2Kn*Weight*Vs.

[0084] Where (Ip-In)=2Kn*Weight*Vs represents the result of multiplying the optical information by the weight.

[0085] In the present disclosure, the 8T pixel computing unit outputs (Ip1-In1)=2Kn*Weight1*Vs and (Ip2-In2)=2Kn*Weight2*Vs in parallel; the 14T pixel computing unit outputs (Ip1-In1)=2Kn*Weight1*Vs, (IP2-In2)=2Kn*Weight2*Vs, (Ip3-In3)=2Kn*Weight3*Vs, and (Ip4-In4)=2Kn*Weight4*Vs in parallel.

[0086] Figure 5 It is a circuit structure schematic diagram of a weight voltage generation circuit according to an exemplary embodiment.

[0087] As Figure 5As shown, in a possible embodiment, the weight voltage generating circuit includes a PMOS transistor M28, a MOS transistor M29, a PMOS transistor M30, a PMOS transistor M31, a PMOS transistor M32, a PMOS transistor M33, a MOS transistor M34, a MOS transistor M35, and an operational amplifier OP1. The source of the MOS transistor M28 is connected to the power supply, the gate of the PMOS transistor M28 is respectively connected to the gate and drain of the MOS transistor M29, the drain of the PMOS transistor M28 is respectively connected to the gate and drain of the MOS transistor M29, the gate and drain of the PMOS transistor M28 and the gate and drain of the MOS transistor M29 are respectively connected to the positive phase input terminal of the operational amplifier OP1, the source of the MOS transistor M29 is grounded, the negative phase input terminal of the operational amplifier OP1 is connected to the output terminal of the operational amplifier OP1, the source of the PMOS transistor M30 is connected to the power supply, the gate of the PMOS transistor M30 is connected to the first weight digital signal, and the PMOS transistor M30 is connected to the first weight digital signal. The drain of M30 is connected to the drain of MOS transistor M34, the gate of MOS transistor M34 is connected to a power supply, the source of MOS transistor M34 is connected to the drain of MOS transistor M35, the gate of MOS transistor M35 is connected to the drain of MOS transistor M34, the source of MOS transistor M35 is connected to the output end of operational amplifier OP1, the source of PMOS transistor M31 is connected to the power supply, the gate of PMOS transistor M31 is connected to the second weighted digital signal, the drain of PMOS transistor M31 is connected to the drain of MOS transistor M34, the source of PMOS transistor M32 is connected to the power supply, the gate of PMOS transistor M32 is connected to the third weighted digital signal, the drain of PMOS transistor M32 is connected to the drain of MOS transistor M34, the source of PMOS transistor M33 is connected to the power supply, the gate of PMOS transistor M33 is grounded, the drain of PMOS transistor M33 is connected to the drain of MOS transistor M34, and the gate of MOS transistor M35 outputs a weighted analog voltage.

[0088] like Figure 3 As shown, in a possible embodiment, in the weight voltage generating circuit, the operational amplifier OP1 is connected in a voltage follower manner, and the PMOS tube M28, MOS tube M29 and operational amplifier OP1 are used to provide a static bias voltage for the source of the MOS tube M35, and the minimum value of the weighted analog voltage Wout output by the weight voltage generating circuit is a preset value.

[0089] The width-to-length ratio W / L of the PMOS transistors M30, M31, and M32 is 1:2:4. The sources of the PMOS transistors M30, M31, and M32 are respectively connected to the power supply VDD, their drains are respectively connected, and their gates receive a 3-bit digital weight input, where the highest bit is Win3, the middle bit is Win2, and the lowest bit is Win1. The digital input is valid at a low level.

[0090] PMOS transistor M33 is connected in parallel with PMOS transistor M30, PMOS transistor M31 and PMOS transistor M32, the gate of PMOS transistor M33 is grounded, and PMOS transistor M33 is used to provide bias. When the digital input is 3'b0, the weight analog voltage output by the weight voltage generation circuit is the minimum voltage in the output swing of the weight voltage generation circuit, rather than 0V.

[0091] In MOS transistor M34, the resistance of MOS transistor M34 can be adjusted by changing the aspect ratio of MOS transistor M34, and the linearity between the output range of the output weight analog voltage Wout and the digital input can be adjusted by the principle of resistance negative feedback.

[0092] The gate of MOS transistor M35 outputs the weight analog voltage Wout.

[0093] Figure 6 FIG. 1 is a circuit structure schematic diagram of a readout circuit according to an exemplary embodiment.

[0094] As Figure 6As shown, in one possible embodiment, the readout circuit includes an operational amplifier OP2, an operational amplifier OP3, an operational amplifier OP4, an operational amplifier OP5, a resistor R1, a resistor R2, a resistor R3, a resistor R4, a resistor R5, a resistor R6, a capacitor C1, a switch S1, a switch S2, a switch S3, a switch S4, a switch S5, a switch S6, a positive input terminal of the operational amplifier OP2 is grounded, a negative input terminal of the operational amplifier OP2 is connected with the first current bus, an input terminal of the resistor R1 is connected with the negative input terminal of the operational amplifier OP2, an output terminal of the resistor R1 is connected with an output terminal of the operational amplifier OP2, a positive input terminal of the operational amplifier OP3 is grounded, a negative input terminal of the operational amplifier OP3 is connected with the second current bus, an input terminal of the resistor R2 is connected with the negative input terminal of the operational amplifier OP3, an output terminal of the resistor R3 is connected with an output terminal of the operational amplifier OP3, one end of the switch S1 is connected with the output terminal of the operational amplifier OP2, the other end of the switch S1 is connected with an input terminal of the resistor R3, one end of the switch S2 is connected with the output terminal of the operational amplifier OP3, the other end of the switch S2 is connected with the input terminal of the resistor R3, one end of the switch S3 is connected with the output terminal of the operational amplifier OP2, the other end of the switch S3 is connected with an input terminal of the resistor R4, one end of the switch S4 is connected with the output terminal of the operational amplifier OP3, the other end of the switch S4 is connected with the input terminal of the resistor R4, the output terminal of the resistor R3 is connected with an input terminal of the resistor R5 and a positive input terminal of the operational amplifier OP4 respectively, the output terminal of the resistor R4 is connected with an input terminal of the resistor R6 and a negative input terminal of the operational amplifier OP4 respectively, the output terminal of the resistor R5 is grounded, an output terminal of the operational amplifier OP5 and an output terminal of the R6 are connected with one end of the switch S5 respectively, the other end of the switch S5 is connected with a negative input terminal of the operational amplifier OP5, the positive input terminal of the operational amplifier OP5 is grounded, an input terminal of the capacitor C1 is connected with the negative input terminal of the operational amplifier OP5, an output terminal of the capacitor C1 is connected with the output terminal of the operational amplifier OP5, one end of the switch S6 is connected with the negative input terminal of the operational amplifier OP5, the other end of the switch S6 is connected with the output terminal of the operational amplifier OP5, and the output terminal of the operational amplifier OP5 outputs a voltage.

[0095] As Figure 6As shown, in the circuit structure of the readout circuit, the negative phase input terminal of the operational amplifier OP2 receives the current Ioutp(i) output on the first current bus of the i-th column of the photoelectric sensing array, and the current Ioutp(i) is converted into the voltage output at the output terminal of the operational amplifier OP2 by using the operational amplifier OP2 and the resistor R1. The negative phase input terminal of the operational amplifier OP3 receives the current Ioutn(i) output on the second current bus of the i-th column of the photoelectric sensing array, and the current Ioutn(i) is converted into the voltage output Vout(i) at the output terminal of the operational amplifier OP3 by using the operational amplifier OP3 and the resistor R2. Wherein, R1=R2=R.

[0096] The operational amplifier OP4, the resistor R3, the resistor R4, the resistor R5, and the resistor R6 constitute a subtraction circuit, and R3=R4=R5=R6.

[0097] As an example, when the switch S1 and the switch S4 are closed, and the switch S2 and the switch S3 are opened, the output voltage of the operational amplifier OP4 satisfies Vm=R*Ioutp-R*Ioutn=R*(Ip-In)=2RKn*Weight*Vs.

[0098] As another example, when the switch S2 and the switch S3 are closed, and the switch S1 and the switch S4 are opened, the output voltage of the operational amplifier OP4 satisfies Vm=R*Ioutn-R*Ioutp=R*(In-Ip)=-2RKn*Weight*Vs.

[0099] The operational amplifier OP5, the switch S5, the sampling capacitor C1, and the switch S6 are used for a sample and hold circuit.

[0100] As an example, when the switch S5 is closed and the switch S6 is opened, it is the sampling stage, and the output terminal voltage of the operational amplifier OP4 charges the capacitor C1.

[0101] As another example, when the switch S5 and the switch S6 are opened, it is the holding stage, and the capacitor C1 holds the stage, and the output terminal voltage of the operational amplifier OP5 does not change.

[0102] As another example, when the switch S5 is opened and the switch S6 is closed, it is the reset stage, the capacitor C1 is discharged, and the voltage across the capacitor C1 becomes 0V, which is ready for the sampling of the multiplication and accumulation of the next row of the photoelectric sensing array.

[0103] In a possible embodiment, the readout circuit is used for correlated double sampling (CDS):

[0104] The differential circuit output corresponding to a group of buses at the first correlated sampling is (Irp-Irn), and the output voltage of the operational amplifier OP4 is Vmr. The differential circuit output corresponding to the same group of buses at the second correlated sampling is (Isp-Isn), and the output voltage of the operational amplifier OP4 is Vms. The time length of the two samplings is T.

[0105] Based on the correlated double sampling (CDS), at the first correlated sampling, the switches S1 and S4 are closed, and the switches S2 and S3 are disconnected, so Vmr=R*(Irp-Irn). At the second correlated sampling, the switches S1 and S4 are disconnected, and the switches S2 and S3 are closed, so Vms=R*(Isn-Isp)=-R*(Isp-Isn).

[0106] Both samplings charge the capacitor C1, and the charge quantity is C=T*Vmr+T*Vms=T*(Vmr+Vms)=T*[(Irp-Irn)+(Isn-Isp)]=T*[(Irp-Irn)-(Isp-Isn)], which realizes the correlated double sampling, and the state of the capacitor is not switched from charging to discharging at the second correlated sampling, so the circuit structure is simplified.

[0107] Figure 7 is a voltage waveform diagram of a clock signal and a control signal of a control module according to an exemplary embodiment.

[0108] As shown in Figure 7 In a possible embodiment, in the control module, CLK represents a digital clock of the control module. Sel(j) represents a control signal of a row selection switch Sel for controlling the connection of the jth row of the pixel calculation unit of the photoelectric sensing array and the current bus; Vrst(k) represents a control signal of a reset signal for controlling the kth row of the photoelectric sensing array; Vtx(k) represents a control signal of a charge transfer signal for controlling the kth row of the photoelectric sensing array; and CDS S / H represents a control signal of a correlated double sampling for controlling the readout circuit, that is, a control signal of the opening time length of the two samplings of the switch S5.

[0109] In the photoelectric sensing array, a row selection switch Sel can be arranged every interval row, and the row selection switch Sel is arranged in the jth row, 1≤j≤(n-1) / 2; a reset signal row line Vrst and a charge transfer signal Vtx can be arranged in each row, and the reset signal row line Vrst and the charge transfer signal Vtx are arranged in the kth row, 1≤k≤n; the jth row selection switch Sel can control the readout of the pixel calculation units in the 2j-1≤k≤2j+1 rows, and the reset signal row line Vrst and the charge transfer signal Vtx in the 2j-1≤k≤2j+1 rows control the reset and charge transfer of the pixel calculation units in the 2j-1≤k≤2j+1 rows respectively.

[0110] The row selection switch in the jth row is opened, and the jth row selection signal controls the reset signal row line Vrst and the charge transfer signal Vtx in the three rows of k=2j-1, k=2j, and k=2j+1. In the convolution operation, the transform relationship of the convolution kernel moving down, the row selection switch in the j+1th row is opened, and then the reset signal in the Vrst(k+2) and the charge transfer signal in the Vtx(k+2) are controlled by the row selection signal in the j+1th row, wherein k+2=2j+1, k+2=2j+2, and k+2=2j+3.

[0111] As an example, in the reset phase, the control module sets the reset signal Vrst and the charge transfer signal Vtx of the pixel calculation units in the adjacent three rows (such as the 2j-1≤k≤2j+1 rows, the same below) of the photoelectric sensing array to high level through the control signal, the row selection switches Sel (such as the row selection switch Sel in the jth row, the same below) corresponding to the pixel calculation units in the adjacent three rows are all disconnected, and the CDS S / H signal is set to low level.

[0112] As another example, in the exposure phase, the control module sets the reset signal Vrst and the charge transfer signal Vtx of the pixel calculation units in the adjacent three rows of the photoelectric sensing array to low level through the control signal, and a group of row selection switches Sel corresponding to the pixel calculation units in the adjacent three rows are all disconnected.

[0113] As another example, in the charge transfer region reset phase, the control module sets the reset signal Vrst of the pixel calculation units in the adjacent three rows of the photoelectric sensing array to high level and the charge transfer signal Vtx to low level through the control signal, and a group of row selection switches Sel corresponding to the pixel calculation units in the adjacent three rows are all opened.

[0114] As another example, in the first related sampling stage, the control module sets the reset signal Vrst and the charge transfer signal Vtx of the pixel calculation units of the three adjacent rows of the photoelectric sensing array to a low level through the control signal, and a group of row selection switches Sel corresponding to the pixel calculation units of the three adjacent rows are all turned on, and the CDS S / H signal is set to a high level.

[0115] As another example, in the charge transfer stage, the control module sets the reset signal Vrst of the pixel computing units in three adjacent rows of the photoelectric sensing array to a low level and the charge transfer signal Vtx to a high level through a control signal. A group of row selection switches Sel corresponding to the pixel computing units in the three adjacent rows are all turned on, and the CDS S / H signal is set to a low level.

[0116] As another example, in the second correlation sampling stage, the control module sets the reset signal Vrst and the charge transfer signal Vtx of the pixel calculation units of the three adjacent rows of the photoelectric sensing array to a low level through the control signal, and a group of row selection switches Sel corresponding to the pixel calculation units of the three adjacent rows are all turned on, and the CDS S / H signal is set to a high level.

[0117] When ADC S / H is high, the charge stored in capacitor C1 in the readout circuit is transferred to the analog-to-digital converter (ADC) in the next stage. The A / D signal represents the time taken for the analog-to-digital conversion process; the OUTPUT signal represents the time required for the ADC output signal to reach the next stage, such as memory, an FPGA, or a PC.

[0118] like Figure 7 As shown, the control signal timing generates three pipelines, and the duration of each pipeline level is the same, which is 7 clock cycles.

[0119] Among them, the first-level pipeline includes the charge transfer area reset phase corresponding to the control signal of the row select switch of the j-th row of the photoelectric sensing array, the first related sampling phase, the charge transfer phase, the second related sampling phase, and the time for the signal of the readout circuit to be transferred to the analog-to-digital converter of the next level; the second-level pipeline includes the analog-to-digital conversion processing time of the j-th row information of the photoelectric sensing array; the third-level pipeline includes the time for the digital signal of the j-th row of the photoelectric sensing array to be output to the next-level device.

[0120] Figure 8 The figure is a schematic diagram showing the relationship between weight analog voltage output and digital weight of a weight voltage generating circuit according to an exemplary embodiment.

[0121] like Figure 8As shown, the weight analog voltage Vout output by the weight voltage generating circuit changes linearly with the digital weight input, and the variation range of the weight analog voltage is limited to 2.1V~2.7V to ensure that the calculation tubes of the pixel calculation unit all operate in the linear region.

[0122] Figure 9 The diagram shows a relationship between a first current representing a positive value output by a 5T pixel calculation unit, optical power, and a digital weight corresponding to a first positive weight voltage of a gate of a MOS tube M4 according to an exemplary embodiment.

[0123] like Figure 9 As shown in the figure, when the optical power is less than 3nW, the relationship between the output current Ip1 of the 5T pixel calculation unit and the product of the optical power and the digital weight is approximately linear, verifying the feasibility of "light intensity × weight" in the pixel calculation unit.

[0124] Adjusting the width-to-length ratio of the MOS tube in the pixel computing unit can achieve a higher linear relationship between the output current of the 5T pixel computing unit and the product of the optical power and the digital weight. However, in order to increase the fill factor of the pixel in the pixel computing unit (photosensitive area / pixel computing unit area) and improve image quality, the width-to-length ratio of all MOS tubes needs to be reduced as much as possible to save area, so some nonlinearity is retained.

[0125] When the optical power is greater than 4nW, the output current reaches saturation and no longer increases with the increase of optical power.

[0126] In a possible embodiment, the present disclosure provides a CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units combined with a convolutional neural network algorithm, which can be applied to image recognition and image processing, and has the characteristics of fast speed, high energy efficiency, and high integration.

[0127] The above describes specific embodiments of the present disclosure. It should be understood that the present disclosure is not limited to the specific embodiments described above, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present disclosure. The above preferred features may be used in any combination as long as they do not conflict with each other.

Claims

1. A CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units, characterized in that: include: Photoelectric sensing array, weight voltage generation module, readout circuit and control module; wherein, The photoelectric sensing array includes multiple groups of hybrid pixel computing units and multiple row selection switches. The types of pixel computing units in the hybrid pixel computing units include 5T, 8T, and 14T. Each group of the hybrid pixel computing units includes a first current bus and a second current bus. The photoelectric sensing array is used to convert the light intensity signal into a current signal, perform convolution multiplication and accumulation operations with the weight voltage of the convolution kernel through the pixel computing unit, and output current through the first current bus and the second current bus; The weight voltage generation module includes a plurality of weight voltage generation circuits, each of which is used to generate a weight voltage corresponding to the convolution kernel and provide weight information to the global photoelectric sensing array; The readout circuit is coupled to the first current bus and the second current bus of the photoelectric sensing array for subtraction operation, current-voltage conversion and correlated double sampling; The control module is coupled to the hybrid pixel calculation unit of the photoelectric sensing array and the row selection switch, and is used to generate a control signal; Each of the mixed pixel calculation units is arranged based on a convolution algorithm, with a 5T pixel calculation unit being arranged at the center of each of the convolution kernels, an 8T pixel calculation unit being arranged at the left and right positions and the upper and lower positions of each of the convolution kernels, and a 14T pixel calculation unit being arranged at the upper left position, the lower left position, the upper right position, and the lower right position of the center position of each of the convolution kernels; The left and right positions or the upper and lower positions of every two convolution kernels overlap with each other, and the lower right, lower left, upper left and upper right positions of every four convolution kernels overlap with each other.

2. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: The 8T pixel calculation unit is used to perform 2 parallel multiplications and output, and the 14T pixel calculation unit is used to perform 4 parallel multiplications and output.

3. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: The 5T pixel calculation unit includes MOS transistors M1, M2, M3, M4, M5, and a photodiode PD1. The gate of the MOS transistor M1 is connected to the row line of the pixel reset signal, the drain of the MOS transistor M1 is connected to the power supply, the source of the MOS transistor M1 is connected to the drain of the MOS transistor M2, the gate of the MOS transistor M2 is connected to the row line of the charge transfer signal, the source of the MOS transistor M2 is connected to the cathode of the photodiode PD1, and the anode of the photodiode PD1 is grounded. The gate of the MOS transistor M3 is connected to the drain of the MOS transistor M2, the drain of the MOS transistor M3 is connected to the power supply, the source of the MOS transistor M3 is connected to the drain of the MOS transistor M4 and the drain of the MOS transistor M5 respectively, the gate of the MOS transistor M4 is connected to a first positive weight voltage, the source of the MOS transistor M4 outputs a first current representing a positive value, the gate of the MOS transistor M5 is connected to a first negative weight voltage, and the source of the MOS transistor M5 outputs a first current representing a negative value.

4. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 3, characterized in that: The 8T pixel calculation unit includes MOS transistors M6, M7, M8, M9, M10, M11, M12, M13 and a photodiode PD2. The gate of the MOS transistor M6 is connected to the row line of the pixel reset signal, the drain of the MOS transistor M6 is connected to the power supply, the source of the MOS transistor M6 is connected to the drain of the MOS transistor M7, the gate of the MOS transistor M7 is connected to the row line of the charge transfer signal, the source of the MOS transistor M7 is connected to the cathode of the photodiode PD2, the anode of the photodiode PD2 is grounded, the gate of the MOS transistor M8 is connected to the drain of the MOS transistor M7, the drain of the MOS transistor M8 is connected to the power supply, and the source of the MOS transistor M8 is connected to the row line of the charge transfer signal. M9 and the drain of the MOS transistor M10 are connected, the gate of the MOS transistor M9 is connected to the second positive weight voltage, the source of the MOS transistor M9 outputs a second current representing a positive value, the gate of the MOS transistor M10 is connected to the second negative weight voltage, the source of the MOS transistor M10 outputs a second current representing a negative value, the gate of the MOS transistor M11 is connected to the drain of the MOS transistor M7, the drain of the MOS transistor M11 is connected to the power supply, the source of the MOS transistor M11 is connected to the drains of the MOS transistors M12 and M13 respectively, the gate of the MOS transistor M12 is connected to the third positive weight voltage, the source of the MOS transistor M12 outputs a third current representing a positive value, the gate of the MOS transistor M13 is connected to the third negative weight voltage, and the source of the MOS transistor M13 outputs a third current representing a negative value.

5. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 4, characterized in that: The 14T pixel calculation unit includes MOS transistors M14, M15, M16, M17, M18, M19, M20, M21, M22, M23, M24, M25, M26, M27 and a photodiode PD3. The gate of the MOS transistor M14 is connected to the row line of the pixel reset signal, the drain of the MOS transistor M14 is connected to the power supply, the source of the MOS transistor M14 is connected to the drain of the MOS transistor M15, the gate of the MOS transistor M15 is connected to the row line of the charge transfer signal, and the source of the MOS transistor M15 is connected to the row line of the charge transfer signal. The cathode of the photodiode PD3 is connected, the anode of the photodiode PD3 is grounded, the gate of the MOS tube M16 is connected to the drain of the MOS tube M15, the drain of the MOS tube M16 is connected to the power supply, the source of the MOS tube M16 is connected to the drain of the MOS tube M17 and the drain of the MOS tube M18 respectively, the gate of the MOS tube M17 is connected to the fourth positive weight voltage, the source of the MOS tube M17 outputs a fourth current representing a positive value, the gate of the MOS tube M18 is connected to the fourth negative weight voltage, the source of the MOS tube M18 outputs a fourth current representing a negative value, the drain of the MOS tube M19 is connected to the power supply, and the gate of the MOS tube M19 is connected to the drain of the MOS tube M19. The MOS transistor M15 is connected to the drain of the MOS transistor M19, the source of the MOS transistor M19 is respectively connected to the drain of the MOS transistor M20 and the drain of the MOS transistor M21, the gate of the MOS transistor M20 is connected to the fifth positive weight voltage, the source of the MOS transistor M20 outputs a fifth current representing a positive value, the gate of the MOS transistor M21 is connected to the fifth negative weight voltage, the source of the MOS transistor M21 outputs a fifth current representing a negative value, the drain of the MOS transistor M22 is connected to the power supply, the gate of the MOS transistor M22 is connected to the drain of the MOS transistor M15, the source of the MOS transistor M22 is respectively connected to the drain of the MOS transistor M23 and the drain of the MOS transistor M24, and the MOS transistor M The gate of the MOS transistor M23 is connected to the sixth positive weight voltage, the source of the MOS transistor M23 outputs a sixth current representing a positive value, the gate of the MOS transistor M24 is connected to the sixth negative weight voltage, the source of the MOS transistor M24 outputs a sixth current representing a negative value, the drain of the MOS transistor M25 is connected to the power supply, the gate of the MOS transistor M25 is connected to the drain of the MOS transistor M15, the source of the MOS transistor M25 is respectively connected to the drain of the MOS transistor M26 and the drain of the MOS transistor M27, the gate of the MOS transistor M26 is connected to the seventh positive weight voltage, the source of the MOS transistor M26 outputs a seventh current representing a positive value, and the gate of the MOS transistor M27 is connected to the seventh negative weight voltage.The source of the MOS tube M27 outputs a seventh current having a negative value.

6. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: The current output lines representing positive values ​​corresponding to the same convolution kernel are respectively connected to the first current bus corresponding to the same convolution kernel, and the current output lines representing negative values ​​corresponding to the same convolution kernel are respectively connected to the second current bus corresponding to the same convolution kernel. The first current bus outputs the current representing the positive value in the result of the convolution multiplication and addition operation, and the second current bus outputs the current representing the negative value in the result of the convolution multiplication and addition operation.

7. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: The weight voltage generating circuit includes a PMOS transistor M28, a MOS transistor M29, a PMOS transistor M30, a PMOS transistor M31, a PMOS transistor M32, a PMOS transistor M33, a MOS transistor M34, a MOS transistor M35, and an operational amplifier OP1. The source of the PMOS transistor M28 is connected to a power supply, the gate of the PMOS transistor M28 is connected to the gate and drain of the MOS transistor M29 respectively, the drain of the PMOS transistor M28 is connected to the gate and drain of the MOS transistor M29 respectively, the gate and drain of the PMOS transistor M28 and the gate and drain of the MOS transistor M29 are respectively connected to the positive phase input terminal of the operational amplifier OP1, the source of the MOS transistor M29 is grounded, the negative phase input terminal of the operational amplifier OP1 is connected to the output terminal of the operational amplifier OP1, the source of the PMOS transistor M30 is connected to the power supply, the gate of the PMOS transistor M30 is connected to the first weight digital signal, and the drain of the PMOS transistor M30 is connected to the MOS transistor M34. The drain of the MOS tube M34 is connected to the power supply, the gate of the MOS tube M34 is connected to the drain of the MOS tube M35, the gate of the MOS tube M35 is connected to the drain of the MOS tube M34, the source of the MOS tube M35 is connected to the output end of the operational amplifier OP1, the source of the PMOS tube M31 is connected to the power supply, the gate of the PMOS tube M31 is connected to the second weighted digital signal, and the drain of the PMOS tube M31 is connected to the The drain of the MOS tube M34 is connected, the source of the PMOS tube M32 is connected to the power supply, the gate of the PMOS tube M32 is connected to the third weighted digital signal, the drain of the PMOS tube M32 is connected to the drain of the MOS tube M34, the source of the PMOS tube M33 is connected to the power supply, the gate of the PMOS tube M33 is grounded, the drain of the PMOS tube M33 is connected to the drain of the MOS tube M34, and the gate of the MOS tube M35 outputs a weighted analog voltage.

8. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: The readout circuit includes an operational amplifier OP2, an operational amplifier OP3, an operational amplifier OP4, an operational amplifier OP5, a resistor R1, a resistor R2, a resistor R3, a resistor R4, a resistor R5, a resistor R6, a capacitor C1, a switch S1, a switch S2, a switch S3, a switch S4, a switch S5, and a switch S6. The positive input terminal of the operational amplifier OP2 is grounded, the negative input terminal of the operational amplifier OP2 is connected to the first current bus of the corresponding column, the input terminal of the resistor R1 is connected to the negative input terminal of the operational amplifier OP2, and the output terminal of the resistor R1 is connected to the positive input terminal of the operational amplifier OP2. The output terminal of the operational amplifier OP3 is connected, the positive input terminal of the operational amplifier OP3 is grounded, the negative input terminal of the operational amplifier OP3 is connected to the second current bus of the corresponding column, the input terminal of the resistor R2 is connected to the negative input terminal of the operational amplifier OP3, the output terminal of the resistor R2 is connected to the output terminal of the operational amplifier OP3, one end of the switch S1 is connected to the output terminal of the operational amplifier OP2, the other end of the switch S1 is connected to the input terminal of the resistor R3, one end of the switch S2 is connected to the output terminal of the operational amplifier OP3, and the other end of the switch S2 is connected to the input terminal of the resistor R3 The operational amplifier OP2 is connected to the output terminal of the switch S3, and the other end of the switch S3 is connected to the input terminal of the resistor R4. The output terminal of the resistor R3 is connected to the input terminal of the resistor R5 and the positive phase input terminal of the operational amplifier OP4 respectively. The output terminal of the resistor R4 is connected to the input terminal of the resistor R6 and the negative phase input terminal of the operational amplifier OP4 respectively. The output terminal of the resistor R5 is grounded. The output end of P4 and the output end of R6 are respectively connected to one end of the switch S5, the other end of the switch S5 is connected to the negative phase input end of the operational amplifier OP5, the positive phase input end of the operational amplifier OP5 is grounded, the input end of the capacitor C1 is connected to the negative phase input end of the operational amplifier OP5, the output end of the capacitor C1 is connected to the output end of the operational amplifier OP5, one end of the switch S6 is connected to the negative phase input end of the operational amplifier OP5, the other end of the switch S6 is connected to the output end of the operational amplifier OP5, and the output end of the operational amplifier OP5 outputs a voltage.

Images