CMOS image sensing and computing integrated circuit based on 5T, 8T and 14T mixed pixel computing unit
By designing a CMOS image sensing computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units in the CMOS image sensor, the problem of redundant data transmission and insufficient throughput of traditional CMOS image sensors is solved, and efficient parallel computing and output is achieved. Adapt to the needs of artificial intelligence algorithms.
Patent Information
- Application Number
- CN202411260538.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-10
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-09-10
AI Technical Summary
Traditional CMOS image sensors have problems such as increased energy consumption and delay caused by redundant data transmission, and fail to meet the high throughput requirements of artificial intelligence algorithms, lack of neural network algorithms - in-pixel parallel convolution architecture - collaborative design of analog domain computing circuits .
Design a CMOS image sensing computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units. Through the photoinductance computing array, weight voltage generation module and readout circuit, parallel calculation and output of the convolution kernel overlap position are realized, and dynamics are reduced. Loss, improve frame rate and computing efficiency.
It realizes a fast speed, high energy efficiency and high integration image sensing computing integrated circuit, reduces data transmission requirements, adapts to the high throughput requirements of artificial intelligence algorithms, and improves image processing efficiency.
Smart Images

Figure CN119031266A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of integrated circuit technology, and in particular, to a CMOS image sensing and computing integrated circuit based on a 5T, 8T, or 14T hybrid pixel computing unit. Background Art
[0002] CMOS Image Sensor (CIS) is the core of the camera module, which can efficiently convert the captured light signal into electrical signals and plays an indispensable role in many industries such as automation, intelligent monitoring, and medical diagnosis. On the other hand, convolution computing greatly simplifies the model structure with its local connection, weight sharing, and dimensionality reduction output characteristics, thereby effectively improving the recognition efficiency of tasks such as object detection, face recognition, image compression, and motion perception, making convolution computing a core technology widely used in the field of machine vision.
[0003] However, in the context of the rapid development of artificial intelligence algorithms, big data, the Internet of Things and other technologies, the problems and challenges faced by traditional CIS are becoming increasingly prominent. First, the sensing and computing units in the traditional von Neumann architecture are separated, and the unprocessed image data contains a lot of redundant information. Transmitting this data will lead to more energy consumption, further delays and additional hardware costs. Secondly, the readout method of pixels in traditional CMOS image sensors has the problem of slow speed, which is no longer suitable for the high throughput requirements of artificial intelligence algorithms. In addition, directly transmitting raw image information through the network is not conducive to the protection of personal privacy. Therefore, the development of CIS needs to keep up with the changes of the times, and it is a general trend to transform traditional CIS into an intelligent one. At present, there is a lack of research on the collaborative design of "neural network algorithm-in-pixel parallel convolution architecture-analog domain computing circuit".
[0004] Smart CIS integrates convolution operations directly into the CIS chip, which enables image processing to be completed inside the image sensor, effectively reducing data transmission requirements, increasing processing speed while reducing power consumption. This efficient "sensing-computing-in-one" machine vision system architecture shows broad development potential and commercial prospects in application scenarios such as autonomous driving, high-end security, virtual reality equipment (such as eye tracking and real-time processing functions), and other smart appliances (such as drones, sweeping robots, and translation pens).
[0005] The CMOS image sensing and computing integrated circuit currently used to implement on-chip convolution calculations has problems such as low weight accuracy and slow processing speed, and there is room for further optimization of power consumption. Specifically, existing related research requires time-sharing and multiple writing of weights in the area where the convolution kernels overlap, but cannot perform parallel calculation and output in the overlapping area of adjacent convolution kernels, resulting in additional processing time and energy consumption. Therefore, it is necessary to optimize the photoelectric sensing array architecture in order to design a more efficient CMOS image sensing and computing integrated circuit for on-chip convolution calculations. Summary of the invention
[0006] In view of the defects in the prior art, the purpose of the present invention is to provide a CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units.
[0007] To achieve the above-mentioned purpose, according to one aspect of the present disclosure, a CMOS image sensing and computing integrated circuit based on a 5T, 8T, 14T hybrid pixel computing unit is provided, comprising: a photoelectric sensing and computing array, a weight voltage generating module, a readout circuit and a control module; wherein,
[0008] The photoelectric sensing array includes multiple groups of hybrid pixel computing units and multiple row selection switches. The types of pixel computing units in the hybrid pixel computing units include 5T, 8T, and 14T. Each group of the hybrid pixel computing units includes a first current bus and a second current bus. The photoelectric sensing array is used to convert a light intensity signal into a current signal, perform a convolution multiplication and accumulation operation with a weight voltage of a convolution kernel through the pixel computing unit, and output a current through the first current bus and the second current bus;
[0009] The weight voltage generation module includes a plurality of groups of weight voltage generation circuits, and the weight voltage generation circuits are used to generate weight voltages corresponding to the convolution kernels and provide weight information to the global photoelectric sensing array;
[0010] The readout circuit is coupled to the first current bus and the second current bus of the photoelectric sensing array for subtraction operation, current-voltage conversion and correlated double sampling;
[0011] The control module is coupled to the hybrid pixel calculation unit of the photoelectric sensing array and the row selection switch, and the control module is used to generate a control signal.
[0012] Optionally, each of the mixed pixel calculation units is arranged based on a convolution algorithm, a 5T pixel calculation unit is set at the center position of each of the convolution kernels, an 8T pixel calculation unit is set at the left and right positions and the upper and lower positions of each of the convolution kernels, and a 14T pixel calculation unit is set at the upper left position, the lower left position, the upper right position and the lower right position of the center position of each of the convolution kernels.
[0013] Optionally, the left and right positions or the upper and lower positions of every two convolution kernels overlap with each other, and the lower right, lower left, upper left and upper right positions of every four convolution kernels overlap with each other.
[0014] Optionally, the 8T pixel computing unit is used to perform 2 parallel multiplications and output, and the 14T pixel computing unit is used to perform 4 parallel multiplications and output.
[0015] Optionally, the 5T pixel calculation unit includes a MOS tube M1, a MOS tube M2, a MOS tube M3, a MOS tube M4, a MOS tube M5 and a photodiode PD1, the gate of the MOS tube M1 is connected to the row line of the pixel reset signal, the drain of the MOS tube M1 is connected to the power supply, the source of the MOS tube M1 is connected to the drain of the MOS tube M2, the gate of the MOS tube M2 is connected to the row line of the charge transfer signal, the source of the MOS tube M2 is connected to the cathode of the photodiode PD1, and the photodiode The anode of the transistor PD1 is grounded, the gate of the MOS transistor M3 is connected to the drain of the MOS transistor M2, the drain of the MOS transistor M3 is connected to the power supply, the source of the MOS transistor M3 is respectively connected to the drain of the MOS transistor M4 and the drain of the MOS transistor M5, the gate of the MOS transistor M4 is connected to the first positive weight voltage, the source of the MOS transistor M4 outputs a first current representing a positive value, the gate of the MOS transistor M5 is connected to the first negative weight voltage, and the source of the MOS transistor M5 outputs a first current representing a negative value.
[0016] Optionally, the 8T pixel calculation unit includes a MOS tube M6, a MOS tube M7, a MOS tube M8, a MOS tube M9, a MOS tube M10, a MOS tube M11, a MOS tube M12, a MOS tube M13 and a photodiode PD2, the gate of the MOS tube M6 is connected to the row line of the pixel reset signal, the drain of the MOS tube M6 is connected to the power supply, the source of the MOS tube M6 is connected to the drain of the MOS tube M7, the gate of the MOS tube M7 is connected to the row line of the charge transfer signal, the source of the MOS tube M7 is connected to the cathode of the photodiode PD2, the anode of the photodiode PD2 is grounded, the gate of the MOS tube M8 is connected to the drain of the MOS tube M7, the drain of the MOS tube M8 is connected to the power supply, and the source of the MOS tube M8 is connected to the row line of the charge transfer signal. The MOS tube M9 and the drain of the MOS tube M10 are connected, the gate of the MOS tube M9 is connected to the second positive weight voltage, the source of the MOS tube M9 outputs a second current representing a positive value, the gate of the MOS tube M10 is connected to the second negative weight voltage, the source of the MOS tube M10 outputs a second current representing a negative value, the gate of the MOS tube M11 is connected to the drain of the MOS tube M7, the drain of the MOS tube M11 is connected to the power supply, the source of the MOS tube M11 is respectively connected to the drains of the MOS tube M12 and the MOS tube M13, the gate of the MOS tube M12 is connected to the third positive weight voltage, the source of the MOS tube M12 outputs a third current representing a positive value, the gate of the MOS tube M13 is connected to the third negative weight voltage, and the source of the MOS tube M13 outputs a third current representing a negative value.
[0017] Optionally, the 14T pixel calculation unit includes MOS tube M14, MOS tube M15, MOS tube M16, MOS tube M17, MOS tube M18, MOS tube M19, MOS tube M20, MOS tube M21, MOS tube M22, MOS tube M23, MOS tube M24, MOS tube M25, MOS tube M26, MOS tube M27 and photodiode PD3, the gate of the MOS tube M14 is connected to the row line of the pixel reset signal, the drain of the MOS tube M14 is connected to the power supply, the source of the MOS tube M14 is connected to the drain of the MOS tube M15, the gate of the MOS tube M15 is connected to the row line of the charge transfer signal, and the MOS The source of the MOS tube M15 is connected to the cathode of the photodiode PD3, the anode of the photodiode PD3 is grounded, the gate of the MOS tube M16 is connected to the drain of the MOS tube M15, the drain of the MOS tube M16 is connected to the power supply, the source of the MOS tube M16 is respectively connected to the drain of the MOS tube M17 and the drain of the MOS tube M18, the gate of the MOS tube M17 is connected to the fourth positive weight voltage, the source of the MOS tube M17 outputs a fourth current representing a positive value, the gate of the MOS tube M18 is connected to the fourth negative weight voltage, the source of the MOS tube M18 outputs a fourth current representing a negative value, and the drain of the MOS tube M19 is connected to the power supply. The gate of the MOS tube M19 is connected to the drain of the MOS tube M15, the source of the MOS tube M19 is respectively connected to the drains of the MOS tube M20 and the MOS tube M21, the gate of the MOS tube M20 is connected to the fifth positive weight voltage, the source of the MOS tube M20 outputs a fifth current representing a positive value, the gate of the MOS tube M21 is connected to the fifth negative weight voltage, the source of the MOS tube M21 outputs a fifth current representing a negative value, the drain of the MOS tube M22 is connected to the power supply, the gate of the MOS tube M22 is connected to the drain of the MOS tube M15, the source of the MOS tube M22 is respectively connected to the drain of the MOS tube M23, the gate of the MOS tube M21 and the fifth negative weight voltage. The drain of the MOS tube M24 is connected, the gate of the MOS tube M23 is connected to the sixth positive weight voltage, the source of the MOS tube M23 outputs a sixth current representing a positive value, the gate of the MOS tube M24 is connected to the sixth negative weight voltage, the source of the MOS tube M24 outputs a sixth current representing a negative value, the drain of the MOS tube M25 is connected to the power supply, the gate of the MOS tube M25 is connected to the drain of the MOS tube M15, the source of the MOS tube M25 is respectively connected to the drain of the MOS tube M26 and the drain of the MOS tube M27, the gate of the MOS tube M26 is connected to the seventh positive weight voltage, and the source of the MOS tube M26 outputs a seventh current representing a positive value,The gate of the MOS tube M27 is connected to the seventh negative weight voltage, and the source of the MOS tube M27 outputs a seventh current representing a negative value.
[0018] Optionally, the current output lines representing positive values corresponding to the same convolution kernel are respectively connected to the first current bus corresponding to the same convolution kernel, and the current output lines representing negative values corresponding to the same convolution kernel are respectively connected to the second current bus corresponding to the same convolution kernel, the first current bus outputs the current representing the positive value in the result of the convolution multiplication and addition operation, and the second current bus outputs the current representing the negative value in the result of the convolution multiplication and addition operation.
[0019] Optionally, the weight voltage generating circuit includes a PMOS tube M28, a MOS tube M29, a PMOS tube M30, a PMOS tube M31, a PMOS tube M32, a PMOS tube M33, a MOS tube M34, a MOS tube M35, and an operational amplifier OP1, the source of the MOS tube M28 is connected to the power supply, the gate of the PMOS tube M28 is respectively connected to the gate and drain of the MOS tube M29, and the drain of the PMOS tube M28 is respectively connected to the gate and drain of the MOS tube M29. The gate and drain of the PMOS tube M28 and the gate and drain of the MOS tube M29 are connected to the positive phase input terminal of the operational amplifier OP1 respectively, the source of the MOS tube M29 is grounded, the negative phase input terminal of the operational amplifier OP1 is connected to the output terminal of the operational amplifier OP1, the source of the PMOS tube M30 is connected to the power supply, the gate of the PMOS tube M30 is connected to the first weighted digital signal, the drain of the PMOS tube M30 is connected to the drain of the MOS tube M34, the gate of the MOS tube M34 is connected to the power supply, the source of the MOS tube M34 is connected to the drain of the MOS tube M35, the gate of the MOS tube M35 is connected to the drain of the MOS tube M34, the source of the MOS tube M35 is connected to the output terminal of the operational amplifier OP1, the source of the PMOS tube M31 is connected to the power supply, the gate of the PMOS tube M31 is connected to the second weighted digital signal, the drain of the PMOS tube M31 is connected to the The drain of the MOS tube M34 is connected, the source of the PMOS tube M32 is connected to the power supply, the gate of the PMOS tube M32 is connected to the third weighted digital signal, the drain of the PMOS tube M32 is connected to the drain of the MOS tube M34, the source of the PMOS tube M33 is connected to the power supply, the gate of the PMOS tube M33 is grounded, the drain of the PMOS tube M33 is connected to the drain of the MOS tube M34, and the gate of the MOS tube M35 outputs a weighted analog voltage.
[0020] Optionally, the readout circuit includes an operational amplifier OP2, an operational amplifier OP3, an operational amplifier OP4, an operational amplifier OP5, a resistor R1, a resistor R2, a resistor R3, a resistor R4, a resistor R5, a resistor R6, a capacitor C1, a switch S1, a switch S2, a switch S3, a switch S4, a switch S5, and a switch S6, wherein the positive phase input terminal of the operational amplifier OP2 is grounded, the negative phase input terminal of the operational amplifier OP2 is connected to the first current bus, the input terminal of the resistor R1 is connected to the negative phase input terminal of the operational amplifier OP2, and the output terminal of the resistor R1 is connected to the OP terminal of the operational amplifier. 2, the positive phase input terminal of the operational amplifier OP3 is grounded, the negative phase input terminal of the operational amplifier OP3 is connected to the second current bus, the input terminal of the resistor R2 is connected to the negative phase input terminal of the operational amplifier OP3, the output terminal of the resistor R3 is connected to the output terminal of the operational amplifier OP3, one end of the switch S1 is connected to the output terminal of the operational amplifier OP2, the other end of the switch S1 is connected to the input terminal of the resistor R3, one end of the switch S2 is connected to the output terminal of the operational amplifier OP3, the other end of the switch S2 is connected to the input terminal of the resistor R3 The operational amplifier OP2 is connected to the output terminal of the switch S3, the other end of the switch S3 is connected to the input terminal of the resistor R4, the output terminal of the resistor R3 is connected to the input terminal of the resistor R5 and the positive phase input terminal of the operational amplifier OP4, the output terminal of the resistor R4 is connected to the input terminal of the resistor R6 and the negative phase input terminal of the operational amplifier OP4, the output terminal of the resistor R5 is connected to the ground, and the operational amplifier OP4 is connected to the output terminal of the resistor R5. 5 and the output end of R6 are connected to one end of the switch S5 respectively, the other end of the switch S5 is connected to the negative phase input end of the operational amplifier OP5, the positive phase input end of the operational amplifier OP5 is grounded, the input end of the capacitor C1 is connected to the negative phase input end of the operational amplifier OP5, the output end of the capacitor C1 is connected to the output end of the operational amplifier OP5, one end of the switch S6 is connected to the negative phase input end of the operational amplifier OP5, the other end of the switch S6 is connected to the output end of the operational amplifier OP5, and the output end of the operational amplifier OP5 outputs a voltage.
[0021] Compared with the prior art, the embodiments of the present disclosure have at least one of the following beneficial effects:
[0022] Through the above calculation scheme, the CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units has the characteristics of fast speed, high energy efficiency, and high integration. It uses a weight voltage generation module to automatically allocate weight voltages for the photoelectric sensing array, and there is no need to rewrite weights at the convolution kernel overlapping positions, thereby reducing the dynamic loss of the circuit structure. 5T, 8T, and 14T pixel computing units are used in the photoelectric sensing array, and overlapping positions are formed to realize parallel calculation and output at the convolution kernel overlapping positions. The combined readout circuit realizes efficient parallel readout of the entire row, thereby improving the frame rate and computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Other features, objects and advantages of the present disclosure will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:
[0024] Figure 1 The present invention is a schematic diagram of the logic structure of a CMOS image sensing and computing integrated circuit based on a 5T, 8T, and 14T hybrid pixel computing unit according to an exemplary embodiment.
[0025] Figure 2 is a schematic diagram of a circuit structure of a 5T pixel calculation unit according to an exemplary embodiment.
[0026] Figure 3 is a schematic diagram of a circuit structure of an 8T pixel calculation unit according to an exemplary embodiment.
[0027] Figure 4 is a schematic diagram of a circuit structure of a 14T pixel calculation unit according to an exemplary embodiment.
[0028] Figure 5 is a circuit structure diagram of a weight voltage generating circuit according to an exemplary embodiment.
[0029] Figure 6 is a schematic diagram of a circuit structure of a readout circuit according to an exemplary embodiment.
[0030] Figure 7 The figure is a schematic diagram of voltage waveforms of a clock signal and a control signal of a control module according to an exemplary embodiment.
[0031] Figure 8 The diagram is a schematic diagram showing the relationship between a weight analog voltage output and a digital weight of a weight voltage generating circuit according to an exemplary embodiment.
[0032] Fig. 9The diagram is a relationship between a first current representing a positive value output by a 5T pixel calculation unit and the optical power and a digital weight corresponding to a first positive weight voltage of a gate of a MOS tube M4 according to an exemplary embodiment. DETAILED DESCRIPTION
[0033] The present disclosure is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present disclosure, but are not intended to limit the present disclosure in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present disclosure. These all fall within the scope of protection of the present disclosure.
[0034] Figure 1 The present invention is a schematic diagram of the logic structure of a CMOS image sensing and computing integrated circuit based on a 5T, 8T, and 14T hybrid pixel computing unit according to an exemplary embodiment.
[0035] like Figure 1 As shown, the present disclosure provides a CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units, including: a photoelectric sensing array, a weight voltage generation module, a readout circuit, and a control module.
[0036] The photoelectric sensing array includes multiple groups of hybrid pixel computing units and multiple row selection switches. The types of pixel computing units in the hybrid pixel computing units include 5T, 8T, and 14T. Each group of hybrid pixel computing units includes a first current bus and a second current bus. The photoelectric sensing array is used to convert light intensity signals into current signals, perform convolution multiplication and accumulation operations with the weight voltage of the convolution kernel through the pixel computing unit, and output current through the first current bus and the second current bus.
[0037] The first current bus is used to output a current representing a positive value of a result of a convolution-multiplication-accumulation operation, and the second current bus is used to output a current representing a negative value of a result of a convolution-multiplication-accumulation operation.
[0038] The photoelectric sensing array includes n×n pixel computing units and multiple row selection switches Sel(j), wherein the row selection switch Sel(j) represents the row selection switch Sel of the jth row. Each row selection switch controls the positive current and the negative current belonging to the same convolution kernel in the adjacent 3×3 pixel computing units to flow into the first current bus and the second current bus respectively. There are a total of The row selection switch is the photoelectric sensing array that executes 3×3 (stride=2) convolutions.
[0039] The weight voltage generation module includes multiple groups of weight voltage generation circuits, which are used to generate weight voltages corresponding to convolution kernels and provide weight information to the global photoelectric sensing array.
[0040] Among them, the weight voltage generation module includes 3×3×2 groups of weight voltage generation circuits, that is, 18 groups of weight voltage generation circuits, which are used to generate ±3bit weight voltages corresponding to the 3×3 convolution kernel, and the result of subtracting the positive weight voltage (≥0V) and the negative weight voltage (≥0V) is used to represent the true weight voltage corresponding to the convolution kernel.
[0041] The readout circuit is coupled to the first current bus and the second current bus of the photoelectric sensing array for subtraction operation, current-voltage conversion and Correlated Double Sampling (CDS).
[0042] The readout circuit is used to receive the current signal output by each group of mixed pixel computing units in the photoelectric sensing array, wherein the readout circuit receives the current representing a positive value output by the first current bus through a convolution multiplication and accumulation operation and the current representing a negative value output by the second current bus through a convolution multiplication and accumulation operation.
[0043] The readout circuit performs a subtraction operation on the received current representing a positive value and the current representing a negative value, obtains the output current of the complete convolution multiplication and accumulation operation, and determines the real current output by the convolution multiplication and accumulation operation.
[0044] The readout circuit performs current-to-voltage conversion and correlated double sampling on the real current output by the convolution multiplication and accumulation operation, and outputs the voltage.
[0045] The control module is coupled to the hybrid pixel computing unit and the row selection switch of the photoelectric sensing array, and is used to generate a control signal.
[0046] Among them, the control module includes a digital circuit, and the control module is used to provide a reset signal and a charge transfer signal for the pixel computing unit of each row of the photoelectric sensing array, and is also used to provide a row selection Sel signal for the row selection switch Sel(j) of each row of the photoelectric sensing array to ensure that the sensing, calculation, and output of the photoelectric sensing array are automatically performed row by row.
[0047] Through the above calculation scheme, the CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units has the characteristics of fast speed, high energy efficiency, and high integration. It uses a weight voltage generation module to automatically allocate weight voltages for the photoelectric sensing array, and there is no need to rewrite weights at the convolution kernel overlapping positions, thereby reducing the dynamic loss of the circuit structure. 5T, 8T, and 14T pixel computing units are used in the photoelectric sensing array, and overlapping positions are formed to realize parallel calculation and output at the convolution kernel overlapping positions. The combined readout circuit realizes efficient parallel readout of the entire row, thereby improving the frame rate and computing efficiency.
[0048] In a possible embodiment, the 5T pixel computing unit, the 8T pixel computing unit, and the 14T pixel computing unit in each hybrid pixel computing unit are arranged based on a 3×3 (stride=2) convolution algorithm.
[0049] like Figure 1 As shown, in a possible embodiment, each mixed pixel computing unit is arranged based on a convolution algorithm, a 5T pixel computing unit is set at the center of each convolution kernel, an 8T pixel computing unit is set at the left and right positions and the upper and lower positions of each convolution kernel, and a 14T pixel computing unit is set at the upper left position, the lower left position, the upper right position and the lower right position of the center position of each convolution kernel.
[0050] In the present disclosure, the left and right positions or the upper and lower positions of every two convolution kernels overlap with each other, and the lower right, lower left, upper left and upper right positions of every four convolution kernels overlap with each other.
[0051] As an example, a 5T pixel computing unit is located at the center of each convolution kernel, an 8T pixel computing unit is located at four positions where two convolution kernels overlap left and right or up and down, and a 14T pixel computing unit is located at a corner position where four convolution kernels overlap.
[0052] In a possible embodiment, an 8T pixel computing unit is used to perform 2 parallel multiplications and output, and a 14T pixel computing unit is used to perform 4 parallel multiplications and output.
[0053] Figure 2 is a schematic diagram of a circuit structure of a 5T pixel calculation unit according to an exemplary embodiment.
[0054] like Figure 2 As shown, in a possible embodiment, the 5T pixel computing unit includes a MOS transistor M1, a MOS transistor M2, a MOS transistor M3, a MOS transistor M4, a MOS transistor M5 and a photodiode PD1.
[0055] The gate of the MOS tube M1 is connected to the row line of the pixel reset signal, the drain of the MOS tube M1 is connected to the power supply, the source of the MOS tube M1 is connected to the drain of the MOS tube M2, the gate of the MOS tube M2 is connected to the row line of the charge transfer signal, the source of the MOS tube M2 is connected to the cathode of the photodiode PD1, the anode of the photodiode PD1 is grounded, the gate of the MOS tube M3 is connected to the drain of the MOS tube M2, the drain of the MOS tube M3 is connected to the power supply, the source of the MOS tube M3 is respectively connected to the drain of the MOS tube M4 and the drain of the MOS tube M5, the gate of the MOS tube M4 is connected to the first positive weight voltage, the source of the MOS tube M4 outputs a first current representing a positive value, the gate of the MOS tube M5 is connected to the first negative weight voltage, and the source of the MOS tube M5 outputs a first current representing a negative value.
[0056] The gate of the MOS transistor M1 is connected to the row line of the k-th row pixel reset signal Vrst(k), and the gate of the MOS transistor M2 is connected to the row line Vtx(k) of the k-th row charge transfer signal. The gate of the MOS transistor M4 is connected to the first positive weight voltage Wp1, and the source of the calculation tube M4 outputs a positive first current Ip1. The gate of the MOS transistor M5 is connected to the first negative weight voltage Wn1, and the source of the calculation tube M5 outputs a negative first current In1.
[0057] In the present disclosure, in the circuit structure of the 5T pixel computing unit, MOS tube M1 is represented as a reset transistor M1, MOS tube M2 is represented as a charge transfer tube M2, MOS tube M3 is represented as a source follower tube M3, MOS tube M4 is represented as a computing tube M4, and MOS tube M5 is represented as a computing tube M5.
[0058] Figure 3 is a schematic diagram of a circuit structure of an 8T pixel calculation unit according to an exemplary embodiment.
[0059] like Figure 3 As shown, in a possible embodiment, the 8T pixel computing unit includes a MOS tube M6, a MOS tube M7, a MOS tube M8, a MOS tube M9, a MOS tube M10, a MOS tube M11, a MOS tube M12, a MOS tube M13 and a photodiode PD2.
[0060] The gate of the MOS tube M6 is connected to the row line of the pixel reset signal, the drain of the MOS tube M6 is connected to the power supply, the source of the MOS tube M6 is connected to the drain of the MOS tube M7, the gate of the MOS tube M7 is connected to the row line of the charge transfer signal, the source of the MOS tube M7 is connected to the cathode of the photodiode PD2, the anode of the photodiode PD2 is grounded, the gate of the MOS tube M8 is connected to the drain of the MOS tube M7, the drain of the MOS tube M8 is connected to the power supply, the source of the MOS tube M8 is respectively connected to the drains of the MOS tubes M9 and M10, the gate of the MOS tube M9 is connected to the second positive weight voltage, and the The source output represents a positive second current, the gate of the MOS tube M10 is connected to the second negative weight voltage, the source output of the MOS tube M10 represents a negative second current, the gate of the MOS tube M11 is connected to the drain of the MOS tube M7, the drain of the MOS tube M11 is connected to the power supply, the source of the MOS tube M11 is respectively connected to the drains of the MOS tube M12 and the MOS tube M13, the gate of the MOS tube M12 is connected to the third positive weight voltage, the source output of the MOS tube M12 represents a positive third current, the gate of the MOS tube M13 is connected to the third negative weight voltage, and the source output of the MOS tube M13 represents a negative third current.
[0061] Among them, the gate of MOS tube M6 is connected to the row line of Vrst(k) of the pixel reset signal of the kth row, and the gate of MOS tube M7 is connected to the row line Vtx(k) of the charge transfer signal of the kth row. The gate of MOS tube M9 is connected to the second positive weight voltage Wp2, and the source output of MOS tube M9 represents the second current Ip2 with a positive value. The gate of MOS tube M10 is connected to the second negative weight voltage Wn2, and the source output of MOS tube M10 represents the second current In2 with a negative value. The gate of MOS tube M12 is connected to the third positive weight voltage Wp3, and the source output of MOS tube M12 represents the third current Ip3 with a positive value. The gate of MOS tube M13 is connected to the third negative weight voltage Wn3, and the source output of MOS tube M13 represents the third current In3 with a negative value.
[0062] In the present disclosure, in the circuit structure of the 8T pixel computing unit, MOS tube M6 represents reset transistor M6, MOS tube M7 represents pixel transfer tube M7, MOS tube M8 represents source follower tube M8, MOS tube M9 represents computing tube M9, MOS tube M10 represents computing tube M10, MOS tube M11 represents source follower M11, MOS tube M12 represents computing tube M12, and MOS tube M13 represents computing tube M13.
[0063] Figure 4 is a schematic diagram of a circuit structure of a 14T pixel calculation unit according to an exemplary embodiment.
[0064] like Figure 4 As shown, in a possible embodiment, the 14T pixel computing unit includes MOS tube M14, MOS tube M15, MOS tube M16, MOS tube M17, MOS tube M18, MOS tube M19, MOS tube M20, MOS tube M21, MOS tube M22, MOS tube M23, MOS tube M24, MOS tube M25, MOS tube M26, MOS tube M27 and photodiode PD3.
[0065] The gate of the MOS tube M14 is connected to the row line of the pixel reset signal, the drain of the MOS tube M14 is connected to the power supply, the source of the MOS tube M14 is connected to the drain of the MOS tube M15, the gate of the MOS tube M15 is connected to the row line of the charge transfer signal, the source of the MOS tube M15 is connected to the cathode of the photodiode PD3, the anode of the photodiode PD3 is grounded, the gate of the MOS tube M16 is connected to the drain of the MOS tube M15, the drain of the MOS tube M16 is connected to the power supply, the source of the MOS tube M16 is connected to the drain of the MOS tube M17 and the drain of the MOS tube M17, respectively. The drain of the MOS tube M18 is connected to the drain of the MOS tube M17, the gate of the MOS tube M17 is connected to the fourth positive weight voltage, the source of the MOS tube M17 outputs a fourth current representing a positive value, the gate of the MOS tube M18 is connected to the fourth negative weight voltage, the source of the MOS tube M18 outputs a fourth current representing a negative value, the drain of the MOS tube M19 is connected to the power supply, the gate of the MOS tube M19 is connected to the drain of the MOS tube M15, the source of the MOS tube M19 is respectively connected to the drains of the MOS tubes M20 and M21, the gate of the MOS tube M20 is connected to the fifth positive weight voltage, and the MOS The source output of the MOS tube M20 represents a fifth current with a positive value, the gate of the MOS tube M21 is connected to the fifth negative weight voltage, the source output of the MOS tube M21 represents a fifth current with a negative value, the drain of the MOS tube M22 is connected to the power supply, the gate of the MOS tube M22 is connected to the drain of the MOS tube M15, the source of the MOS tube M22 is respectively connected to the drain of the MOS tube M23 and the drain of the MOS tube M24, the gate of the MOS tube M23 is connected to the sixth positive weight voltage, the source output of the MOS tube M23 represents a sixth current with a positive value, the gate of the MOS tube M24 is connected to the sixth positive weight voltage, The sixth negative weight voltage is connected, the source of the MOS tube M24 outputs a sixth current representing a negative value, the drain of the MOS tube M25 is connected to the power supply, the gate of the MOS tube M25 is connected to the drain of the MOS tube M15, the source of the MOS tube M25 is respectively connected to the drain of the MOS tube M26 and the drain of the MOS tube M27, the gate of the MOS tube M26 is connected to the seventh positive weight voltage, the source of the MOS tube M26 outputs a seventh current representing a positive value, the gate of the MOS tube M27 is connected to the seventh negative weight voltage, and the source of the MOS tube M27 outputs a seventh current representing a negative value.
[0066] Among them, the gate of MOS tube M14 is connected to the row line of the pixel reset signal Vrst(k) of the kth row, and the gate of MOS tube M15 is connected to the row line of the charge transfer signal Vtx(k) of the kth row. The gate of MOS tube M17 is connected to the fourth positive weight voltage Wp4, and the source output of MOS tube M17 represents the fourth current Ip4 with a positive value. The gate of MOS tube M18 is connected to the fourth negative weight voltage Wn4, and the source output of MOS tube M18 represents the fourth current In4 with a negative value. The gate of MOS tube M20 is connected to the fifth positive weight voltage Wp5, and the source output of MOS tube M20 represents the fifth current Ip5 with a positive value. The gate of MOS tube M21 is connected to the fifth negative weight voltage Wn5, and the source output of MOS tube M21 represents the fifth current In5 with a negative value. The gate of the MOS tube M23 is connected to the sixth positive weight voltage Wp6, and the source output of the MOS tube M23 represents a sixth current Ip6 with a positive value. The gate of the MOS tube M24 is connected to the sixth negative weight voltage Wn6, and the source output of the MOS tube M24 represents a sixth current In6 with a negative value. The gate of the MOS tube M26 is connected to the seventh positive weight voltage Wp7, and the source output of the MOS tube M26 represents a seventh current Ip7 with a positive value. The gate of the MOS tube M27 is connected to the seventh negative weight voltage Wn7, and the source output of the MOS tube M27 represents a seventh current In7 with a negative value.
[0067] In the present disclosure, in the circuit structure of the 14T pixel computing unit, MOS transistor M14 represents reset transistor M14, MOS transistor M15 represents pixel transfer transistor M15, MOS transistor M16 represents source follower M16, MOS transistor M17 represents computing tube M17, MOS transistor M18 represents computing tube M18, MOS transistor M19 represents source follower M19, MOS transistor M20 represents computing tube M20, MOS transistor M21 represents computing tube M21, MOS transistor M22 represents source follower M22, MOS transistor M23 represents computing tube M23, MOS transistor M24 represents computing tube M24, MOS transistor M25 represents source follower M25, MOS transistor M26 represents computing tube M26, and MOS transistor M27 represents computing tube M27.
[0068] In a possible embodiment, the current output lines representing positive values corresponding to the same convolution kernel are respectively connected to the first current bus corresponding to the same convolution kernel, and the current output lines representing negative values corresponding to the same convolution kernel are respectively connected to the second current bus corresponding to the same convolution kernel. The first current bus outputs the current representing the positive value in the result of the convolution multiplication and addition operation, and the second current bus outputs the current representing the negative value in the result of the convolution multiplication and addition operation.
[0069] A group of mixed pixel calculation units corresponding to each convolution kernel includes a group of buses, wherein the group of buses includes a first current bus and a second current bus.
[0070] In the present disclosure, 9 current output lines representing positive values belonging to the same convolution kernel are respectively connected to the first current bus belonging to the same convolution kernel, and 9 current output lines representing negative values belonging to the same convolution kernel are respectively connected to the second current bus belonging to the same convolution kernel, and the connection between the current output lines and the current buses is controlled by the row selection switch Sel.
[0071] As an example, corresponding to the same convolution kernel, the source of a MOS tube M4 of a 5T pixel computing unit, the source of a MOS tube M9 of two 8T pixel computing units, the source of a MOS tube M12 of two 8T pixel computing units, the source of a MOS tube M17 of a 14T pixel computing unit, the source of a MOS tube M20 of a 14T pixel computing unit, the source of a MOS tube M23 of a 14T pixel computing unit, and the source of a MOS tube M26 of a 14T pixel computing unit are respectively connected to the first current bus as current output lines representing positive values. The source of the MOS tube M5 of a 5T pixel calculation unit corresponding to the same convolution kernel, the source of the MOS tube M10 of two 8T pixel calculation units, the source of the MOS tube M13 of two 8T pixel calculation units, the source of the MOS tube M18 of a 14T pixel calculation unit, the source of the MOS tube M21 of a 14T pixel calculation unit, the source of the MOS tube M24 of a 14T pixel calculation unit, and the source of the MOS tube M27 of a 14T pixel calculation unit are respectively connected to the second current bus as current output lines representing negative values. The positive current bus outputs the positive current of the convolution multiplication and accumulation operation, and the negative current bus outputs the negative current of the convolution multiplication and accumulation operation.
[0072] Among them, the source of the MOS tube M9 and the source of the MOS tube M12 of the same 8T pixel calculation unit are used as current output lines representing positive values and correspond to different convolution kernels respectively; the source of the MOS tube M10 and the source of the MOS tube M13 of the same 8T pixel calculation unit are used as current output lines representing negative values and correspond to different convolution kernels respectively. The source of the MOS tube M17, the source of the MOS tube M20, the source of the MOS tube M23, and the source of the MOS tube M26 of the same 14T pixel calculation unit are used as current output lines representing positive values and correspond to different convolution kernels respectively; the source of the MOS tube M18, the source of the MOS tube M21, the source of the MOS tube M24, and the source of the MOS tube M27 of the same 14T pixel calculation unit are used as current output lines representing negative values and correspond to different convolution kernels respectively.
[0073] like Figures 2 to 4As shown, in a possible embodiment, when the reset signal Vrst(k) and the charge transfer signal Vtx(k) are at a high level, the parasitic capacitance of the photodiode PD1, the photodiode PD2, and the photodiode PD3 is charged until the voltage of the photodiode PD1, the photodiode PD2, and the photodiode PD3 is close to the power supply voltage VDD, and the reset signal Vrst(k) and the charge transfer signal Vtx(k) are both set to a low level, and the parasitic capacitance of the photodiode PD1, the photodiode PD2, and the photodiode PD3 is stopped from being charged.
[0074] When the pixel computing unit of the photoelectric sensing array is exposed to a preset light intensity, the current flowing through the photodiode PD1, the photodiode PD2, and the photodiode PD3 is the photocurrent, which is proportional to the light intensity of the incident light. The parasitic capacitance inside the photodiode PD1, the photodiode PD2, and the photodiode PD3 is discharged, and the voltage across the two ends of each photodiode PD1, the photodiode PD2, and the photodiode PD3 is reduced.
[0075] Keep the reset signal Vrst(k) at a low level and set the charge transfer signal Vtx(k) to a high level to clear the electrons that leak from the cathodes of photodiode PD1, photodiode PD2, and photodiode PD3 through the transfer tube to the charge transfer region of the transfer tube during the exposure process. At this time, the voltage of the charge transfer region is Vr.
[0076] For example, the cathode of the photodiode PD1 leaks electrons to the drain of the MOS tube M2 through the MOS tube M2; the cathode of the photodiode PD2 leaks electrons to the drain of the MOS tube M7 through the MOS tube M7; the cathode of the photodiode PD3 leaks electrons to the drain of the MOS tube M15 through the MOS tube M15.
[0077] The reset signal Vrst(k) and the charge transfer signal Vtx(k) are both set to low level, and the voltage Vr of the charge transfer region is sampled, that is, the voltage of the drain of the MOS tube M2, the MOS tube M7, and the MOS tube M15 is sampled. The voltage Vr of the charge transfer region is output to the readout circuit through the source follower tube and the calculation tube. The readout circuit performs a correlated double sampling (CDS) operation, and the voltage Vr is the first sampling signal of the correlated double sampling of the readout circuit.
[0078] After sampling the voltage Vr, the charge transfer stage is entered, the reset signal Vrst(k) is kept at a low level, and the charge transfer signal Vtx(k) is set to a high level. Electrons leak from the cathodes of the photodiodes PD1, PD2, and PD3 through the transfer tubes to the charge transfer regions of their transfer tubes.
[0079] The reset signal Vrst(k) and the charge transfer signal Vtx(k) are both set to low level, and the voltage Vr of the charge transfer region is sampled. Based on the functions of the source follower tube M3, the source follower tube M8, the source follower tube M11, the source follower tube M16, the source follower tube M19, the source follower tube M22, and the source follower tube M25, the voltage change of the source of the source follower tube M3, the source follower tube M8, the source follower tube M11, the source follower tube M16, the source follower tube M19, the source follower tube M22, and the source follower tube M25 follows the voltage Vr of the charge transfer region, and the source voltage of each source follower tube is represented by the signal Vs. At this time, computing tube M4, computing tube M5, computing tube M9, computing tube M10, computing tube M12, computing tube M13, computing tube M17, computing tube M18, computing tube M20, computing tube M21, computing tube M23, computing tube M24, computing tube M26, and computing tube M27 operate in the linear region, and the positive voltage Wp or negative voltage Wn of their gate voltage comes from the weighted analog voltage Wout output by the weight voltage generating circuit, and the difference between the gate voltages of the two computing tubes connected to the same source follower tube is used as the weight, that is, (Wp-Wn).
[0080] The voltage formula of MOS tube working in the linear region is:
[0081]
[0082]
[0083] (Ip-In)=2Kn[(Wp-Wn)Vs]→ΔI=2Kn*Weight*Vs.
[0084] Wherein, (Ip-In)=2Kn*Weight*Vs represents the result of multiplying the light information by the weight.
[0085] In the present disclosure, the 8T pixel computing unit outputs in parallel (Ip1-In1)=2Kn*Weight1*Vs and (Ip2-In2)=2Kn*Weight2*Vs; the 14T pixel computing unit outputs in parallel (Ip1-In1)=2Kn*Weight1*Vs, (IP2-In2)=2Kn*Weight2*Vs, (Ip3-In3)=2Kn*Weight3*Vs and (Ip4-In4)=2Kn*Weight4*Vs.
[0086] Figure 5 is a circuit structure diagram of a weight voltage generating circuit according to an exemplary embodiment.
[0087] like Figure 5As shown, in a possible embodiment, the weight voltage generating circuit includes a PMOS tube M28, a MOS tube M29, a PMOS tube M30, a PMOS tube M31, a PMOS tube M32, a PMOS tube M33, a MOS tube M34, a MOS tube M35, and an operational amplifier OP1, the source of the MOS tube M28 is connected to the power supply, the gate of the PMOS tube M28 is respectively connected to the gate and drain of the MOS tube M29, the drain of the PMOS tube M28 is respectively connected to the gate and drain of the MOS tube M29, the gate and drain of the PMOS tube M28 and the gate and drain of the MOS tube M29 are respectively connected to the positive phase input terminal of the operational amplifier OP1, the source of the MOS tube M29 is grounded, the negative phase input terminal of the operational amplifier OP1 is connected to the output terminal of the operational amplifier OP1, the source of the PMOS tube M30 is connected to the power supply, the gate of the PMOS tube M30 is connected to the first weight digital signal, and the PMOS tube The drain of PMOS tube M30 is connected to the drain of MOS tube M34, the gate of MOS tube M34 is connected to the power supply, the source of MOS tube M34 is connected to the drain of MOS tube M35, the gate of MOS tube M35 is connected to the drain of MOS tube M34, the source of MOS tube M35 is connected to the output end of operational amplifier OP1, the source of PMOS tube M31 is connected to the power supply, the gate of PMOS tube M31 is connected to the second weighted digital signal, the drain of PMOS tube M31 is connected to the drain of MOS tube M34, the source of PMOS tube M32 is connected to the power supply, the gate of PMOS tube M32 is connected to the third weighted digital signal, the drain of PMOS tube M32 is connected to the drain of MOS tube M34, the source of PMOS tube M33 is connected to the power supply, the gate of PMOS tube M33 is grounded, the drain of PMOS tube M33 is connected to the drain of MOS tube M34, and the gate of MOS tube M35 outputs the weighted analog voltage.
[0088] like Figure 3 As shown, in a possible embodiment, in the weight voltage generating circuit, the operational amplifier OP1 is connected in a voltage follower manner, and the PMOS tube M28, MOS tube M29 and operational amplifier OP1 are used to provide a static bias voltage for the source of the MOS tube M35, and the minimum value of the weight analog voltage Wout output by the weight voltage generating circuit is a preset value.
[0089] The width-to-length ratio W / L of the PMOS tube M30, PMOS tube M31, and PMOS tube M32 is 1:2:4. The sources of the PMOS tube M30, PMOS tube M31, and PMOS tube M32 are respectively connected to the power supply VDD, and their drains are respectively connected. Their gates receive digital weight 3-bit inputs, respectively, the highest bit is Win3, the middle bit is Win2, and the lowest bit is Win1. The digital input low level is valid.
[0090] The PMOS tube M33 is connected in parallel with the PMOS tube M30, the PMOS tube M31 and the PMOS tube M32. The gate of the PMOS tube M33 is grounded. The PMOS tube M33 is used to provide bias. When the digital input is 3'b0, the weighted analog voltage output by the weighted voltage generating circuit is the minimum voltage within the output swing of the weighted voltage generating circuit, rather than 0V.
[0091] In the MOS tube M34 , the resistance of the MOS tube M34 can be adjusted by changing the aspect ratio of the MOS tube M34 , and the linearity between the output range of the output weighted analog voltage Wout and the digital input can be adjusted by the principle of resistor negative feedback.
[0092] The gate of the MOS tube M35 outputs a weighted analog voltage Wout.
[0093] Figure 6 is a schematic diagram of a circuit structure of a readout circuit according to an exemplary embodiment.
[0094] like Figure 6As shown, in a possible embodiment, the readout circuit includes an operational amplifier OP2, an operational amplifier OP3, an operational amplifier OP4, an operational amplifier OP5, a resistor R1, a resistor R2, a resistor R3, a resistor R4, a resistor R5, a resistor R6, a capacitor C1, a switch S1, a switch S2, a switch S3, a switch S4, a switch S5, and a switch S6, the positive phase input terminal of the operational amplifier OP2 is grounded, the negative phase input terminal of the operational amplifier OP2 is connected to the first current bus, the input terminal of the resistor R1 is connected to the negative phase input terminal of the operational amplifier OP2 The output end of the resistor R1 is connected to the output end of the operational amplifier OP2, the positive phase input end of the operational amplifier OP3 is grounded, the negative phase input end of the operational amplifier OP3 is connected to the second current bus, the input end of the resistor R2 is connected to the negative phase input end of the operational amplifier OP3, the output end of the resistor R3 is connected to the output end of the operational amplifier OP3, one end of the switch S1 is connected to the output end of the operational amplifier OP2, the other end of the switch S1 is connected to the input end of the resistor R3, one end of the switch S2 is connected to the output end of the operational amplifier OP3, and the switch S2 The other end of is connected to the input end of the resistor R3, one end of the switch S3 is connected to the output end of the operational amplifier OP2, the other end of the switch S3 is connected to the input end of the resistor R4, one end of the switch S4 is connected to the output end of the operational amplifier OP3, the other end of the switch S4 is connected to the input end of the resistor R4, the output end of the resistor R3 is respectively connected to the input end of the resistor R5 and the positive phase input end of the operational amplifier OP4, the output end of the resistor R4 is respectively connected to the input end of the resistor R6 and the negative phase input end of the operational amplifier OP4, the output end of the resistor R5 is grounded, the output end of the operational amplifier OP5 and the output end of R6 are respectively connected to one end of the switch S5, the other end of the switch S5 is connected to the negative phase input end of the operational amplifier OP5, the positive phase input end of the operational amplifier OP5 is grounded, the input end of the capacitor C1 is connected to the negative phase input end of the operational amplifier OP5, the output end of the capacitor C1 is connected to the output end of the operational amplifier OP5, one end of the switch S6 is connected to the negative phase input end of the operational amplifier OP5, the other end of the switch S6 is connected to the output end of the operational amplifier OP5, and the output end of the operational amplifier OP5 outputs a voltage.
[0095] like Figure 6As shown, in the circuit structure of the readout circuit, the negative phase input terminal of the operational amplifier OP2 receives the current Ioutp(i) outputted from the first current bus of the i-th column of the photoelectric sensing array, and the operational amplifier OP2 and the resistor R1 are used to convert the current Ioutp(i) into a voltage outputted from the output terminal of the operational amplifier OP2. The negative phase input terminal of the operational amplifier OP3 receives the current Ioutn(i) outputted from the second current bus of the i-th column of the photoelectric sensing array, and the operational amplifier OP3 and the resistor R2 are used to convert the current Ioutn(i) into a voltage outputted from the output terminal of the operational amplifier OP3. Wherein, R1=R2=R.
[0096] The operational amplifier OP4, the resistor R3, the resistor R4, the resistor R5, and the resistor R6 form a subtraction circuit, and R3=R4=R5=R6.
[0097] As an example, when the switches S1 and S4 are closed and the switches S2 and S3 are opened, the output voltage of the operational amplifier OP4 satisfies Vm=R*Ioutp-R*Ioutn=R*(Ip-In)=2RKn*Weight*Vs.
[0098] As another example, when the switches S2 and S3 are closed and the switches S1 and S4 are open, the output voltage of the operational amplifier OP4 satisfies Vm=R*Ioutn-R*Ioutp=R*(In-Ip)=-2RKn*Weight*Vs.
[0099] Operational amplifier OP5, switch S5, sampling capacitor C1, and switch S6 are used for a sample and hold circuit.
[0100] As an example, when the switch S5 is closed and the switch S6 is open, it is a sampling stage, and the output voltage of the operational amplifier OP4 charges the capacitor C1.
[0101] As another example, when the switches S5 and S6 are disconnected, it is the holding stage, the capacitor C1 is in the holding stage, and the output voltage of the operational amplifier OP5 remains unchanged.
[0102] As another example, when the switch S5 is opened and the switch S6 is closed, it is a reset stage, the capacitor C1 is discharged, and the voltage across the capacitor C1 becomes 0V, preparing for the multiplication and accumulation sampling of the next row of the photoelectric sensing array.
[0103] In one possible embodiment, a readout circuit is used to perform correlated double sampling (CDS):
[0104] The output of the differential circuit on a bus corresponding to the photoelectric sensing array at the first relevant sampling is (Irp-Irn), and the voltage at the output of the operational amplifier OP4 is Vmr. The output of the differential circuit on the same bus corresponding to the second relevant sampling is (Isp-Isn), and the voltage at the output of the operational amplifier OP4 is Vms. The time length of the two samplings is T.
[0105] Based on correlated double sampling (CDS), at the first correlated sampling, switches S1 and S4 are closed, switches S2 and S3 are open, and Vmr = R*(Irp-Irn). At the second correlated sampling, switches S1 and S4 are open, switches S2 and S3 are closed, and Vms = R*(Isn-Isp) = -R*(Isp-Isn).
[0106] The capacitor C1 is charged during both samplings, and the amount of charge is expressed as C=T*Vmr+T*Vms=T*(Vmr+Vms)=T*[(Irp-Irn)+(Isn-Isp)]=T*[(Irp-Irn)-(Isp-Isn)], thereby realizing correlated double sampling. There is no need to switch the state of the capacitor from charging to discharging during the second correlated sampling, thereby simplifying the circuit structure.
[0107] Figure 7 The figure is a schematic diagram of voltage waveforms of a clock signal and a control signal of a control module according to an exemplary embodiment.
[0108] like Figure 7 As shown, in a possible embodiment, in the control module, CLK represents the digital clock of the control module. Sel(j) represents the control signal of the row selection switch Sel for controlling the connection between the pixel calculation unit of the jth row of the photoelectric sensing array and the current bus; Vrst(k) represents the control signal for controlling the reset signal of the kth row of the photoelectric sensing array; Vtx(k) represents the control signal for controlling the charge transfer signal of the kth row of the photoelectric sensing array; CDS S / H represents the control signal for controlling the correlated double sampling of the readout circuit, that is, the control signal for the opening time length of the two samplings of the switch S5.
[0109] In the photoelectric sensing array, a row selection switch Sel can be set every other row, and the sorting behavior of the row selection switch Sel is the jth row, 1≤j≤(n-1) / 2; a reset signal row line Vrst and a charge transfer signal Vtx can be set in each row, and the sorting behavior of the reset signal row line Vrst and the charge transfer signal Vtx is the kth row, 1≤k≤n; the jth row selection switch Sel can control the readout of the pixel calculation unit of the 2j-1≤k≤2j+1 row, and the reset signal row line Vrst and the charge transfer signal Vtx of the 2j-1≤k≤2j+1 row respectively control the reset and charge transfer of the pixel calculation unit of the 2j-1≤k≤2j+1 row.
[0110] Turn on the row select switch of the jth row, the row select signal of the jth row corresponds to the reset signal row line Vrst and the charge transfer signal Vtx controlling the three rows k=2j-1, k=2j, and k=2j+1. In the convolution operation, after the transformation relationship of the convolution kernel moving downward, turn on the row select switch of the j+1th row, then the row select signal of the j+1th row corresponds to the reset signal controlling Vrst(k+2) and the charge transfer signal Vtx(k+2), wherein the reset signal row line Vrst and the charge transfer signal Vtx controlling the three rows k+2=2j+1, k+2=2j+2, and k+2=2j+3.
[0111] As an example, in the reset stage, the control module sets the reset signal Vrst and the charge transfer signal Vtx of the pixel computing units in the three adjacent rows (such as the 2j-1≤k≤2j+1 row, the same below) of the photoelectric sensing array to a high level through a control signal, and the row selection switches Sel (such as the row selection switch Sel of the jth row, the same below) corresponding to the pixel computing units in the three adjacent rows are all disconnected, and the CDS S / H signal is set to a low level.
[0112] As another example, in the exposure stage, the control module sets the reset signal Vrst and the charge transfer signal Vtx of the pixel computing units in three adjacent rows of the photoelectric sensing array to a low level through a control signal, and a group of row selection switches Sel corresponding to the pixel computing units in the three adjacent rows are all disconnected.
[0113] As another example, in the reset stage of the charge transfer area, the control module sets the reset signal Vrst of the pixel computing units in three adjacent rows of the photoelectric sensing array to a high level and the charge transfer signal Vtx to a low level through a control signal, and a group of row selection switches Sel corresponding to the pixel computing units in three adjacent rows are all turned on.
[0114] As another example, in the first related sampling stage, the control module sets the reset signal Vrst and the charge transfer signal Vtx of the pixel calculation units in three adjacent rows of the photoelectric sensing array to a low level through a control signal, and a group of row selection switches Sel corresponding to the pixel calculation units in the three adjacent rows are all turned on, and the CDS S / H signal is set to a high level.
[0115] As another example, in the charge transfer stage, the control module sets the reset signal Vrst of the pixel computing units in three adjacent rows of the photoelectric sensing array to a low level and the charge transfer signal Vtx to a high level through a control signal, and a group of row selection switches Sel corresponding to the pixel computing units in the three adjacent rows are all turned on, and the CDS S / H signal is set to a low level.
[0116] As another example, in the second correlation sampling stage, the control module sets the reset signal Vrst and the charge transfer signal Vtx of the pixel calculation units in three adjacent rows of the photoelectric sensing array to a low level through a control signal, and a group of row selection switches Sel corresponding to the pixel calculation units in the three adjacent rows are all turned on, and the CDS S / H signal is set to a high level.
[0117] When ADC S / H is high, the charge stored in capacitor C1 in the readout circuit is transferred to the analog-to-digital converter (ADC) of the next stage. The A / D signal represents the analog-to-digital conversion, that is, the time consumed by the A / D conversion process; the OUTPUT signal represents the time required for the ADC output signal to be sent to the next stage of memory or FPGA or PC and other devices.
[0118] like Figure 7 As shown, the control signal timing generates three pipelines, and the duration of each pipeline level is the same, which is 7 clock cycles.
[0119] Among them, the first-level pipeline includes the charge transfer area reset phase corresponding to the control signal of the row selection switch of the j-th row of the photoelectric sensing array, the first related sampling phase, the charge transfer phase, the second related sampling phase, and the time for the signal of the readout circuit to be transferred to the analog-to-digital converter of the next level; the second-level pipeline includes the analog-to-digital conversion processing time of the j-th row information of the photoelectric sensing array; the third-level pipeline includes the time for the j-th row digital signal of the photoelectric sensing array to be output to the next-level device.
[0120] Figure 8 The diagram is a schematic diagram showing the relationship between a weight analog voltage output and a digital weight of a weight voltage generating circuit according to an exemplary embodiment.
[0121] like Figure 8As shown, the weight analog voltage Vout output by the weight voltage generating circuit varies linearly with the digital weight input, and the variation range of the weight analog voltage is limited to 2.1V to 2.7V to ensure that the calculation tubes of the pixel calculation unit all operate in the linear region.
[0122] Fig. 9 The diagram is a relationship between a first current representing a positive value output by a 5T pixel calculation unit and the optical power and a digital weight corresponding to a first positive weight voltage of a gate of a MOS tube M4 according to an exemplary embodiment.
[0123] like Fig. 9 As shown, when the optical power is less than 3nW, the relationship between the output current Ip1 of the 5T pixel calculation unit and the product of the optical power and the digital weight is approximately linear, verifying the feasibility of "light intensity × weight" in the pixel calculation unit.
[0124] Adjusting the width-to-length ratio of the MOS tube in the pixel computing unit can achieve a higher linear relationship between the output current of the 5T pixel computing unit and the product of the optical power and the digital weight. However, in order to increase the fill factor of the pixel in the pixel computing unit (photosensitive area / pixel computing unit area) and improve the image quality, the width-to-length ratio of all MOS tubes needs to be reduced as much as possible to save area, so some nonlinearity is retained.
[0125] When the optical power is greater than 4nW, the output current reaches saturation and no longer increases with the increase of optical power.
[0126] In a possible embodiment, a CMOS image sensing and computing integrated circuit based on a 5T, 8T, and 14T hybrid pixel computing unit provided by the present disclosure is combined with a convolutional neural network algorithm and can be applied to image recognition and image processing, and has the characteristics of fast speed, high energy efficiency, and high integration.
[0127] The above describes the specific embodiments of the present disclosure. It should be understood that the present disclosure is not limited to the above specific embodiments, and those skilled in the art can make various deformations or modifications within the scope of the claims, which does not affect the essence of the present disclosure. The above preferred features can be used in any combination without conflicting with each other.
Claims
1. A CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units, characterized in that: include: Photoelectric sensing array, weight voltage generation module, readout circuit and control module; wherein, The photoelectric sensing array includes multiple groups of hybrid pixel computing units and multiple row selection switches. The types of pixel computing units in the hybrid pixel computing units include 5T, 8T, and 14T. Each group of the hybrid pixel computing units includes a first current bus and a second current bus. The photoelectric sensing array is used to convert a light intensity signal into a current signal, perform a convolution multiplication and accumulation operation with a weight voltage of a convolution kernel through the pixel computing unit, and output a current through the first current bus and the second current bus; The weight voltage generation module includes a plurality of groups of weight voltage generation circuits, and the weight voltage generation circuits are used to generate weight voltages corresponding to the convolution kernels and provide weight information to the global photoelectric sensing array; The readout circuit is coupled to the first current bus and the second current bus of the photoelectric sensing array for subtraction operation, current-voltage conversion and correlated double sampling; The control module is coupled to the hybrid pixel calculation unit of the photoelectric sensing array and the row selection switch, and the control module is used to generate a control signal.
2. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: Each of the mixed pixel calculation units is arranged based on the convolution algorithm, the 5T pixel calculation unit is set at the center position of each of the convolution kernels, the 8T pixel unit is set at the left and right positions and the upper and lower positions of each of the convolution kernels, and the 14T pixel calculation unit is set at the upper left position, lower left position, upper right position and lower right position of the center position of each of the convolution kernels.
3. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 2, characterized in that: The left and right positions or the upper and lower positions of every two convolution kernels overlap with each other, and the lower right, lower left, upper left and upper right positions of every four convolution kernels overlap with each other.
4. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 2, characterized in that: The 8T pixel calculation unit is used to perform 2 parallel multiplications and output, and the 14T pixel calculation unit is used to perform 4 parallel multiplications and output.
5. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 2, characterized in that: The 5T pixel calculation unit includes a MOS tube M1, a MOS tube M2, a MOS tube M3, a MOS tube M4, a MOS tube M5 and a photodiode PD1. The gate of the MOS tube M1 is connected to the row line of the pixel reset signal, the drain of the MOS tube M1 is connected to the power supply, the source of the MOS tube M1 is connected to the drain of the MOS tube M2, the gate of the MOS tube M2 is connected to the row line of the charge transfer signal, the source of the MOS tube M2 is connected to the cathode of the photodiode PD1, the anode of the photodiode PD1 is grounded, the gate of the MOS tube M3 is connected to the drain of the MOS tube M2, the drain of the MOS tube M3 is connected to the power supply, the source of the MOS tube M3 is respectively connected to the drain of the MOS tube M4 and the drain of the MOS tube M5, the gate of the MOS tube M4 is connected to the first positive weight voltage, the source of the MOS tube M4 outputs a first current representing a positive value, the gate of the MOS tube M5 is connected to the first negative weight voltage, and the source of the MOS tube M5 outputs a first current representing a negative value.
6. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 5, characterized in that: The 8T pixel calculation unit includes MOS tube M6, MOS tube M7, MOS tube M8, MOS tube M9, MOS tube M10, MOS tube M11, MOS tube M12, MOS tube M13 and photodiode PD2, the gate of the MOS tube M6 is connected to the row line of the pixel reset signal, the drain of the MOS tube M6 is connected to the power supply, the source of the MOS tube M6 is connected to the drain of the MOS tube M7, the gate of the MOS tube M7 is connected to the row line of the charge transfer signal, the source of the MOS tube M7 is connected to the cathode of the photodiode PD2, the anode of the photodiode PD2 is grounded, the gate of the MOS tube M8 is connected to the drain of the MOS tube M7, the drain of the MOS tube M8 is connected to the power supply, and the source of the MOS tube M8 is connected to the row line of the charge transfer signal. The MOS tube M7 is connected to the drain of the MOS tube M7 and the drain of the MOS tube M7. The drain of the MOS tube M7 is connected to the power supply. The source of the MOS tube M7 is connected to the drain of the MOS tube M7 and the drain of the MOS tube M7. The gate ... gate of the MOS tube M7 is connected to the drain of the MOS tube M7. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the drain of the MOS tube M7. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the drain of the MOS tube M7. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply. The gate of the MOS tube M7 is connected to the power supply.
7. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 6, characterized in that: The 14T pixel calculation unit includes MOS tubes M14, M15, M16, M17, M18, M19, M20, M21, M22, M23, M24, M25, M26, M27 and a photodiode PD3, wherein the gate of the MOS tube M14 is connected to the row line of the pixel reset signal, the drain of the MOS tube M14 is connected to the power supply, the source of the MOS tube M14 is connected to the drain of the MOS tube M15, the gate of the MOS tube M15 is connected to the row line of the charge transfer signal, the source of the MOS tube M15 is connected to the photodiode PD3, and the gate of the MOS tube M15 is connected to the row line of the charge transfer signal. The cathode of the photodiode PD3 is connected, the anode of the photodiode PD3 is grounded, the gate of the MOS tube M16 is connected to the drain of the MOS tube M15, the drain of the MOS tube M16 is connected to the power supply, the source of the MOS tube M16 is respectively connected to the drain of the MOS tube M17 and the drain of the MOS tube M18, the gate of the MOS tube M17 is connected to the fourth positive weight voltage, the source of the MOS tube M17 outputs a fourth current representing a positive value, the gate of the MOS tube M18 is connected to the fourth negative weight voltage, the source of the MOS tube M18 outputs a fourth current representing a negative value, the drain of the MOS tube M19 is connected to the power supply, and the gate of the MOS tube M19 is connected to the fourth positive weight voltage. The MOS tube M15 is connected to the drain of the MOS tube M19, the source of the MOS tube M19 is respectively connected to the drain of the MOS tube M20 and the drain of the MOS tube M21, the gate of the MOS tube M20 is connected to the fifth positive weight voltage, the source of the MOS tube M20 outputs a fifth current representing a positive value, the gate of the MOS tube M21 is connected to the fifth negative weight voltage, the source of the MOS tube M21 outputs a fifth current representing a negative value, the drain of the MOS tube M22 is connected to the power supply, the gate of the MOS tube M22 is connected to the drain of the MOS tube M15, the source of the MOS tube M22 is respectively connected to the drain of the MOS tube M23 and the drain of the MOS tube M24, and the MOS The gate of the MOS tube M23 is connected to the sixth positive weight voltage, the source of the MOS tube M23 outputs a sixth current representing a positive value, the gate of the MOS tube M24 is connected to the sixth negative weight voltage, the source of the MOS tube M24 outputs a sixth current representing a negative value, the drain of the MOS tube M25 is connected to the power supply, the gate of the MOS tube M25 is connected to the drain of the MOS tube M15, the source of the MOS tube M25 is respectively connected to the drain of the MOS tube M26 and the drain of the MOS tube M27, the gate of the MOS tube M26 is connected to the seventh positive weight voltage, the source of the MOS tube M26 outputs a seventh current representing a positive value, the gate of the MOS tube M27 is connected to the seventh negative weight voltage,The source output of the MOS tube M27 represents a seventh current of a negative value.
8. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: The current output lines representing positive values corresponding to the same convolution kernel are respectively connected to the first current bus corresponding to the same convolution kernel, and the current output lines representing negative values corresponding to the same convolution kernel are respectively connected to the second current bus corresponding to the same convolution kernel. The first current bus outputs the current representing the positive value in the result of the convolution multiplication and addition operation, and the second current bus outputs the current representing the negative value in the result of the convolution multiplication and addition operation.
9. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: The weight voltage generating circuit includes a PMOS tube M28, a MOS tube M29, a PMOS tube M30, a PMOS tube M31, a PMOS tube M32, a PMOS tube M33, a MOS tube M34, a MOS tube M35, and an operational amplifier OP1. The source of the MOS tube M28 is connected to a power supply, the gate of the PMOS tube M28 is respectively connected to the gate and drain of the MOS tube M29, the drain of the PMOS tube M28 is respectively connected to the gate and drain of the MOS tube M29, the gate and drain of the PMOS tube M28 and the gate and drain of the MOS tube M29 are respectively connected to the positive phase input terminal of the operational amplifier OP1, the source of the MOS tube M29 is grounded, the negative phase input terminal of the operational amplifier OP1 is connected to the output terminal of the operational amplifier OP1, the source of the PMOS tube M30 is connected to the power supply, the gate of the PMOS tube M30 is connected to the first weight digital signal, the drain of the PMOS tube M30 is connected to the drain of the MOS tube M34. The gate of the MOS tube M34 is connected to the power supply, the source of the MOS tube M34 is connected to the drain of the MOS tube M35, the gate of the MOS tube M35 is connected to the drain of the MOS tube M34, the source of the MOS tube M35 is connected to the output end of the operational amplifier OP1, the source of the PMOS tube M31 is connected to the power supply, the gate of the PMOS tube M31 is connected to the second weighted digital signal, and the drain of the PMOS tube M31 is connected to the output end of the operational amplifier OP1. The drain of the OS tube M34 is connected, the source of the PMOS tube M32 is connected to the power supply, the gate of the PMOS tube M32 is connected to the third weighted digital signal, the drain of the PMOS tube M32 is connected to the drain of the MOS tube M34, the source of the PMOS tube M33 is connected to the power supply, the gate of the PMOS tube M33 is grounded, the drain of the PMOS tube M33 is connected to the drain of the MOS tube M34, and the gate of the MOS tube M35 outputs a weighted analog voltage.
10. The CMOS image sensing and computing integrated circuit based on 5T, 8T, and 14T hybrid pixel computing units according to claim 1, characterized in that: The readout circuit includes an operational amplifier OP2, an operational amplifier OP3, an operational amplifier OP4, an operational amplifier OP5, a resistor R1, a resistor R2, a resistor R3, a resistor R4, a resistor R5, a resistor R6, a capacitor C1, a switch S1, a switch S2, a switch S3, a switch S4, a switch S5, and a switch S6. The positive phase input terminal of the operational amplifier OP2 is grounded, the negative phase input terminal of the operational amplifier OP2 is connected to the first current bus of the corresponding column, the input terminal of the resistor R1 is connected to the negative phase input terminal of the operational amplifier OP2, and the output terminal of the resistor R1 is connected to the output terminal of the operational amplifier OP2. The output end of the operational amplifier OP3 is connected to the positive input end of the operational amplifier OP3, the negative input end of the operational amplifier OP3 is grounded, the input end of the resistor R2 is connected to the negative input end of the operational amplifier OP3, the output end of the resistor R3 is connected to the output end of the operational amplifier OP3, one end of the switch S1 is connected to the output end of the operational amplifier OP2, the other end of the switch S1 is connected to the input end of the resistor R3, one end of the switch S2 is connected to the output end of the operational amplifier OP3, the other end of the switch S2 is connected to the input end of the resistor R3 The output end of the resistor R3 is connected to the input end of the resistor R5 and the positive phase input end of the operational amplifier OP4, the output end of the resistor R4 is connected to the input end of the resistor R6 and the negative phase input end of the operational amplifier OP4, the output end of the resistor R5 is connected to the input end of the resistor R5 and the positive phase input end of the operational amplifier OP4, the output end of the resistor R4 is connected to the input end of the resistor R6 and the negative phase input end of the operational amplifier OP4, the output end of the resistor R5 is grounded, and the operational amplifier OP4 is grounded. The output end of P5 and the output end of R6 are respectively connected to one end of the switch S5, the other end of the switch S5 is connected to the negative phase input end of the operational amplifier OP5, the positive phase input end of the operational amplifier OP5 is grounded, the input end of the capacitor C1 is connected to the negative phase input end of the operational amplifier OP5, the output end of the capacitor C1 is connected to the output end of the operational amplifier OP5, one end of the switch S6 is connected to the negative phase input end of the operational amplifier OP5, the other end of the switch S6 is connected to the output end of the operational amplifier OP5, and the output end of the operational amplifier OP5 outputs a voltage.
Citation Information
Patent Citations
Sensing, storing and calculating integrated circuit structure for realizing positive and negative weight calculation in pixels
CN113596361A
Architecture for supporting convolution kernel calculation of any size and shape
CN117808050A
Complementary metal oxide semiconductor (CMOS) pixel internal sensing and calculation integrated circuit system and integrated operation method
CN118118812A
Convolutional neural networks using resistive processing unit array
US9646243B1
