Infrared focal plane binary convolution interconnect readout circuit with charge domain multiplication and accumulation

Abstract

This application relates to the field of integrated circuit technology and provides an infrared focal plane binary convolution interconnect readout circuit with charge domain multiplication and accumulation. This circuit adds convolution calculation functionality to an infrared focal plane readout chip without sacrificing area, power consumption, or time, enabling the chip to have intelligent data preprocessing capabilities, including noise reduction and edge extraction. It completes edge-side computation, effectively reducing a large amount of analog-to-digital conversion and data transfer, thus alleviating the data pressure on backend intelligent vision applications. This solution is a hardware convolutional network applicable to infrared focal plane signal readout, which can significantly improve the energy efficiency ratio of infrared intelligent vision.

Description

Technical Field

[0001] This application relates to the field of integrated circuit technology, and in particular to an infrared focal plane binary convolution interconnect readout circuit with charge domain multiplication and accumulation. Background Technology

[0002] When performing infrared image recognition in complex environments, visual sensing systems often face problems such as low signal-to-background ratio and poor signal-to-noise ratio of infrared signals. Especially in long-distance detection or weak target detection scenarios, infrared target signals are easily obscured by complex backgrounds. Such scenarios with low recognition accuracy and high false positive rates place stringent requirements on intelligent visual recognition algorithms, demanding large-scale and high-computing power.

[0003] Most mainstream infrared vision hardware recognition systems currently adopt the "von Neumann" discrete architecture. Infrared focal plane arrays (FPAs) generally adopt a pixel integration architecture with capacitive transimpedance amplifiers (CTIAs) as the core. The readout link is mostly column parallel / time division multiplexing: pixel integration → column readout → on-chip / off-chip analog-to-digital conversion → digital interface output → board-level or host computer to perform visual algorithm calculations such as convolutional neural networks. The main problems with this process are: First, the data transfer process consumes a lot of power and bandwidth, placing a heavy load on the on-chip bus, input / output (I / O) interfaces, and back-end processor. Second, the multiple link layers and accumulated latency limit edge-side response in fast-moving / complex scenarios. Third, the separation of algorithms and sensing locations leads to repeated memory accesses and data copies between sensing and computation, requiring extensive analog-to-digital conversion and cross-module data transfer. High-bit digital domain computation is needed to complete neural networks such as convolution algorithms, creating a bottleneck between storage and computation, making it difficult to simultaneously meet the requirements of high resolution and low power consumption. These problems result in high system power consumption and long processing latency, making it difficult to meet the urgent needs of edge intelligent vision systems for real-time performance and energy efficiency. Especially on energy-constrained devices at the edge, its role is very limited.

[0004] As an essential core component of infrared detectors, the readout circuit currently only has the simple function of reading analog data, and most readout circuits in related technologies suffer from the aforementioned shortcomings. Therefore, how to reduce power consumption and complete data calculation without affecting readout speed is a technical challenge in the field of intelligent visual sensing. Summary of the Invention

[0005] In view of this, embodiments of this application provide an infrared focal plane binary convolution interconnect readout circuit with charge domain multiplication and accumulation to solve the problem that existing readout circuits cannot simultaneously achieve end-side computation and low power consumption.

[0006] A first aspect of this application provides an infrared focal plane binary convolution interconnect readout circuit with charge domain multiplication and accumulation, the circuit comprising: an intra-pixel integration unit, a charge domain focal plane convolution interconnect readout unit, and an output unit;

[0007] The intra-pixel integration unit includes an improved capacitive feedback transimpedance amplifier, which receives the photocurrent and divides the integrated charge of the photocurrent into N equal parts before outputting them to the charge domain focal plane convolution interconnect readout unit; N is a positive integer.

[0008] The charge domain focal plane convolution interconnect readout unit includes a static random access memory (SRAM). The charge domain focal plane convolution interconnect readout unit performs convolution operations based on the convolution kernel weights stored in the SRAM and the received N charges, and outputs the calculation results through the output unit.

[0009] In some embodiments, the improved capacitive feedback transimpedance amplifier includes an operational amplifier, N parallel integrating capacitors, and N first switches;

[0010] The intra-pixel integration unit also includes a second switch, a sampling switch, and a sampling capacitor;

[0011] The first input terminal of the operational amplifier is connected to the reference voltage output terminal, and the second input terminal is connected to the photocurrent output terminal.

[0012] The first terminal of each integrating capacitor is connected to the second input terminal of the operational amplifier, the second terminal of each integrating capacitor is connected to the first terminal of a first switch, and the second terminal of each first switch is connected to the output terminal of the operational amplifier.

[0013] The output of the operational amplifier is also connected to the first terminal of the second switch. The second terminal of the second switch is connected to the first terminal of the sampling switch, the first terminal of the sampling capacitor, and the input terminal of the output unit. The second terminal of the sampling switch and the second terminal of the sampling capacitor are both grounded.

[0014] In some embodiments, the charge domain focal plane convolutional interconnect readout unit further includes N third switches and N fourth switches;

[0015] In this configuration, the first terminal of each third switch is connected to the second terminal of an integrating capacitor, and the second terminal of each third switch is connected to the first terminal of a fourth switch.

[0016] The fourth switch is controlled by SRAM, and the second terminal of each fourth switch is connected to the first terminal of the sampling capacitor of this infrared focal plane binary convolution interconnect readout circuit and the first terminal of the sampling capacitor of the adjacent infrared focal plane binary convolution interconnect readout circuit, respectively.

[0017] In some embodiments, N equals 9, and the infrared focal plane binary convolutional interconnect readout circuit and its adjacent infrared focal plane binary convolutional interconnect readout circuits form a 3-row 3-column circuit array, with the infrared focal plane binary convolutional interconnect readout circuit located in the 2nd row and 2nd column of the circuit array.

[0018] The second terminal of the fourth switch is connected to the first terminal of the sampling capacitor of this infrared focal plane binary convolution interconnect readout circuit and the first terminal of the sampling capacitor of the adjacent infrared focal plane binary convolution interconnect readout circuit in the following manner:

[0019] The second terminal of the first fourth switch in the infrared focal plane binary convolutional interconnect readout circuit is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the 3rd row and 3rd column of the circuit array. The second terminal of the second fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the 3rd row and 2nd column of the circuit array. The second terminal of the third fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the 3rd row and 1st column of the circuit array. The second terminal of the fourth fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the 2nd row and 3rd column of the circuit array. The second terminal of the fifth fourth switch... The first terminal of the sampling capacitor connected to this infrared focal plane binary convolutional interconnect readout circuit, the second terminal of the sixth fourth switch connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the second row and first column of the circuit array, the second terminal of the seventh fourth switch connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the first row and third column of the circuit array, the second terminal of the eighth fourth switch connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the first row and second column of the circuit array, and the second terminal of the ninth fourth switch connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the first row and first column of the circuit array.

[0020] In some embodiments, the output unit includes a flip-flop voltage follower;

[0021] The flip-voltage follower includes a constant current source, a first transistor, and a second transistor connected in series.

[0022] The first terminal of the constant current source is grounded, and the second terminal is connected to the drain of the first transistor;

[0023] The gate of the first transistor is the input terminal of the flip voltage follower, the source of the first transistor is connected to the drain of the second transistor as the output terminal of the flip voltage follower, and the drain of the first transistor is also connected to the gate of the second transistor.

[0024] The source of the second transistor is connected to the power supply.

[0025] A second aspect of this application provides an imaging method based on an infrared focal plane binary convolution interconnect readout circuit using charge domain multiplication and accumulation, the method being executed by the circuit in the first aspect;

[0026] The method includes:

[0027] Close the first switch, and open the second switch, sampling switch, and third switch;

[0028] The photocurrent is input from the second input terminal of the capacitor feedback transimpedance amplifier and integrated over N parallel-connected integrating capacitors to obtain the integrated voltage.

[0029] In response to the determination that the integration termination condition is met, the sampling switch is closed to clear the charge between the upper and lower plates of the sampling capacitor;

[0030] Disconnect the sampling switch and close the second switch to sample the integrated voltage to the sampling capacitor;

[0031] Disconnect the second switch and determine that the voltage at the first terminal of the sampling capacitor is the imaging voltage.

[0032] In some embodiments, the imaging voltage is ;in, For imaging voltage, It is an integrating capacitor. Photocurrent, This is the preset time for scoring.

[0033] A third aspect of the embodiments of this application provides a convolution calculation method for an infrared focal plane binary convolution interconnect readout circuit based on charge domain multiplication and accumulation, the method being executed by the circuit in the first aspect;

[0034] The method includes:

[0035] Close the third switch, and open the first switch, the second switch, and the sampling switch;

[0036] The SRAM is controlled to output a control signal based on the convolution kernel weights. The control signal is used to control the fourth switch so that the fourth switch with a convolution kernel weight of 1 is closed and the fourth switch with a convolution kernel weight of 0 is opened.

[0037] The accumulated charge of the sampling capacitor is obtained; the accumulated charge is obtained by adding up the charges of the integrating capacitors corresponding to each closed fourth switch.

[0038] The convolution voltage is determined based on the accumulated charge.

[0039] In some embodiments, the convolution voltage is ;in, The convolution voltage, To accumulate charge, For sampling capacitor, It is the integrating capacitor corresponding to the fourth switch with a kernel weight of 1. These are the kernel weights.

[0040] In some embodiments, the value of the sampling capacitor is... ;in, It is an integrating capacitor.

[0041] The beneficial effects of this application's embodiments compared to existing technologies are as follows: This application's embodiments add convolutional computation functionality to the infrared focal plane readout chip without sacrificing area, power consumption, or time. This enables the chip to possess intelligent data preprocessing computational capabilities, including noise reduction and edge extraction, completing edge-side computation and effectively reducing a large amount of analog-to-digital conversion and data transfer, thus alleviating the data pressure on backend intelligent vision applications. This solution is a hardware convolutional network applicable to infrared focal plane signal readout, which can significantly improve the energy efficiency ratio of infrared intelligent vision.

[0042] Meanwhile, the technical solution provided in this application not only has a high degree of design freedom, but also allows for editable convolution calculation weights, making it compatible with various infrared focal plane detectors that can be interconnected with indium pillars, including but not limited to short-wave, mid-wave, and long-wave, thus significantly improving the application capabilities of this architecture. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 This is a block diagram of a charge domain multiplication and accumulation infrared focal plane binary convolution interconnect readout circuit provided in an embodiment of this application.

[0045] Figure 2 This is a schematic diagram of a charge domain multiplication and accumulation infrared focal plane binary convolution interconnect readout circuit provided in an embodiment of this application.

[0046] Figure 3 This is a schematic diagram of the circuit structure of the flip voltage follower provided in the embodiments of this application.

[0047] Figure 4 This is a schematic diagram of the current flow between the integrating capacitors of the integrating unit within a pixel in the circuit array provided in this application embodiment.

[0048] Figure 5This is a schematic diagram of the simulation results of the flip voltage follower provided in the embodiments of this application.

[0049] Figure 6 This is a simulation effect diagram provided in an embodiment of this application.

[0050] Figure 7 This is another simulation effect diagram provided in the embodiments of this application.

[0051] Figure 8 This is another simulation effect diagram provided in the embodiments of this application.

[0052] Figure 9 This is a diagram illustrating the infrared focal plane binary convolution interconnect readout circuit based on charge domain multiplication and accumulation provided in this application in the field of infrared intelligent vision applications. Detailed Implementation

[0053] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0054] The following describes in detail, with reference to the accompanying drawings, an infrared focal plane binary convolution interconnect readout circuit based on an embodiment of this application, and its operation method.

[0055] like Figure 1 As shown, the infrared focal plane binary convolution interconnect readout circuit with charge domain multiplication and accumulation includes an intra-pixel integration unit, a charge domain focal plane convolution interconnect readout unit, and an output unit.

[0056] The pixel-integration unit includes an improved capacitive feedback transimpedance amplifier, which receives the photocurrent and divides the integrated charge of the photocurrent into N equal parts before outputting them to the charge domain focal plane convolution interconnect readout unit; N is a positive integer.

[0057] The charge domain focal plane convolution interconnect readout unit includes a static random-access memory (SRAM). The charge domain focal plane convolution interconnect readout unit performs convolution operations based on the convolution kernel weights stored in the SRAM and the received N charges, and outputs the calculation results through the output unit.

[0058] like Figure 2As shown, the improved capacitive feedback transimpedance amplifier includes an operational amplifier Amp, N parallel integrating capacitors, and N first switches. N is a positive integer and can be set according to actual needs; no restriction is imposed here. For ease of description and explanation, the following description uses N=9 as an example. That is, the improved capacitive feedback transimpedance amplifier may include nine parallel integrating capacitors C1 to C9, each integrated capacitor connected in series with a first switch C.

[0059] In some embodiments of this application, the intra-pixel integration unit may further include a second switch SH, a sampling switch Samp, and a sampling capacitor C0. The first input terminal of the operational amplifier is connected to the reference voltage output V. ref The second input terminal is connected to the photocurrent output terminal. In some embodiments, a fifth switch (int) can also be provided between the photocurrent input terminal and the second input terminal of the operational amplifier.

[0060] In some embodiments of this application, the operational amplifier may be a five-transistor operational amplifier, with each integrating capacitor having the same size, and both the integrating capacitor and the sampling capacitor may be metal-insulator-metal (MIM) capacitors.

[0061] In some embodiments of this application, the first terminal of each integrating capacitor Ci is connected to the second input terminal of an operational amplifier, the second terminal of each integrating capacitor Ci is connected to the first terminal of a first switch C, and the second terminal of each first switch C is connected to the output terminal of the operational amplifier. Here, i is a positive integer greater than or equal to 1 and less than or equal to N.

[0062] In some embodiments of this application, the output terminal of the operational amplifier is also connected to the first terminal of the second switch SH, the second terminal of the second switch SH is connected to the first terminal of the sampling switch samp, the first terminal of the sampling capacitor C0 and the input terminal of the output unit, and the second terminal of the sampling switch samp and the second terminal of the sampling capacitor C0 are both grounded.

[0063] In some embodiments of this application, the charge domain focal plane convolution interconnect readout unit further includes N third switches and N fourth switches, for example, including 9 third switches S and 9 fourth switches S1 to S9.

[0064] In this configuration, the first terminal of each third switch S is connected to the second terminal of an integrating capacitor Ci, and the second terminal of each third switch S is connected to the first terminal of a fourth switch Si.

[0065] The fourth switches S1 to S9 are controlled by SRAM, and the second terminal of each fourth switch Si is connected to the first terminal of the sampling capacitor of the current infrared focal plane binary convolutional interconnect readout circuit and the first terminal of the sampling capacitor of the adjacent infrared focal plane binary convolutional interconnect readout circuit. In one example, the second terminal of each fourth switch can be denoted as outi.

[0066] If N=9 is set, this infrared focal plane binary convolutional interconnect readout circuit can form a 3-row, 3-column circuit array with its adjacent infrared focal plane binary convolutional interconnect readout circuits, and this infrared focal plane binary convolutional interconnect readout circuit is located at the center of the circuit array, that is, in the 2nd row and 2nd column of the circuit array.

[0067] At this time, the second terminal of the fourth switch is connected to the first terminal of the sampling capacitor of this infrared focal plane binary convolution interconnect readout circuit and the first terminal of the sampling capacitor of the adjacent infrared focal plane binary convolution interconnect readout circuit in the following manner:

[0068] The second terminal of the first fourth switch in this infrared focal plane binary convolutional interconnect readout circuit is connected to the first terminal (i.e., the upper plate) of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the 3rd row and 3rd column of the circuit array. The second terminal of the second fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the 3rd row and 2nd column of the circuit array. The second terminal of the third fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the 3rd row and 1st column of the circuit array. The second terminal of the fourth fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnect readout circuit in the 2nd row and 3rd column of the circuit array. The fifth fourth switch... The second terminal is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnection readout circuit. The second terminal of the sixth fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnection readout circuit in the second row and first column of the circuit array. The second terminal of the seventh fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnection readout circuit in the first row and third column of the circuit array. The second terminal of the eighth fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnection readout circuit in the first row and second column of the circuit array. The second terminal of the ninth fourth switch is connected to the first terminal of the sampling capacitor of the infrared focal plane binary convolutional interconnection readout circuit in the first row and first column of the circuit array.

[0069] In some embodiments of this application, the output unit may include a flipped voltage follower (FVF).

[0070] like Figure 3As shown, a flip-flop voltage follower may include a constant current source, a first transistor, and a second transistor connected in series. The first terminal of the constant current source is grounded, and the second terminal is connected to the drain of the first transistor. The gate of the first transistor serves as the input terminal of the flip-flop voltage follower, and the source of the first transistor is connected to the drain of the second transistor as the output terminal of the flip-flop voltage follower. The drain of the first transistor is also connected to the gate of the second transistor. The source of the second transistor is connected to a power supply.

[0071] like Figure 4 As shown, PE1 to PE9 represent the integration units within a pixel, and the data flow direction corresponding to the arrows in the figure refers to the direction of charge flow on the second end (i.e., the right electrode plate) of C1 to C9.

[0072] In other words, the integral unit (PE) within each pixel i The MIM capacitors are interconnected on the array using convolution kernels via metal lines. Within each pixel's integration unit, when switch S is closed, since the right plate of each MIM capacitor (C1 to C9) is connected to a switch (S1 to S9 respectively), the on / off state of switches S1 to S9 is determined by... Figure 2 In the SRAM array control, when the SRAM stores a high level "1", switch Si closes, and the charge on the right plate of Ci can be transferred to the outi interface through Si, and then... Figure 4 The flow is directed to the sampling capacitor of the target infrared focal plane binary convolution interconnect readout circuit via the interconnection of the metal wires, and a voltage is formed on the sampling capacitor.

[0073] In some embodiments of this application, the infrared focal plane binary convolution interconnect readout circuit based on charge domain multiplication and accumulation can operate in imaging mode as follows: close the first switch, and open the second switch, sampling switch, and third switch; the photocurrent is input from the second input terminal of the capacitor feedback transimpedance amplifier and integrated over N parallel-connected integrating capacitors to obtain the integrated voltage; in response to determining that the integration termination condition is met, close the sampling switch to clear the charge between the upper and lower plates of the sampling capacitor; open the sampling switch and close the second switch to sample the integrated voltage onto the sampling capacitor; open the second switch and determine that the voltage value at the first terminal of the sampling capacitor is the imaging voltage.

[0074] In other words, in normal imaging mode, the first switch C, which acts as a complementary switch, is closed, the third switch S is open, the second switch SH is open, and the sampling switch samp is open. At this time, the photocurrent flows in from the int terminal. Under the voltage bias of AMP, the photocurrent is integrated on the nine parallel integrating capacitors C1 to C9 to form the integrated voltage, which is the imaging voltage.

[0075] After a certain integration time, the sampling switch samp next to the sampling capacitor C0 is closed, clearing the charge on the upper and lower plates of capacitor C0 to zero, making the voltage on C0 zero. Then the sampling switch samp is opened, and the second switch SH is closed, sampling the integrated voltage formed on C1 to C9 onto the sampling capacitor C0. Then the second switch SH is opened, and the voltage on C0 is the voltage signal formed by integrating the photocurrent generated by the infrared detector after receiving light for a period of time.

[0076] In one example, the formula for the imaging voltage is: ;in, For imaging voltage, It is an integrating capacitor. Photocurrent, This is the preset time for scoring.

[0077] In some embodiments of this application, the infrared focal plane binary convolution interconnect readout circuit based on charge domain multiplication and accumulation can perform convolution calculation in the following manner: close the third switch, and open the first switch, the second switch, and the sampling switch; control the SRAM to output a control signal based on the convolution kernel weight, and control the fourth switch based on the control signal so that the fourth switch with a corresponding convolution kernel weight of 1 is closed, and the fourth switch with a corresponding convolution kernel weight of 0 is opened; obtain the accumulated charge of the sampling capacitor; the accumulated charge is obtained by accumulating the charge of the integrating capacitor corresponding to each closed fourth switch; determine the convolution voltage based on the accumulated charge.

[0078] In other words, in convolution calculation mode, initially, the complementary switch C is closed, the third switch S is open, the second switch SH is open, and the sampling switch samp is open. The photocurrent is integrated across capacitors C1 to C9 via the int terminal, forming an integrated voltage and accumulating positive charge on the right plates of capacitors C1 to C9. Since C1 to C9 are connected in parallel, the voltage is the same at the connection points on the right plates, and the amount of charge accumulated on the right plate of each integrating capacitor is also the same. Afterward, the second switch SH remains open, and the sampling switch samp closes, clearing the charge on the upper and lower plates of capacitor C0 to zero, making the voltage on C0 0. Then, the sampling switch samp opens, and the third switch S closes.

[0079] The fourth switches S1 to S9 are switched on and off under the control of the SRAM, and the closed switch Si transfers the charge of the corresponding right plate of Ci to outi, and then according to... Figure 4 The voltage is generated by the interconnected metal wires flowing to the sampling capacitor of the target infrared focal plane binary convolution interconnected readout circuit.

[0080] When the SRAM stores a high level "1", switch Si is closed, and the charge on the right electrode of Ci is transferred to the outi interface through Si, which is equivalent to the information value in the integration unit multiplied by the weight "1" (wi=1). When the SRAM stores a low level "0", switch Si is open, and the charge on the right electrode of Ci cannot be transferred to the outi interface through Si, which is equivalent to the information value in the integration unit multiplied by the weight "0" (wi=0). This completes the multiplication operation between pixel information and weights, and the voltage formed on the sampling capacitor in the target unit is equivalent to the result of accumulating the information within the convolution kernel.

[0081] In one example, the convolution voltage can be ;in, The convolution voltage, To accumulate charge, For sampling capacitor, It is the integrating capacitor corresponding to the fourth switch with a kernel weight of 1. These are the kernel weights.

[0082] As can be seen from the above formula, this calculation method does not strictly follow the linear result obtained by convolution calculation. samp The presence of causes the resulting voltage V2 to exhibit nonlinearity. samp A larger value for C allows for the capture of more charge, resulting in better linearity of the calculated voltage. However, this leads to a smaller voltage value and a decrease in resolution. samp A smaller capacitance value allows for the accumulation of charge to form a voltage with clear resolution, but it reduces the linearity of the calculation. This is a trade-off process. Through simulation testing, after weighing linearity and voltage resolution, the optimal capacitance value is... ;in, The integrating capacitor is referred to here.

[0083] The flip-flop voltage follower provided in this application significantly improves the output linearity of the voltage follower by decoupling the relationship between the output current and the current source current through structural negative feedback. This application embodiment demonstrates the flip-flop voltage follower using DC simulations with Cadence Virtuoso. Figure 5 It can be seen that the FVF input range is 0-2.5V, the output range is 0.75-3.3V, and the linear fit R0 is [value missing]. 2 =0.99968, which is significantly better than the traditional source follower.

[0084] To verify the technical effects of the technical solutions provided in the embodiments of this application, the following experiment was designed:

[0085] Example 1:

[0086] Different photocurrents were injected into the improved intra-pixel integration unit, and the simulation was performed using Cadence Virtuoso software, following the timing operation of the imaging mode described in the specific implementation. The invention recorded the simulated voltage values ​​output by the improved intra-pixel integration unit in the imaging mode to verify whether the improved capacitor-transimpedance amplifier structure affected or burdened the original imaging (increased power consumption, time delay, etc.). By comparing the integration output results with those of a conventional imaging unit with the same structure, the improved intra-pixel integration unit did not cause any impact on the imaging function.

[0087] Example 2:

[0088] Different photocurrents are injected into the in-pixel integration units, and the timing operation of the convolution calculation mode is simulated using Cadence Virtuoso software, following the specific implementation method. This invention records the simulated voltage values ​​of the in-pixel integration units after charge convolution calculations only to verify the effectiveness and feasibility of the calculation. Generally, changing the photocurrent and weight ratio clearly alters the output simulated voltage value; a higher output voltage indicates a higher photocurrent injection or a larger weight ratio. The results clearly demonstrate that the in-pixel integration units can effectively obtain spatial domain information in the convolution calculation mode.

[0089] Test Example 1:

[0090] Using Cadence Virtuoso software, an equivalent incident photocurrent of 5-60 nA was injected into a single pixel unit to observe whether the improved pixel's integrator unit suffered from a loss of nonlinearity or a power consumption burden. Simulation results are as follows: Figure 6 As shown. Meanwhile, for a 4 An equivalent incident photocurrent of 5nA to 70nA was injected into certain pixels of a 32-pixel array. By setting the timing of row and column selection, the analog signal in the pixel was transmitted to the analog output port. The load was set to 3pF (a conventional post-stage capacitor load), and the output effect was tested. The simulation results are as follows: Figure 7 As shown. Figure 6 and Figure 7 The results all indicate that the improved simulation output structure did not cause any loss in imaging quality or increase in power consumption.

[0091] Test Example 2:

[0092] Using Cadence Virtuoso software, photocurrents of 30nA, 80nA, 130nA, 150nA, and 180nA generated by an equivalent detector were injected into an in-pixel integration unit. In convolution calculation mode, the analog voltage values ​​output by the in-pixel integration unit were obtained, as shown in the following results. Figure 8As shown, this invention demonstrates that different photocurrents with varying weight ratios can produce voltage outputs with excellent linearity and clear resolution. This result clearly proves the feasibility of the charge-domain multiplication-accumulation infrared focal plane binary convolution interconnect readout circuit in convolution calculation mode.

[0093] like Figure 9 As shown in the embodiment of this application, the application demonstration of the infrared focal plane binary convolutional interconnect readout circuit based on charge domain multiplication and accumulation in the field of infrared intelligent vision includes an infrared focal plane detector (DET) and a convolutional interconnect readout circuit chip. Utilizing an existing infrared focal plane detector, the received infrared light information is converted into current information. Through flip-chip interconnect technology based on indium pillar interconnects, this current is connected to the infrared focal plane convolutional interconnect readout circuit shown in this application, indicating that the current representing the infrared light information will be injected into each improved intra-pixel integration unit (PE). i The front end of the readout unit (Focal Plane Convolutional Interconnection) enables convolutional feature extraction of infrared image signals at the pixel level, thus completing part of the image feature calculation during the readout stage. The convolutional features of the image are then transmitted off-chip to a neural network classifier to complete the final target recognition and classification task.

[0094] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0095] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An infrared focal plane binary convolution interconnection readout circuit with charge domain multiply-accumulate, characterized by, The circuit comprises an in-pixel integration unit, a charge domain focal plane convolution interconnection readout unit and an output unit; The in-pixel integration unit comprises an improved capacitor feedback transimpedance amplifier for receiving a photoelectric current and outputting charges obtained by integrating the photoelectric current and averaging the charges into N parts to the charge domain focal plane convolution interconnection readout unit; N is a positive integer; The charge domain focal plane convolution interconnection readout unit comprises a static random access memory; the charge domain focal plane convolution interconnection readout unit completes convolution operation based on convolution kernel weights stored in the static random access memory and the N parts of received charges, and outputs a calculation result through the output unit; The improved capacitor feedback transimpedance amplifier comprises an operational amplifier, N parallel integral capacitors and N first switches; The in-pixel integration unit further comprises a second switch, a sampling switch and a sampling capacitor; The first input end of the operational amplifier is connected to a reference voltage output end, and the second input end is connected to a photoelectric current output end; The first end of each integral capacitor is connected to the second input end of the operational amplifier, and the second end of each integral capacitor is connected to the first end of a first switch, respectively; the second end of each first switch is connected to the output end of the operational amplifier; The output end of the operational amplifier is further connected to the first end of the second switch, the second end of the second switch is connected to the first end of the sampling switch, the first end of the sampling capacitor and the input end of the output unit, and the second end of the sampling switch and the second end of the sampling capacitor are both grounded.

2. The circuit of claim 1, wherein, The charge domain focal plane convolution interconnection readout unit further comprises N third switches and N fourth switches; The first end of each third switch is connected to the second end of an integral capacitor, respectively, and the second end of each third switch is connected to the first end of a fourth switch, respectively; The fourth switch is controlled by the static random access memory, and the second end of each fourth switch is connected to the first end of the sampling capacitor of the current infrared focal plane binary convolution interconnection readout circuit and the first end of the sampling capacitor of an adjacent infrared focal plane binary convolution interconnection readout circuit, respectively.

3. The circuit of claim 2, wherein, N is equal to 9, the current infrared focal plane binary convolution interconnection readout circuit and the adjacent infrared focal plane binary convolution interconnection readout circuit form a circuit array of 3 rows and 3 columns, and the current infrared focal plane binary convolution interconnection readout circuit is located in the second row and the second column of the circuit array; The second end of the fourth switch is connected to the first end of the sampling capacitor of the current infrared focal plane binary convolution interconnection readout circuit and the first end of the sampling capacitor of the adjacent infrared focal plane binary convolution interconnection readout circuit in the following manner: A second end of a first fourth switch in the infrared focal plane binary convolution interconnection readout circuit is connected to a first end of a sampling capacitor of a third column and a third row infrared focal plane binary convolution interconnection readout circuit in the circuit array, a second end of a second fourth switch is connected to a first end of a sampling capacitor of a second column and a third row infrared focal plane binary convolution interconnection readout circuit in the circuit array, a second end of a third fourth switch is connected to a first end of a sampling capacitor of a first column and a third row infrared focal plane binary convolution interconnection readout circuit in the circuit array, a second end of a fourth fourth switch is connected to a first end of a sampling capacitor of a third column and a second row infrared focal plane binary convolution interconnection readout circuit in the circuit array, a second end of a fifth fourth switch is connected to a first end of a sampling capacitor of the infrared focal plane binary convolution interconnection readout circuit, a second end of a sixth fourth switch is connected to a first end of a sampling capacitor of a first column and a second row infrared focal plane binary convolution interconnection readout circuit in the circuit array, a second end of a seventh fourth switch is connected to a first end of a sampling capacitor of a third column and a first row infrared focal plane binary convolution interconnection readout circuit in the circuit array, a second end of an eighth fourth switch is connected to a first end of a sampling capacitor of a second column and a first row infrared focal plane binary convolution interconnection readout circuit in the circuit array, and a second end of a ninth fourth switch is connected to a first end of a sampling capacitor of a first column and a first row infrared focal plane binary convolution interconnection readout circuit in the circuit array.

4. The circuit of claim 2, wherein, The output unit comprises a flip voltage follower; The flip voltage follower comprises a constant current source, a first transistor and a second transistor connected in series; A first end of the constant current source is grounded, and a second end of the constant current source is connected to a drain of the first transistor; A gate of the first transistor is an input end of the flip voltage follower, a source of the first transistor is connected to a drain of the second transistor as an output end of the flip voltage follower, and a drain of the first transistor is further connected to a gate of the second transistor; A source of the second transistor is connected to a power supply.

5. An imaging method of infrared focal plane binary convolution interconnection readout circuit based on charge domain multiply-accumulate, characterized in that, The method is performed by the circuit according to any one of claims 2 to 4; The method comprises: closing the first switch, and opening the second switch, the sampling switch and the third switch; the photocurrent is input from the second input end of the capacitor feedback transimpedance amplifier and integrated on N parallel connected integration capacitors to obtain an integration voltage; in response to determining that the integration termination condition is met, closing the sampling switch to clear the charge between the upper and lower plates of the sampling capacitor; opening the sampling switch and closing the second switch to sample the integration voltage to the sampling capacitor; opening the second switch and determining the voltage value at the first end of the sampling capacitor as an imaging voltage.

6. The method of claim 5, wherein, The imaging voltage is ; wherein, The imaging voltage is The integral capacitance is The photocurrent is The preset integral duration is 7. A method for convolution computation of an infrared focal plane binary convolution interconnection readout circuit based on charge domain multiply-accumulate, characterized in that, The method is performed by the circuit according to any one of claims 2 to 4; The method comprises: closing the third switch, and opening the first switch, the second switch and the sampling switch; The static random access memory controls the fourth switch based on the control signal of the convolution kernel weight output, controls the fourth switch based on the control signal, so that the fourth switch corresponding to the convolution kernel weight of 1 is closed, and the fourth switch corresponding to the convolution kernel weight of 0 is opened; The accumulated charge of the sampling capacitor is obtained; the accumulated charge is obtained by accumulating the charge of the integral capacitor corresponding to each closed fourth switch; The convolution voltage is determined based on the accumulated charge.

8. The method of claim 7, wherein, The convolution voltage is ; wherein, is the convolution voltage, is the accumulated charge, is the sampling capacitance, is the integral capacitance corresponding to the fourth switch pair with a convolution kernel weight of 1, is the convolution kernel weight.

9. The method of claim 8, wherein, The sampling capacitor has a value of ; wherein is the integration capacitor.

Images