Image sensing system and image acquisition device

Abstract

The invention provides an image sensing system and an image acquisition device. The image sensing system comprises a first structure and a second structure, the first structure is used for imaging and energy collection, the second structure is used for imaging and calculation, each of the first structure and the second structure comprises a plurality of photodiodes, and the photodiodes are set to be in a zero bias mode. By arranging the first structure used for imaging and energy collection and the second structure used for imaging and calculation, the first structure collects solar energy to supply power to the system, the power consumption of the system is effectively reduced, and due to the fact that the number of used devices is small, the purpose of reducing the power consumption and the area of the system is achieved.

Description

Technical Field

[0001] The present invention belongs to the field of image acquisition technology, and in particular relates to an image sensing system and an image acquisition device. Background Art

[0002] With the development of the Internet of Things (IoT) era, intelligent sensing devices are becoming increasingly popular in different application scenarios and are widely used in factories, automobiles, transportation, homes and other fields. However, traditional IoT devices rely on batteries, and the energy storage capacity of batteries is limited, which shortens the service life of the equipment. The equipment needs to replace the battery frequently to maintain long-term operation, which not only increases the maintenance cost of the equipment, but also is not environmentally friendly. In addition, in the traditional intelligent visual perception system, the information received by the perception node is mainly an analog signal with noise, and the information processed by the computing node is a digital signal in the classic von Neumann computing architecture. Traditional intelligent perception includes two parts: visual perception and data processing. Visual perception uses CMOS image sensors to convert the light signal in physical vision into an electrical signal, which is then converted into a digital image signal through an analog-to-digital converter (ADC) and output to the computing node. The processed digital image signal can be uploaded to a server or cloud device.

[0003] The in-sensor processing (PIS) architecture pre-processes the image directly within the sensor so that the output image feature map contains less data, thereby reducing power consumption. However, this structure still relies on the ADC to convert the analog signal into a digital signal for processing, which brings its own power overhead. In addition, addressing the complexity of network parameters in recognition devices, especially those using convolutional neural networks (CNNs), presents another power consumption challenge. To address this issue, a binary neural network (BNN) was introduced to quantize the weights and activation values ​​to +1 or -1, thereby reducing the complexity of network parameters. In addition, an in-sensor computing structure for the first layer of BNN calculations was proposed, in which the analog signal captured by the sensor is directly processed in the analog domain, saving the power consumption and conversion associated with the original data transmission. However, this structure focuses more on low-power computing and does not prioritize energy supply issues. In terms of addressing the energy supply challenge, the prior art proposes a solution: integrating a photodiode (PD) array within a CMOS sensor for energy harvesting. By utilizing the photovoltaic mode of these PDs, solar energy can be captured directly within the pixel. This approach enables simultaneous image sensing and energy harvesting in the same focal plane, thereby establishing an efficient and self-sufficient power management system. However, the increase in the number of switches used to switch the PD operating mode also leads to power consumption.

[0004] CMOS pixel array energy harvesting image sensors only extract edge features within pixels, but a large amount of raw data needs to be converted and transmitted to the backend processor, which will lead to high energy consumption, high latency, memory bottlenecks, low edge feature extraction speed, etc. In addition, the increase in the number of switches used to achieve energy harvesting within pixels will also bring power consumption overhead to the system.

[0005] The solar-powered proximity event detection sensor uses an external discrete solar energy harvesting array for power supply, which has a large system area overhead. And no energy harvesting method is used on-chip. The system uses digital domain-based computing, which faces greater power consumption overhead when expanding to more complex tasks. Summary of the invention

[0006] In view of the problems existing in the prior art, the present invention provides an image sensing system and an image acquisition device, which at least partially solve the problem of high power consumption existing in the prior art.

[0007] In a first aspect, an embodiment of the present disclosure provides an image sensing system, comprising a first structure and a second structure, wherein the first structure is used for imaging and energy collection, and the second structure is used for imaging and calculation, and the first structure and the second structure both comprise a plurality of photodiodes, and the photodiodes are set to a zero bias mode.

[0008] Optionally, when the system operates in a detection mode, the second structure is used for detection, and the first structure is used for energy collection. When a target is detected, both the first structure and the second structure are used for imaging.

[0009] Optionally, the second structure includes a first photodiode component, a capacitor, a first switch pair and a second switch pair, the first photodiode component includes a first photodiode and a first field effect transistor, the anode of the first photodiode is electrically connected to the gate of the first field effect transistor, the capacitor is connected in parallel with the first photodiode component, one end of the capacitor is electrically connected to the drain of the first field effect transistor, and the other end of the capacitor is electrically connected to the cathode of the first photodiode and the source of the first field effect transistor, respectively, the first switch pair and the second switch pair each include two switches, the first switch pair and the second switch pair are controlled by reverse signals, and the first switch pair and the second switch pair are used to electrically connect the first photodiode component and the capacitor to the V+ power line and the V- power line.

[0010] Optionally, based on the linear relationship between the drain current of the first field effect transistor and the photocurrent, the second structure implements the first layer of the BNN and directly outputs a binary activation value, which is expressed as:

[0011]

[0012] Among them, x irepresents the input light intensity, w i represents the weight, and y represents the output result.

[0013] Optionally, when w=1, When , it represents the weight w i =1, the first photodiode component is connected to the V+ power line, the capacitor is connected to the V- power line, when w=0, When , it represents the weight w i = 0, the first photodiode component is connected to the V- power line, the capacitor is connected to the V+ power line, w and It is a reverse signal.

[0014] Optionally, the first structure includes field effect transistor M1, field effect transistor M2, field effect transistor M3 and a second photodiode, the source of the field effect transistor M3 is electrically connected to the anode of the second photodiode, the drain of the field effect transistor M3 is electrically connected to the cathode of the second photodiode, the source of the field effect transistor M1 is electrically connected to the anode of the second photodiode, and the source of the field effect transistor M2 is electrically connected to the anode of the second photodiode.

[0015] Optionally, the working process of the first structure includes, first, turning on the field effect tube M3 for reset, so that the voltage of the second photodiode is reset to GND, then, the voltage of the anode of the second photodiode increases to a level proportional to the incident light, turning on the field effect tube M1 to output the pixel voltage, and finally, turning on the field effect tube M2 to output the collected energy, at which time the power of the energy output by the second photodiode is proportional to the incident light.

[0016] Optionally, the first structure and the second structure are in the same focal plane.

[0017] Optionally, the number ratio of the first structure to the second structure is 15:1.

[0018] In a second aspect, an embodiment of the present disclosure further provides an image acquisition device, comprising: an image sensing system as described in any one of the first aspects.

[0019] The image sensing system and image acquisition device provided by the present invention, wherein the image sensing system, by providing a first structure for imaging and energy collection and a second structure for imaging and calculation, has the first structure collecting solar energy to power the system, and the second structure performing calculations in the analog domain, thereby effectively reducing the power consumption of the system. Furthermore, because the photodiode operates in a zero-bias mode, a smaller number of devices are used, thereby achieving the purpose of reducing the power consumption and area of ​​the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0021] Figure 1a A block diagram of a 4×4 submodule structure of an image sensing system provided in an embodiment of the present disclosure;

[0022] Figure 1b An electronic circuit diagram of a second structure of an image sensing system provided by an embodiment of the present disclosure;

[0023] Figure 1c An electronic circuit diagram of a first structure of an image sensing system provided by an embodiment of the present disclosure;

[0024] Figure 1d A second structural timing diagram of the image sensing system provided by an embodiment of the present disclosure;

[0025] Figure 2 A block diagram of the self-powered in-pixel computing system architecture provided for an embodiment of the present disclosure;

[0026] Figure 3 A schematic diagram of an application scenario of a self-powered visual sensor system with pixel computing provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0028] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0029] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0030] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The illustrations only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0031] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0032] PD is photodiode, EH is energy harvesting, IPC is In Pixel Computing, BNN is binary neural network, and IM is imaging.

[0033] For ease of understanding, Figure 1a As shown, this embodiment discloses an image sensing system, including a first structure and a second structure, the first structure is used for imaging and energy collection, the second structure is used for imaging and calculation, the first structure and the second structure both include multiple photodiodes, and the photodiodes are set to zero bias mode.

[0034] Optionally, the number ratio of the first structure to the second structure is 15:1.

[0035] The system includes a 4×4 sub-pixel array, including 15 EH&IM pixel structures that can realize imaging and energy harvesting, i.e. the first structure. And 1 IM&IPC structure that can realize imaging and calculation, i.e. the second structure. It can effectively reduce the power consumption of the system and effectively reduce the area of ​​the system because the number of devices used is small.

[0036] Optionally, when the system operates in a detection mode, the second structure is used for detection, and the first structure is used for energy collection. When a target is detected, both the first structure and the second structure are used for imaging.

[0037] The system includes a 128×128 pixel array. In detection mode, 32×32 second structure pixels are used for detection, and the remaining first structure PDs in the same focal plane are used for energy harvesting. The collected energy is then used to power the second structure pixels involved in the calculation, thereby achieving self-powered calculation. After detecting the target, the system sends a trigger signal and starts the monitoring mode, configuring all 128×128 pixel arrays to imaging mode and taking pictures with full pixels.

[0038] Optionally, the second structure includes a first photodiode component, a capacitor, a first switch pair and a second switch pair, the first photodiode component includes a first photodiode and a first field effect transistor, the anode of the first photodiode is electrically connected to the gate of the first field effect transistor, the capacitor is connected in parallel with the first photodiode component, one end of the capacitor is electrically connected to the drain of the first field effect transistor, and the other end of the capacitor is electrically connected to the cathode of the first photodiode and the source of the first field effect transistor, respectively, the first switch pair and the second switch pair each include two switches, the first switch pair and the second switch pair are controlled by reverse signals, and the first switch pair and the second switch pair are used to electrically connect the first photodiode component and the capacitor to the V+ power line and the V- power line.

[0039] Optionally, based on the linear relationship between the drain current of the first field effect transistor and the photocurrent, the second structure implements the first layer of the BNN and directly outputs a binary activation value, which is expressed as:

[0040]

[0041] Among them, x i represents the input light intensity, w i represents the weight, and y represents the output result.

[0042] Optionally, when w=1, When , it represents the weight w i =1, the first photodiode component is connected to the V+ power line, the capacitor is connected to the V- power line, when w=0, When , it represents the weight w i = 0, the first photodiode component is connected to the V- power line, the capacitor is connected to the V+ power line, w and It is a reverse signal.

[0043] like Figure 1aAs shown in Figure 1, the system includes 15 EH&IM pixel structures that can realize imaging and energy collection and 1 IM&IPC structure that can realize imaging and calculation. Traditional visual perception processing requires the photoelectric signal to be converted into a digital signal through an ADC and then input into the processor. The power consumption of data conversion and data transmission caused by the ADC is very large. In this work, a structure for in-pixel calculation is proposed as shown in Figure 1 (b). This structure processes the signal in the analog domain and realizes the multiplication and accumulation process of the BNN convolution calculation.

[0044] In Pixel Computing (IPC) structure such as Figure 1b As shown in the figure, the structure processes the signal in the analog domain and realizes the multiplication and accumulation process of the BNN convolution calculation. In the IPC circuit, the photodiode works in zero-bias photovoltaic mode. When there is light, the photocurrent proportional to the light intensity will generate an open circuit voltage, which is expressed as:

[0045]

[0046] Where K is the Boltzmann constant, T is the temperature, and e is the electron charge. By applying a bias voltage slightly lower than the threshold voltage (V GS =V PD <V T ), so that the M1 tube works in the subthreshold region. In this mode, the transistor generates a drain current, which is expressed as:

[0047]

[0048] Where W is the channel width and L is the channel length. In zero bias mode, the photocurrent I generated by the photodiode ph Much larger than the dark current I diff , so according to formula 1, we can get the following approximate ratio, which is expressed as:

[0049]

[0050] Where n is the subthreshold slope factor. In the subsequent design, assuming n = 1, the proportional relationship between the transistor drain current and the photocurrent is obtained, and its expression is:

[0051] i D ∝I ph (4)

[0052] Therefore, using the linear relationship between transistor drain current and photocurrent, the IPC array can implement the first layer of BNN and directly output binary activation values, which are expressed as:

[0053]

[0054] Among them, x i Represents the input light intensity, which is divided into two categories according to the weights. The binary activation value y is calculated by comparing the sum of the elements in these two sets.

[0055] In the IPC structure, a photodiode and a capacitor C with the same equivalent capacitance are connected. PD In parallel, there is a V+ power line and a V- power line in each IPC structure. A pair of analog switches controlled by reverse signals are used to connect the photodiode and C PD Connect them to different V+ and V- lines to realize binary weights. For example, when w=1, When , it represents the weight w i =1, PD is connected to V+ line, capacitor C PD Connect to V-line. When w=0, When , it represents the weight w i =0, the photodiode is connected to the V- line, the capacitor C PD Connect to the V+ line.

[0056] like Figure 1d As shown, the IPC circuit realizes the calculation process in the pixel as follows: First, when w = 1, PD is connected to the V+ power line, and the capacitor C PD Connect to the V- power line. When the photocurrent starts to be calculated, the Precharge signal is set high to wait for PD and C PD Precharge to VDD and then disconnect, as capacitor PD and C PD Start to discharge to V+ and V- power lines. Among them, PD is affected by light, so that PD generates opposite photocurrent due to photoelectric effect, which affects the discharge speed of PD capacitor. Therefore, PD and C PD The difference in discharge speed will be reflected in the exposure time T exp The voltage on the V+ and V- power lines. The discharge amount of V+ and V- is ΔV + and ΔV - The expression is:

[0057]

[0058] After the exposure ends, a comparator is used to compare the voltages of V+ and V-. When V+ > V-, it indicates that the result of the multiply-accumulate operation is +1. The voltage on the V+ power line will be set to VDD, representing logic "1", and the voltage on the V- power line will be set to GND, representing logic "0". When V+ < V-, it indicates that the result of the multiply-accumulate operation is -1. The voltage on the V+ power line will be set to GND, representing logic "0", and the voltage on the V- power line will be set to VDD, representing logic "1". Finally, the voltage value on V+ being logic "0" or logic "1" is uniformly used as the result of the BNN convolution calculation.

[0059] Optionally, the first structure includes a field-effect transistor M1, a field-effect transistor M2, a field-effect transistor M3, and a second photodiode. The source of the field-effect transistor M3 is electrically connected to the anode of the second photodiode, the drain of the field-effect transistor M3 is electrically connected to the cathode of the second photodiode, the source of the field-effect transistor M1 is electrically connected to the anode of the second photodiode, and the source of the field-effect transistor M2 is electrically connected to the anode of the second photodiode.

[0060] Optionally, the working process of the first structure includes: first, turn on the field-effect transistor M3 for resetting, so that the voltage of the second photodiode is reset to GND. Subsequently, the voltage at the anode of the second photodiode increases to a level proportional to the incident light, turn on the field-effect transistor M1 to output the pixel voltage. Finally, turn on the field-effect transistor M2 to output the collected energy. At this time, the power of the energy output by the second photodiode is proportional to the incident light.

[0061] The first structure is a zero-biased dual-mode pixel structure as Figure 1c shown. The zero-biased dual-mode pixel structure uses a zero-biased photodiode. The zero-biased photodiode is usually used in the photovoltaic mode. When the zero-biased photodiode is used in the imaging mode, the voltage value can be directly read and mapped to the imaging image. The disadvantage of this zero-biased photodiode is that its response speed to light is slow. In an environment with low light intensity, a longer exposure time is required for sensing, and it is not suitable for image sensors with high frame rates. The working process of the pixel structure is as follows: first, turn on the field-effect transistor M3 for resetting, so that the PD voltage is reset to GND. Subsequently, the voltage at the anode of the PD increases to a level proportional to the incident light, turn on the field-effect transistor M1 to output the pixel voltage. Finally, turn on the field-effect transistor M2 to output the collected energy. At this time, the power of the energy output by the PD is proportional to the incident light. The collected energy will be supplied to the IPC module after power management.

[0062] Optionally, the first structure and the second structure are on the same focal plane.

[0063] The system disclosed in this embodiment includes a 128×128 pixel array. In detection mode, 32×32 pixels are used for detection, and the remaining PDs in the same focal plane are used for energy harvesting. The collected energy is then used to power the pixels involved in the calculation, thereby achieving self-powered calculation. After detecting the target, the system sends a trigger signal and starts the monitoring mode, configuring all 128×128 pixel arrays to imaging mode and taking pictures with full pixels.

[0064] The system operates in two modes: detection mode and monitoring mode. In the detection mode, in the 4×4 submodule, one In Pixel Computing (IPC) pixel is used for detection, and the remaining 15 pixels are configured in photovoltaic mode to harvest energy during the detection mode. The 32×32 pixel array in the pixel array is used for target detection, while all other pixels are used for energy harvesting to power the IPC array, thereby achieving self-powered computing. In order to reduce system power consumption, the IPC array performs the first layer of BNN multiplication-accumulation (MAC) operations in the analog domain and directly outputs binary activation values. The feature map of the first layer of the BNN is input to the off-chip BNN processing module for subsequent operations. After the target is detected, the trigger signal starts the monitoring mode, and all 128×128 pixels are imaged at full resolution, and then the full-precision image is output through the readout circuit.

[0065] In this embodiment, the system is designed based on the perception, energy collection, and in-pixel computing structure of zero-bias photodiodes. This dual-mode pixel structure effectively reduces the power consumption of the system and effectively reduces the area of ​​the system because of the smaller number of devices used.

[0066] In addition, this embodiment also discloses an image acquisition device, including: the image sensing system disclosed in this embodiment. The image acquisition device includes a camera and the like.

[0067] In a specific application, the image acquisition device architecture is as follows Figure 2As shown in the figure, it includes a pixel array of size 128×128. Among them, the pixel array is divided into 32×32 sub-modules containing 4×4 pixel units. It also includes a register array for storing data, a comparator array for calculation, a readout circuit for imaging, a digital controller for controlling the system, a power management unit (Power Management Unit, PMU) for energy harvesting outside the chip, and a binary neural network processing module. The system has two working modes: detection mode and monitoring mode. In the detection mode, in the 4×4 sub-module, 1 pixel of the in-pixel computing structure (In Pixel Computing, IPC) is used for detection, and the remaining 15 pixels are configured in photovoltaic mode for energy harvesting during the detection mode. The 32×32 in-pixel computing structure array in the pixel array is used for target detection, while all other pixels are used for energy harvesting to power the IPC array, thereby realizing self-powered computing. In order to reduce system power consumption, the IPC array performs the first layer of BNN multiply accumulate (MAC) operation in the analog domain and directly outputs binary activation values. The feature map of the first layer of the BNN is input to the off-chip BNN processing module for subsequent operations. After the target is detected, the trigger signal starts the monitoring mode, and all 128×128 pixels are imaged at full resolution, and then the full-precision image is output through the readout circuit.

[0068] In specific applications, intelligent visual perception devices are widely deployed in IoT application scenarios, such as smart factories, smart transportation, new energy vehicles, etc. Figure 3 As shown, the new energy vehicle has smart cameras deployed on both sides of the vehicle body, which are used to monitor the surrounding environment in sentinel mode. For example, the official instructions for use of a new energy vehicle point out that when the sentinel mode is active, power consumption may increase, and the high power consumption sentinel mode will reduce the mileage of the new energy vehicle. Based on the above scenario, this embodiment proposes a self-powered sensing and computing (self-senputing) architecture with in-pixel computing. Figure 3 As shown in the figure, the system includes a 128×128 pixel array. In detection mode, 32×32 pixels are used for detection, and the remaining PDs in the same focal plane are used for energy harvesting. The collected energy is then used to power the pixels involved in the calculation, thereby achieving self-powered calculation. After detecting the target, the system sends a trigger signal and starts the monitoring mode, configuring all 128×128 pixel arrays into imaging mode, and taking pictures with all focal plane pixels.

[0069] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0070] In the present disclosure, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. The block diagrams of the devices, devices, equipment, and systems involved in the present disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagram. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including", "comprising", "having", etc. are open words, referring to "including but not limited to", and can be used interchangeably with them. The words "or" and "and" used here refer to the words "and / or" and can be used interchangeably with them, unless the context clearly indicates otherwise. The words "such as" used here refer to the phrase "such as but not limited to", and can be used interchangeably with them.

[0071] Additionally, as used herein, "or" used in a list of items beginning with "at least one" indicates a separate list, so that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the example described is preferred or better than other examples.

[0072] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0073] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.

[0074] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0075] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An image sensing system, characterized in that: The invention comprises a first structure and a second structure, wherein the first structure is used for imaging and energy collection, and the second structure is used for imaging and calculation, and the first structure and the second structure both comprise a plurality of photodiodes, and the photodiodes are set to a zero bias mode.

2. The image sensing system according to claim 1, characterized in that: When the system works in the detection mode, the second structure is used for detection, and the first structure is used for energy collection. When a target is detected, both the first structure and the second structure are used for imaging.

3. The image sensing system according to claim 1, characterized in that: The second structure includes a first photodiode component, a capacitor, a first switch pair and a second switch pair. The first photodiode component includes a first photodiode and a first field effect transistor. The anode of the first photodiode is electrically connected to the gate of the first field effect transistor. The capacitor is connected in parallel with the first photodiode component. One end of the capacitor is electrically connected to the drain of the first field effect transistor, and the other end of the capacitor is electrically connected to the cathode of the first photodiode and the source of the first field effect transistor respectively. The first switch pair and the second switch pair each include two switches. The first switch pair and the second switch pair are controlled by reverse signals. The first switch pair and the second switch pair are used to electrically connect the first photodiode component and the capacitor to the V+ power line and the V- power line.

4. The image sensing system according to claim 3, characterized in that: Based on the linear relationship between the drain current and photocurrent of the first field effect tube, the second structure implements the first layer of BNN and directly outputs the binary activation value, which is expressed as: Among them, x i represents the input light intensity, w i represents the weight, and y represents the output result.

5. The image sensing system according to claim 4, characterized in that: When w = 1, When , it represents the weight w i =1, the first photodiode component is connected to the V+ power line, the capacitor is connected to the V- power line, when w=0, When , it represents the weight w i = 0, the first photodiode component is connected to the V- power line, the capacitor is connected to the V+ power line, w and It is a reverse signal.

6. The image sensing system according to claim 1, characterized in that: The first structure includes a field effect transistor M1, a field effect transistor M2, a field effect transistor M3 and a second photodiode, the source of the field effect transistor M3 is electrically connected to the anode of the second photodiode, the drain of the field effect transistor M3 is electrically connected to the cathode of the second photodiode, the source of the field effect transistor M1 is electrically connected to the anode of the second photodiode, and the source of the field effect transistor M2 is electrically connected to the anode of the second photodiode.

7. The image sensing system according to claim 6, characterized in that: The working process of the first structure includes, firstly, turning on the field effect tube M3 for resetting so that the voltage of the second photodiode is reset to GND, then, the voltage of the anode of the second photodiode increases to a level proportional to the incident light, turning on the field effect tube M1 to output the pixel voltage, and finally, turning on the field effect tube M2 to output the collected energy, at which time the power of the energy output by the second photodiode is proportional to the incident light.

8. The image sensing system according to claim 1, characterized in that: The first structure and the second structure are in the same focal plane.

9. The image sensing system according to claim 1, characterized in that: The number ratio of the first structure to the second structure is 15:

1.

10. An image acquisition device, characterized in that: include: An image sensing system as claimed in any one of claims 1 to 9.

Images