Image sensing circuit and method, chip, and electronic device therefor

Abstract

The present disclosure relates to an image sensing circuit and a method, a chip and an electronic device used therefor. The image sensing circuit includes a macro-pixel block array, a column data buffer circuit, a row scanning circuit and a column data selection circuit. The macro-pixel block array includes a plurality of macro-pixel blocks. The macro-pixel block includes a plurality of macro-pixels. The macro-pixel includes a first sensor for sensing a change in light intensity. The row scanning circuit causes the first and second information of each macro-pixel block in a first row to be transmitted from the macro-pixel array to the column data buffer circuit when the first sensor of the macro-pixel block in the first row senses a change in light intensity that exceeds a threshold value. The first information indicates the change in light intensity sensed by the macro-pixel block. The second information indicates the change in light intensity sensed by the macro-pixel. The column data selection circuit outputs the second information of each macro-pixel in a first macro-pixel block and the position and time information of the macro-pixel block when it is determined that the first information of the macro-pixel block indicates a change in light intensity that exceeds the threshold value.

Description

Technical Field

[0001] This disclosure relates generally to the field of circuit technology, and more specifically to image sensing circuits and methods, chips, and electronic devices used therein. Background Technology

[0002] With the rapid development of image acquisition technology, people's requirements for image quality are increasing daily. They not only want to capture images of static scenes but also those of dynamic scenes. When photographing fast-moving objects, unrealistic distortions and motion blur may occur, a phenomenon also known as the rolling shutter effect. Therefore, some image sensing circuits utilize sensors that detect changes in light intensity to help recover blurred information in high-speed photography scenes. Summary of the Invention

[0003] This disclosure provides an image sensing circuit, a method for using an image sensing circuit, a chip, and an electronic device.

[0004] According to a first aspect of this disclosure, an image sensing circuit is provided. The image sensing circuit includes a macropixel block array, a column data buffer circuit, a row scanning circuit, and a column data selection circuit. The macropixel block array includes a plurality of macropixel blocks arranged in an array. Each macropixel block includes a plurality of macropixels arranged in an array. Each macropixel includes a first sensor for sensing changes in illumination intensity. The row scanning circuit is configured to, when the first sensor in a first row of macropixel blocks senses a change in illumination intensity exceeding a threshold, transmit first information and second information of each macropixel block in the first row from the macropixel array to the column data buffer circuit. The first information indicates the illumination intensity change sensed by a single macropixel block. The second information indicates the illumination intensity change sensed by each macropixel in the single macropixel block. The column data selection circuit is configured to acquire the first information and second information stored by the column data buffer circuit, and, upon determining that the first information of the first macropixel block indicates a change in illumination intensity exceeding a threshold, output the second information of each macropixel in the first macropixel block, as well as the position and time information of the first macropixel block.

[0005] In this image sensing circuit, changes in illumination intensity are represented in units of macropixel blocks. Only information for macropixel blocks where an event has occurred (a change in illumination intensity exceeding a threshold) is output, thus reducing the amount of data in the output information. If the first information for a macropixel block indicates that the change in illumination intensity has not exceeded the threshold, then there is no need to output the second information for each macropixel within that macropixel block.

[0006] In some implementations of the first aspect, the image sensing circuit further includes multiple row control circuits. One of these row control circuits is configured to determine third information for a given row based on first information of each macropixel block, and to send a scan request signal to the row scanning circuit instructing it to scan the row if the third information indicates that the change in illumination intensity exceeds a threshold. Specifically, the third information for the row indicates that the change in illumination intensity exceeds the threshold if the first information of at least one macropixel block in the row indicates that the change in illumination intensity exceeds the threshold. The row scanning circuit is further configured to determine, upon receiving a scan request signal from the row control circuit corresponding to the first row, that the first sensor in the macropixel block of the first row has sensed a change in illumination intensity exceeding the threshold. This implementation, by setting up a row control circuit for each row of macropixel blocks to collect the first information for that row, facilitates the row scanning circuit in quickly understanding whether the entire row of macropixel blocks has sensed a change in illumination intensity.

[0007] In some implementations of the first aspect, each macropixel in each macropixel block is connected to the column bus of its corresponding column via its own switch. The row scanning circuit is further configured to, when the first sensor in the first row of macropixel blocks senses a change in illumination intensity exceeding a threshold, open the switches between each macropixel in the first row and its corresponding column bus, thereby transmitting the first and second information of each macropixel block in the first row from the macropixel array to the column data buffer circuit. This implementation controls information transmission by setting switches connected to the column bus, thus it is simple to implement and occupies a small area.

[0008] In some implementations of the first aspect, a block control circuit is provided in each macropixel block. The block control circuit is configured to generate first and second information for the corresponding macropixel block. After the switch between each macropixel in the first row of macropixel blocks and its corresponding column bus is opened, the block control circuit in each macropixel block of the first row transmits the first and second information to the corresponding column bus, thereby enabling the first and second information to be transmitted to the column data buffer circuit. This implementation, by providing a block control circuit for each macropixel block, facilitates the statistical analysis of information within the block and simplifies the control logic of the image sensing circuit.

[0009] In some implementations of the first aspect, the image sensing circuit further includes a pixel value output circuit. Each of the plurality of macropixels also includes a second sensor that converts illumination intensity into a pixel value. The row scanning circuit is further configured to scan each row of macropixels in the macropixel block array line by line and provide the pixel values ​​converted by the second sensors in each scanned row to the pixel value output circuit. The pixel value output circuit is configured to store and output the pixel values ​​converted by the second sensors in each row. Compared to a scheme where event readout and pixel value scanning are implemented by two separate image sensing circuits, this implementation integrates event readout and pixel value scanning into a single image sensing circuit, thereby reducing the size of the image sensing circuit.

[0010] In some implementations of the first aspect, the first and second sensors in a single macropixel each include their own photodiodes. Since the first and second sensors do not share photodiodes, the first sensor can perform event detection while the second sensor is being exposed, without affecting the continuity of event detection. Event information and pixel value information at any given time can be obtained, sensed, and read independently.

[0011] In some implementations of the first aspect, the distance between photodiodes in any two adjacent first sensors among the plurality of macropixels is equal, and the distance between photodiodes in any two adjacent second sensors among the plurality of macropixels is also equal. This implementation, by arranging photodiodes at equal intervals, enables the resolution of the image sensing circuit to be proportional.

[0012] In some implementations of the first aspect, a single macropixel includes P second sensors, where P is a positive integer. P photodiodes of the P second sensors are arranged centered on the photodiodes of the first sensor in the macropixel. This implementation, by arranging the photodiodes of the first and second sensors in a centrally symmetrical manner, helps to proportionally align event information and pixel value information in space.

[0013] In some implementations of the first aspect, the photodiodes in the first and second sensors are shaped as rectangles or regular polygons. Setting the photodiode as a rectangle facilitates the overall layout and planning of the image sensing circuit. Setting the photodiode as a regular polygon allows for easier carrier flow by utilizing the obtuse angles of the polygon's boundaries. This is beneficial for the flow of photoelectric charge.

[0014] In some implementations of the first aspect, the pixel circuitry in the first sensor within a single macropixel is arranged on the same layer as the photodiode of the first sensor. In some alternative implementations, the pixel circuitry in the first sensor within a single macropixel is arranged on the lower layer of the photodiodes of the first and second sensors via a three-dimensional stacking method. In other words, the embodiments of this disclosure are applicable to both two-dimensional and three-dimensional chips, have a wide range of applications, and are not limited by chip package type.

[0015] In some implementations of the first aspect, the photodiode in the first sensor within a single macropixel comprises multiple separate parts. The geometric center of these multiple parts is defined as the center point of the photodiode. The distance between the center points of any two adjacent photodiodes in the first sensor within these macropixels is equidistant. If a capacitor is present in the pixel circuitry of the first sensor, and if the capacitor used in the pixel circuitry needs to traverse the position of the photodiode in the first sensor, the photodiode must make way for the capacitor. This is because front-facing exposure requires the photodiode to be completely exposed to the surrounding metal; therefore, the photodiode is not vertically compatible with either a metal-insulator-metal capacitor (MIM capacitor) or a metal-silicon oxide-metal capacitor (MOM capacitor) composed of the metal above it. In complementary metal oxide semiconductor (CMOS) processes, capacitors fabricated with a metal-insulator-metal structure are called MIM capacitors. Capacitors fabricated with a metal-silicon oxide-metal structure are called MOM capacitors.

[0016] According to a second aspect of this disclosure, a chip is provided. The chip includes an image sensing circuit according to a first aspect of this disclosure, and a package structure. The package structure surrounds the image sensing circuit.

[0017] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes an image sensing circuit according to a first aspect of this disclosure, and a power supply circuit. The power supply circuit is configured to supply power to the image sensing circuit.

[0018] According to a fourth aspect of this disclosure, a method for an image sensing circuit is provided. The image sensing circuit includes: a macropixel block array, a row scanning circuit, a column data buffer circuit, and a column data selection circuit. The macropixel block array includes a plurality of macropixel blocks arranged in an array. Each macropixel block includes a plurality of macropixels arranged in an array. Each macropixel includes a first sensor for sensing changes in illumination intensity. The method includes generating first information and second information for each macropixel block. The first information indicates a change in illumination intensity sensed by a single macropixel block. The second information indicates a change in illumination intensity sensed by each macropixel in the single macropixel block. The method further includes sending a scan request signal instructing the row scanning circuit to scan the row if the first information of a macropixel block in a first row indicates a change in illumination intensity exceeding a threshold. The method further includes, upon receiving the scan request signal, causing the row scanning circuit to transfer the first information and second information of each macropixel block in the row corresponding to the scan request signal from the macropixel array to the column data buffer circuit. The method further includes the column data buffer circuit storing the first information and the second information. The method further includes a column data selection circuit obtaining first information and second information from a column data cache circuit, and when it is determined that the change in illumination intensity of the first information of the first macropixel block exceeds a threshold, outputting the second information of each macropixel in the first macropixel block as well as the position information and time information of the macropixel block.

[0019] It is understood that the chip of the second aspect, the electronic device of the third aspect, and the method for the image sensing circuit of the fourth aspect provided above have the same beneficial effects as the image sensing circuit of the first aspect. Therefore, the explanation or description of the first aspect also applies to the second, third, and fourth aspects. Attached Figure Description

[0020] The above and other objects, features and advantages of this disclosure will become more apparent from the accompanying drawings, in which like reference numerals generally denote like parts.

[0021] Figure 1 The illustration shows a schematic diagram of an example environment in which an image sensing circuit and / or a method for an image sensing circuit, according to embodiments of the present disclosure, may be implemented.

[0022] Figure 2A An exemplary block diagram of an image sensing circuit according to an embodiment of the present disclosure is shown.

[0023] Figure 2B An exemplary schematic diagram of a macropixel block according to an embodiment of the present disclosure is shown.

[0024] Figure 3Another exemplary block diagram of an image sensing circuit according to an embodiment of the present disclosure is shown.

[0025] Figure 4 An exemplary flowchart of a method for an image sensing circuit according to an embodiment of the present disclosure is shown.

[0026] Figure 5 Another exemplary block diagram of an image sensing circuit according to an embodiment of the present disclosure is shown.

[0027] Figure 6 An exemplary schematic diagram of a macropixel according to an embodiment of the present disclosure is shown.

[0028] Figure 7 A first exemplary arrangement diagram of a plurality of macro pixels according to an embodiment of the present disclosure is shown.

[0029] Figure 8 A second exemplary arrangement diagram of a plurality of macropixels according to an embodiment of the present disclosure is shown.

[0030] Figure 9A A third exemplary arrangement diagram of a plurality of macro pixels according to an embodiment of the present disclosure is shown.

[0031] Figure 9B A fourth exemplary arrangement diagram of a plurality of macro pixels according to an embodiment of the present disclosure is shown.

[0032] Figure 10 A fifth exemplary arrangement diagram of a plurality of macro pixels according to an embodiment of the present disclosure is shown.

[0033] Figure 11 A sixth exemplary arrangement diagram of a plurality of macropixels according to embodiments of the present disclosure is shown.

[0034] Figure 12 A seventh exemplary arrangement diagram of a plurality of macro pixels according to an embodiment of the present disclosure is shown.

[0035] Figure 13 An eighth exemplary arrangement diagram of a plurality of macro pixels according to an embodiment of the present disclosure is shown.

[0036] Figure 14 Another exemplary schematic diagram of a macropixel according to an embodiment of the present disclosure is shown.

[0037] Figure 15 A comparison diagram is shown of the amount of data that an image sensing circuit according to an embodiment of the present disclosure needs to process, compared to another image sensing circuit.

[0038] In the various accompanying figures, the same or corresponding reference numerals indicate the same or corresponding parts. The elements in the accompanying figures are schematic and not drawn to scale. Detailed Implementation

[0039] The technical solutions of this disclosure will now be described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments.

[0040] The technical solutions of the embodiments of this disclosure will now be described with reference to the accompanying drawings. In the description of the embodiments of this disclosure, unless otherwise stated, " / " signifies "and / or," for example, A / B can mean A or B, or A and B. "And / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Furthermore, in the description of the embodiments of this disclosure, "plural" or "multiple" refers to two or more than two.

[0041] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0042] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. As used in the specification and appended claims of this disclosure, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this disclosure, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0043] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this disclosure include a particular feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "one embodiment," "some embodiments," "another embodiment," "other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0044] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this subject matter pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the specification and in the relevant art, and shall not be interpreted in an idealized or overly formal form unless otherwise explicitly defined herein. As used herein, the statement of “connecting” or “coupling” two or more parts together shall mean that these parts are directly joined together or joined through at least one intermediate component.

[0045] For ease of description, spatial relative terms such as “above,” “below,” “left,” “right,” “top,” and “bottom” are used here to describe the spatial positional relationship of a device or element to other devices or elements, as shown in the figures. For example, the terms “on,” “above,” “above,” “on the upper surface of,” “above,” “positioned on,” or “positioned on top of” mean that a first element, such as a first structure, exists on a second element, such as a second structure, where an intermediate element may or may not be present between the first and second elements. The term “contact” means connecting a first element, such as a first structure, and a second element, such as a second structure, where there may or may not be other elements at the interface between the two elements. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation in addition to the orientation of the device as described in the figures. For example, if the device in the figures is inverted, a device described as “above” or “on top of” other devices or structures will subsequently be positioned “below” or “under” other devices or structures. Thus, the exemplary term “above” can include both orientations of “above” and “below”. The device can also be positioned in other different ways, rotated 90 degrees, or in other orientations, and the spatial relative descriptions used here are explained accordingly.

[0046] The numbers or values ​​used in this specification are illustrative and are intended only to facilitate understanding of the techniques of the embodiments of this disclosure, and are by no means intended to limit the scope of this disclosure.

[0047] The methods provided in this disclosure can be applied to mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), or other electronic devices with camera functions. This disclosure does not limit the specific type of electronic device.

[0048] The following section will first provide a detailed description of some terms that may be involved in the embodiments of this disclosure.

[0049] 1. Dynamic range

[0050] Dynamic range refers to a sensor's ability to detect differences in brightness within the same frame, which is the ratio of the highest detectable brightness to the lowest detectable brightness, expressed in decibels (dB).

[0051] 2. Address Event Representation (AER) Method

[0052] The AER (Asynchronous Erector Design) method uses an asynchronous circuit to scan each pixel on the image sensor and read its address and event information to characterize the event. Here, the event refers to a change in light intensity.

[0053] 3. Dark current

[0054] Dark current refers to the direct current generated by a reverse-biased photodiode when there is no light input.

[0055] 4. Three-dimensional stacking

[0056] Three-dimensional stacking refers to the vertical integration of two chips through silicon vias and bumps, or copper-copper (Cu-Cu) bonding. The interaction between the upper and lower chips is carried out through silicon vias or Cu-Cu bonding.

[0057] 5. Correlated double sampling (CDS)

[0058] CDS removes background noise from pixels by sampling the output signals of the same pixel under both lit and unlit conditions and subtracting the accumulated charge.

[0059] 6. Global Exposure

[0060] Global exposure refers to the simultaneous exposure of all pixels on an image sensor, starting and ending at the same time. The accumulated charge of each pixel is read out by the readout circuitry. The resulting image frame represents the real scene at a specific moment.

[0061] 7. Roller shutter exposure

[0062] Rolling shutter exposure refers to the sequential exposure of pixels on an image sensor, row by row. After the first row is exposed, the accumulated charge of the pixels in that row is read sequentially. Simultaneously, after the first row is exposed, the next row begins the exposure-readout process. This pipeline-like operation is called rolling shutter exposure. In high-speed photography, the timing of each row's original image capture may be inconsistent, resulting in the rolling shutter effect.

[0063] 8. Jelly effect

[0064] The rolling shutter effect refers to the physical tilting and repetition that occurs when photographing a high-speed object using progressive exposure. The image resembles a swaying jelly, hence the name.

[0065] 9. Metal-Insulator-Metal Capacitor (MIM Capacitor)

[0066] In complementary metal oxide semiconductor (CMOS) technology, capacitors made with a metal-insulator-metal structure are called MIM capacitors.

[0067] 10. Metal-silicon oxide-metal capacitors (MOM capacitors)

[0068] In CMOS technology, capacitors made with a metal-silicon oxide-metal structure are called MOM capacitors.

[0069] In some image sensing circuits, traditional optical imaging sensors (i.e., sensors that sense light intensity or convert light intensity into pixel values) are used to record the environment and image texture, that is, to acquire the pixel values ​​of each pixel in the image (which can also be referred to as "grayscale information" in this context). Such optical imaging sensors include grayscale sensors or red-green-blue (RGB) sensors. RGB sensors are implemented by adding optical filters to grayscale cameras. In optical imaging sensors, charge is accumulated by exposing the photodiodes (PDs) in each pixel to the light for a period of time. The accumulated charge is read out by CMOS circuitry and then read through readout circuits (ROICs) such as CDS, amplifiers, and analog-to-digital converters (ADCs). All pixels after exposure constitute a frame of image. The time difference of the original information between the signals of pixels in each frame is determined by the exposure method. Typically, global exposure records information at the same time. Other exposure methods, such as rolling shutter exposure, operate in a streamlined manner, exposing and converting data line by line. While this increases overall circuit utilization, it can cause a rolling shutter effect when photographing fast-moving objects. Grayscale sensors include active pixel sensors (APS) and passive pixel sensors (PPS). Active pixel sensors are also known as active pixel sensors. Passive pixel sensors are also known as passive pixel sensors.

[0070] To mitigate the rolling shutter effect, sensors that detect changes in light intensity are incorporated into some image sensing circuits. These sensors can be called event-triggered sensors or event sensors. Examples of event sensors include dynamic visual sensors (DVS) and event-based vision sensors (EVS). When a pixel senses a change in light intensity (increase / decrease), the event sensor calculates its logarithm and compares the result to a threshold for positive / negative events (increase / decrease) to determine whether an event has occurred at that pixel, i.e., a change in light intensity. The presence or absence of an event is represented by 1 bit, and the positive or negative (increase / decrease) of the event is also represented by 1 bit. Event sensors can use AER (Advanced Erratic Response) readout, which involves scanning all pixels using row and column scanning circuits and sequentially outputting the 2-bit event information from the pixel where the event occurred, along with the time and pixel address. In short, event sensors can be used to capture signals of changes in light intensity as events. Based on the way event sensors record information, they possess a high dynamic range. Event sensors capture event information for each pixel, which can help supplement grayscale information in high-speed camera scenes and help restore blurred information.

[0071] Therefore, a multimodal vision imaging system utilizing the two types of sensors described above can leverage the advantages of various sensors. The fusion of grayscale and event sensors can simultaneously generate two types of image information, overcoming the limitations of the dynamic range of grayscale sensors and providing grayscale sensors with image processing possibilities such as high-speed photography deblurring. The two sensors complement each other, making them suitable for computer vision tasks such as high-speed photography deblurring or strong light exposure reduction.

[0072] Although event sensors can complement grayscale sensors in terms of their strengths and weaknesses, event sensors output a larger amount of information (or data). Therefore, given the limited data capacity of downstream devices, the number of times event information can be captured within a single image frame is limited. Increasing the frequency of event information capture, even with limited data, is more beneficial for supplementing details in grayscale information and helping to recover blurred information.

[0073] To address this, this disclosure proposes an image sensing circuit. The image sensing circuit includes a macropixel block array, a column data buffer circuit, a row scanning circuit, and a column data selection circuit. The macropixel block array includes a plurality of macropixel blocks arranged in an array. Each macropixel block includes a plurality of macropixels arranged in an array. Each macropixel includes a first sensor for sensing changes in illumination intensity. The row scanning circuit is configured such that, when the first sensor in a first row of macropixel blocks senses a change in illumination intensity exceeding a threshold, first information and second information of each macropixel block in the first row are transmitted from the macropixel array to the column data buffer circuit. The first information indicates the change in illumination intensity sensed by a single macropixel block. The second information indicates the change in illumination intensity sensed by each macropixel in that single macropixel block. The column data selection circuit is configured to acquire the first and second information stored in the column data buffer circuit, and, upon determining that the first information of the first macropixel block indicates a change in illumination intensity exceeding a threshold, output the second information of each macropixel in the first macropixel block, as well as the position and time information of the first macropixel block.

[0074] In this image sensing circuit, changes in illumination intensity are represented in units of macropixel blocks. Only information for macropixel blocks where an event has occurred (a change in illumination intensity exceeding a threshold) is output, thus reducing the amount of data in the output information. If the first information for a macropixel block indicates that the change in illumination intensity has not exceeded the threshold, then there is no need to output the second information for each macropixel within that macropixel block.

[0075] Figure 1 A schematic diagram of an example environment 100 in which the image sensing circuit and / or methods for the image sensing circuit of embodiments of the present disclosure may be implemented is shown. The image sensing circuit 200 is, for example, disposed in an image acquisition device 110. The image acquisition device 110 includes, for example, a mobile phone, tablet computer, wearable device, in-vehicle device, AR / VR device, laptop computer, UMPC, netbook, PDA, or other electronic device with a camera function. The embodiments of the present disclosure do not limit the specific type of the image acquisition device 110. The image sensing circuit 200 according to embodiments of the present disclosure can be used to acquire event information of high-speed moving objects. High-speed moving objects include, for example, a high-speed rotating fan blade 120, a high-speed train (not shown in the diagram). Figure 1 (as shown in the figure). It should be understood that the image sensing circuit 200 according to embodiments of the present disclosure is also applicable to acquiring event information of static objects.

[0076] Figure 2AAn exemplary block diagram of an image sensing circuit 200 according to an embodiment of the present disclosure is shown. The image sensing circuit 200 includes a macropixel block array 210, a column data buffer circuit 220, a row scanning circuit 240, and a column data selection circuit 230.

[0077] Macropixel block array 210 is coupled to row scan circuit 240 and column data buffer circuit 220. Macropixel block array 210 includes multiple macropixel blocks arranged in an array. Figure 2A In the example, the macro pixel block array 210 includes i×j macro pixel blocks. That is, the macro pixel block array 210 includes macro pixel blocks in row i and column j. For example, macro pixel block MP11 represents the macro pixel block with row number 1 and column number 1. Macro pixel block MP1j represents the macro pixel block with row number 1 and column number j. Macro pixel block MPi1 represents the macro pixel block with row number i and column number 1. Macro pixel block MPij represents the macro pixel block with row number i and column number j. It should be noted that in Figure 2A Not all macro pixel blocks in macro pixel block array 210 are shown. Macro pixel blocks MP11, MP1j, MPi1, and MPij are shown only as example macro pixel blocks in macro pixel block array 210.

[0078] The macropixel blocks comprise multiple macropixels arranged in an array. Each macropixel includes a first sensor for sensing changes in light intensity. The first sensor may include, for example, a DVS or EVS. An exemplary structure of a macropixel block according to embodiments of this disclosure can be found... Figure 2B .exist Figure 2B In the example, a macropixel block comprises m × n macropixels. That is, a macropixel block consists of macropixels in m rows and n columns. For example, macropixel (1, 1) represents a macropixel with row number 1 and column number 1. Macropixel (1, n) represents a macropixel with row number 1 and column number n. Macropixel (m, 1) represents a macropixel with row number m and column number 1. Macropixel (m, n) represents a macropixel with row number m and column number n. Note that the row and column numbers of a macropixel represent its row and column numbers within the corresponding macropixel block. Each macropixel block can include the same macropixel structure. Figure 2B Not all macro pixels in the macro pixel block are shown. Macro pixels (1,1), (1,n), (m,1), and (m,n) are shown only as example macro pixels in the macro pixel block.

[0079] Row scanning circuit 240 is coupled to macropixel block array 210. Row scanning circuit 240 is configured to sequentially scan each row of macropixel blocks. Row scanning circuit 240 is also configured to, when a first sensor in a macropixel block of the first row senses a change in illumination intensity exceeding a threshold, transmit first and second information of each macropixel block in the first row from the macropixel array to column data buffer circuit 220. First information indicates the illumination intensity change sensed by a single macropixel block. Second information indicates the illumination intensity change sensed by each macropixel in that single macropixel block. Note that, in this context, "first row" refers to any row, not specifically the first row. "Illumination intensity change sensed by a macropixel block" includes whether the illumination intensity of the macropixel block has changed (which can be indicated by 1 bit). "Illumination intensity change sensed by a macropixel" includes whether the illumination intensity of the macropixel has changed (which can be indicated by 1 bit), and the direction of the change in the illumination intensity of the macropixel (increase or decrease, which can be indicated by 1 bit).

[0080] Assumption Figure 2A If the first sensor in macropixel block MP1j senses a change in illumination intensity exceeding a threshold, the row scanning circuit 240 can transmit the first and second information of each macropixel block MP11, ..., MP1j with row number 1 from the macropixel array to the column data buffer circuit 220. In this case, the first information of macropixel block MP1j is a first value, indicating that the change in illumination intensity exceeds the threshold. If the first sensor in macropixel block MP11 senses a change in illumination intensity that does not exceed the threshold, the first information of macropixel block MP11 is a second value, indicating that the change in illumination intensity does not exceed the threshold. If the first sensor in macropixel block MP11 senses a change in illumination intensity that exceeds the threshold, the first information of macropixel block MP11 is a first value, indicating that the change in illumination intensity exceeds the threshold. In one example, the first value is 1 and the second value is 0. In another example, the first value is 0 and the second value is 1.

[0081] In some embodiments of this disclosure, the first bit of the second information indicates whether the light intensity has changed, and the second bit of the second information indicates the direction of the change in light intensity. In one example, if the first bit of the second information is a first value, it indicates that the change in light intensity exceeds a threshold. If the first bit of the second information is a second value, it indicates that the change in light intensity does not exceed the threshold. In another example, if the second bit of the second information is a first value, it indicates that the direction of the change in light intensity is increasing. If the second bit of the second information is a second value, it indicates that the change in light intensity is decreasing. In one example, the first value is 1 and the second value is 0. In another example, the first value is 0 and the second value is 1.

[0082] In other embodiments of this disclosure, the first bit of the second information indicates the direction of change in light intensity, and the second bit of the second information indicates whether the light intensity has changed. In one example, if the first bit of the second information is a first value, it indicates that the direction of change in light intensity is increasing. If the first bit of the second information is a second value, it indicates that the amount of change in light intensity is decreasing. In another example, if the second bit of the second information is a first value, it indicates that the amount of change in light intensity exceeds a threshold. If the second bit of the second information is a second value, it indicates that the amount of change in light intensity does not exceed a threshold. In one example, the first value is 1 and the second value is 0. In another example, the first value is 0 and the second value is 1.

[0083] Column data cache circuit 220 is coupled to macropixel block array 210 and column data selection circuit 230. Column data cache circuit 220 is capable of storing first information and second information from the macropixel array. When multiple first sensors in multiple rows of macropixel blocks sense changes in light intensity exceeding a threshold, column data cache circuit 220 is capable of storing the first and second information corresponding to these rows in the chronological order in which these rows were scanned.

[0084] Column data selection circuit 230 is coupled to column data buffer circuit 220. Column data selection circuit 230 is configured to acquire first information and second information stored by column data buffer circuit 220, and, if it is determined that the change in illumination intensity indicated by the first information of the first macropixel block exceeds a threshold, output the second information of each macropixel in the first macropixel block, as well as the position and time information of the first macropixel block. It should be noted that, in this context, "first macropixel block" refers to any macropixel block, not specifically the first macropixel block.

[0085] In this image sensing circuit 200, changes in illumination intensity are indicated on a macropixel block basis. Only information for macropixel blocks where an event occurs (a change in illumination intensity exceeding a threshold) is output, thus reducing the amount of data in the output information. If the first information indicating a change in illumination intensity for a macropixel block does not exceed the threshold, there is no need to output second information for each macropixel within that macropixel block. Compared to an image sensing circuit 200 that outputs second information for each macropixel in the macropixel block array 210, the image sensing circuit 200 according to embodiments of the present disclosure can significantly reduce the amount of data in the output information. Thus, with a limited total amount of output information, the image sensing circuit 200 according to embodiments of the present disclosure can increase the frequency of generating event information, thereby outputting more frames of event information in the same amount of time.

[0086] Figure 3 A more detailed exemplary block diagram of an image sensing circuit 200 according to an embodiment of the present disclosure is shown. Figure 3 In the example, the image sensing circuit 200 also includes multiple row control circuits. Each row control circuit corresponds to a row of macropixel blocks. Each row control circuit is configured to determine third information for that row based on first information of each macropixel block in the corresponding row, and to send a scan request signal to the row scanning circuit 240 indicating that the change in illumination intensity exceeds a threshold if the third information indicates that the change exceeds a threshold. Specifically, the third information for that row indicates that the change in illumination intensity exceeds a threshold if the first information of at least one macropixel block in the row indicates that the change exceeds a threshold. If the first sensor in macropixel block MP1j senses that the change in illumination intensity exceeds a threshold, the first information of macropixel block MP1j indicates that the change exceeds a threshold. Accordingly, the third information for row number 1 indicates that the change exceeds a threshold. The first row control circuit sends a scan request signal X_Greq1 to the row scanning circuit 240 indicating that the row is being scanned. The third information may be included in the scan request signal X_Greq1. If the first sensor in macropixel block MPi1 detects a change in light intensity exceeding a threshold, the first information of macropixel block MPi1 indicates that the change in light intensity exceeds the threshold. Correspondingly, the third information for row number i indicates that the change in light intensity exceeds the threshold. The i-th row control circuit sends a scan request signal X_Greqi to the row scanning circuit 240, instructing it to scan that row.

[0087] The row scanning circuit 240 is configured to determine, upon receiving a scan request signal from the row control circuit corresponding to the first row, that the first sensor in the macropixel block of the first row senses a change in illumination intensity exceeding a threshold. For example, upon receiving a scan request signal X_Greq1 indicating the row to be scanned (row number 1), the row scanning circuit 240 determines that the first sensor in the macropixel block of row number 1 senses a change in illumination intensity exceeding the threshold. Similarly, upon receiving a scan request signal X_Greqi indicating the row to be scanned (row number i), the row scanning circuit 240 determines that the first sensor in the macropixel block of row number i senses a change in illumination intensity exceeding the threshold. By setting up a row control circuit for each row of macropixel blocks to collect the first information for that row, the row scanning circuit 240 can quickly determine whether the entire row of macropixel blocks has sensed a change in illumination intensity.

[0088] In some embodiments of this disclosure, each macropixel in each macropixel block is connected to the column bus of the corresponding column via its own switch (not shown). Each macropixel is connected to the column bus of the corresponding column via a separate switch. The row scanning circuit 240 is configured to, when the first sensor in the macropixel block of the first row senses a change in illumination intensity exceeding a threshold, open the switch between each macropixel in the macropixel block of the first row and the corresponding column bus, thereby transmitting the first and second information of each macropixel block of the first row from the macropixel array to the column data buffer circuit 220. When the first sensor in macropixel block MP11 senses a change in illumination intensity exceeding a threshold, the first information of macropixel block MP11 indicates that the change in illumination intensity exceeds the threshold. The first row control circuit sends a scan request signal X_Greq1 to the row scanning circuit 240, instructing it to scan the row. The row scanning circuit 240 responds with a response signal X_Gack1 to the macropixel block array 210, controlling the switch between each macropixel in all macropixel blocks of the first row (row number 1) and the corresponding column bus to open (i.e., enable). Thus, the first information Y_Greq1, ..., Y_Greqj of macropixel blocks MP11 to MP1j, and the second information, are transmitted to the data buffer circuit via the corresponding column bus. In one example, the second information of each macropixel in macropixel block MP11 is merged into 2×m×n-bit data Data1 in a predetermined order. Data Data1 is transmitted to the data buffer circuit as a whole. Similarly, the second information of each macropixel in macropixel block MP1j is merged into 2×m×n-bit data Dataj in a predetermined order. Data Dataj is transmitted to the data buffer circuit as a whole. When the first sensor in macropixel block MPi1 senses a change in light intensity exceeding a threshold, the first information of macropixel block MPi1 indicates that the change in light intensity exceeds the threshold. The i-th row control circuit sends a scan request signal X_Greqi to the row scanning circuit 240, instructing it to scan that row. The row scanning circuit 240 replies with a response signal X_Gacki to the macropixel block array 210, controlling the switch between each macropixel in all macropixel blocks of the i-th row and the corresponding column bus to open (i.e., enable). In this way, the first information Y_Greq1, ..., Y_Greqj of macropixel blocks MPi1 to MPij, and the second information, are transmitted to the data buffer circuit via the corresponding column bus. This embodiment controls the information transmission by setting a switch connected to the column bus, thus making the implementation simple and occupying a small area.

[0089] In some embodiments of this disclosure, each macropixel block is provided with a block control circuit (not shown). The block control circuit is configured to generate first information and second information for the corresponding macropixel block. After the switch between each macropixel in each macropixel block of the first row and the corresponding column bus is opened, the block control circuit in each macropixel block of the first row transmits the first information and second information to the corresponding column bus, thereby transmitting the first information and second information to the column data buffer circuit 220. When the row scan circuit 240 replies with the response signal X_Gack1 to the macropixel block array 210, the switch between each macropixel in macropixel block MP11 and the corresponding column bus is opened, and the block control circuit in macropixel block MP11 transmits the first information and second information of macropixel block MP11 to these column buses. Similarly, the switch between each macropixel in macropixel block MP1j and the corresponding column bus is opened, and the block control circuit in macropixel block MP1j transmits the first information and second information of macropixel block MP1j to these column buses. This implementation method facilitates the statistical analysis of information within each block and simplifies the control logic of the image sensing circuit 200 by setting up block control circuits for each macro-pixel block.

[0090] exist Figure 3 In the example, column data selection circuit 230 may include readout control circuit 232 and column scan circuit 231. Readout control circuit 232 may be configured to generate a row scan control signal RCL based on a clock signal and send the row scan control signal to row scan circuit 240. Row scan circuit 240 may scan scan request signals issued by each row control circuit according to the row scan control signal RCL. After receiving a scan request signal to scan the first row, row scan circuit 240 may send the row number (row address X_Addr) of that row to readout control circuit 232. Readout control circuit 232 may output the scan time TIME and row address X_Addr of the scan request signal. Readout control circuit 232 may also be configured to generate a column scan control signal CCL based on a clock signal and send the column scan control signal CCL to column data buffer circuit 220 and column scan circuit 231.

[0091] The column data cache circuit 220 can be configured to synchronously store the first and second information of each macro pixel block in the same row according to the column scan control signal CCL. When the first and second information of macro pixel blocks in multiple rows need to be stored, the column data cache circuit 220 can store the first and second information corresponding to these rows in the order in which these rows are scanned.

[0092] The column scan circuit 231 can be configured to acquire first and second information stored by the column data buffer circuit 220 according to the column scan control signal CCL, and determine which macropixel block(s) indicates that the change in illumination intensity exceeds a threshold based on the first information. If it is determined that the first information of the first macropixel block indicates that the change in illumination intensity exceeds the threshold, the column scan circuit 231 outputs the second information of each macropixel in the first macropixel block and the column address Y_Addr of that macropixel block.

[0093] In some embodiments of this disclosure, the column scan circuit 231 may send a readout scan signal X_Greqx to the column data buffer circuit 220. The readout scan signal X_Greqx can be used to read the first and second information stored in the column data buffer circuit 220 for each row. The column scan circuit 231 may determine which macropixel block(s) indicates a change in illumination intensity exceeding a threshold based on the first information for each macropixel block. If it is determined that the first information for a first macropixel block indicates a change in illumination intensity exceeding a threshold, the column scan circuit 231 outputs the second information of each macropixel in the first macropixel block (the second information of each macropixel is included in the data Data and output) and the column address Y_Addr of that macropixel block. The time TIME and row address X_Addr output by the readout control circuit 232, and the second information and column address Y_Addr output by the column scan circuit 231, can be jointly provided to downstream devices so that downstream devices know which macropixel block(s) sensed a change in illumination intensity at what time. Figure 3 In the example, the time information mentioned above includes the time TIME. The location information mentioned above includes the row address X_Addr and the column address Y_Addr.

[0094] Figure 4 An exemplary flowchart of a method for an image sensing circuit 200 according to an embodiment of the present disclosure is shown. The following is in conjunction with... Figure 3 To illustrate with examples Figure 4 The process.

[0095] At block 402, the block control circuitry in each macropixel block generates first information and second information for each macropixel block. The first information indicates the illumination intensity change sensed by a single macropixel block. The second information indicates the illumination intensity change sensed by each macropixel in that single macropixel block. In some embodiments of this disclosure, the first information is generated based on the second information. The first information of a macropixel block is a first value if the first bit of the second information of one or more macropixels in a single macropixel block is a first value. The first information of a macropixel block is a second value if the first bit of the second information of all macropixels in a single macropixel block is a second value. In one example, the first value is 1 and the second value is 0. In another example, the first value is 0 and the second value is 1.

[0096] At box 404, the block control circuit in each macropixel block of each row determines which macropixel block's first information indicates that the change in illumination intensity exceeds a threshold. If the first information of the first macropixel block indicates that the change in illumination intensity exceeds the threshold, the row control circuit of the first row containing the first macropixel block generates third information indicating that the change in illumination intensity exceeds the threshold. In this case, based on the third information, it can be considered that the first information of the first macropixel block in the first row indicates that the change in illumination intensity exceeds the threshold ("yes" at box 404), and at box 406, the row control circuit of the first row sends a scan request signal instructing the row to be scanned to the row scanning circuit 240.

[0097] At block 408, upon receiving a scan request signal, the row scanning circuit 240 transmits the first and second information of each macropixel block in the row corresponding to the scan request signal from the macropixel array to the column data buffer circuit 220. In one example, after receiving the scan request signal, the row scanning circuit 240 may send a response signal to the first row. The response signal is used to open the switch between each macropixel in each macropixel block of the first row and the corresponding column bus. After the switch is opened, the block control circuit in each macropixel block of the first row transmits the first and second information of the corresponding macropixel block to the column data buffer circuit 220 through the corresponding column bus. This completes the process of transmitting the first and second information of each macropixel block of the first row from the macropixel array to the column data buffer circuit 220.

[0098] At box 410, column data cache circuit 220 stores first information and second information. In one example, column data cache circuit 220 can synchronously store the first information and second information of each macro pixel block in the same row according to the column scan control signal CCL. Suppose that within a scan cycle, S rows of macro pixel blocks sense a change in illumination intensity exceeding a threshold, then the first information and second information for those S rows will be stored in column data cache circuit 220 in the order in which those S rows were scanned.

[0099] At block 412, column data selection circuit 230 obtains first information and second information from column data buffer circuit 220, and if it determines that the first information of the first macropixel block indicates that the change in illumination intensity exceeds a threshold, it outputs the second information of each macropixel in the first macropixel block, as well as the position and time information of the first macropixel block. In one example, column data selection circuit 230 can control the timing of obtaining the first and second information for each row from column data buffer circuit 220 by means of readout scan signal X_Greqx. Then, column data selection circuit 230 can determine which macropixel block(s) sensed that the change in illumination intensity exceeded the threshold based on the obtained first information. If it is determined that the first information of the first macropixel block indicates that the change in illumination intensity exceeds the threshold, the second information of each macropixel in the first macropixel block, as well as the position and time information of the first macropixel block, is output. If it is determined that the first information of the first macropixel block indicates that the change in illumination intensity does not exceed the threshold, no information for the first macropixel block is output.

[0100] As described above, an image acquisition device can accommodate both an event sensor and an optical imaging sensor. Embodiments of this disclosure propose an arrangement that saves space in the image acquisition device. In this arrangement, the event sensor and the optical imaging sensor are placed in the same image sensing circuit 200. Figure 5 An exemplary block diagram of an image sensing circuit 200 according to such an embodiment is shown. Figure 5 In the example, Figure 2A or Figure 3 Building upon the example, the image sensing circuit 200 further includes a pixel value output circuit 550. The macropixels among the plurality of macropixels also include a second sensor that converts light intensity into pixel values ​​(in...). Figure 5 (Not shown in the image). The second sensor includes, for example, a grayscale sensor, an RGB sensor, etc. The row scanning circuit 240 is configured to scan each row of macro pixels in the macro pixel block array 210 line by line and provide the pixel values ​​converted by the second sensor in each scanned row to the pixel value output circuit 550. The pixel value output circuit 550 is configured to store and output the pixel values ​​converted by the second sensor in each row.

[0101] exist Figure 5In the example, the pixel value output circuit 550 includes correlated double sampling circuits CDS1, CDS2, ..., CDSj, programmable gain amplifiers PGA1, PGA2, ..., PGAj, analog-to-digital converters ADC1, ADC2, ..., ADCj, and a pixel value buffer circuit. The correlated double sampling circuits CDS1, CDS2, ..., CDSj sample the output signals of the same pixel under both illuminated and unilluminated conditions and use the accumulated charge for subtraction to remove background noise from the pixel. The programmable gain amplifiers PGA1, PGA2, ..., PGAj amplify the output signals of the correlated double sampling circuits CDS1, CDS2, ..., CDSj. The analog-to-digital converters ADC1, ADC2, ..., ADCj convert the amplified signals into digital signals. The pixel value buffer circuit stores the converted digital signals. The converted digital signals correspond to the pixel values ​​of the respective pixels.

[0102] Compared to a scheme where event readout and pixel value scanning are implemented by two image sensing circuits 200, this implementation integrates event readout and pixel value scanning into a single image sensing circuit 200, which can reduce the size of the image sensing circuit 200.

[0103] The following is combined Figures 6 to 14 Introduction in Figure 5 Some exemplary arrangements of macro pixels in the example. Figure 6 An exemplary schematic diagram of a macropixel according to an embodiment of this disclosure is shown. Figure 6 In the example, the first and second sensors in a single macropixel each include a photodiode. The photodiode of the first sensor is denoted as DVS_DS. The pixel circuitry of the first sensor (shown in shaded area) is denoted as DVS_CIR. The pixel circuitry of the second sensor is small and negligible. Figure 6 The example illustrates one first sensor and four second sensors. The photodiodes of the four second sensors are designated APS1, APS2, APS3, and APS4, respectively. Since the first and second sensors do not share photodiodes, the first sensor can perform event detection while the second sensor is exposing itself; therefore, the second sensor does not affect the continuity of event detection by the first sensor. Event information and pixel value information at any given time can be obtained, sensed, and read independently.

[0104] It should be noted that, although in Figure 6 The example shown depicts four second sensors in a single macropixel, but embodiments of this disclosure can support the arrangement of more or fewer second sensors in a single macropixel. For example, P second sensors can be arranged in a single macropixel, where P is a positive integer. Figures 8 to 11 Examples of several possible values ​​for P are shown. Figure 8 In the example, P is 1. Figure 9A and Figure 9B In the example, P is 2. Figure 10 In the example, P is 3. Figure 11 In the example, P is 8.

[0105] In some embodiments of this disclosure, P photodiodes in P second sensors are arranged centered on the photodiode in the first sensor of the macropixel. (See reference...) Figure 6 For example, a second sensor's photodiode is positioned at the top left, bottom left, top right, and bottom right corners of the photodiode in the first sensor. This places the photodiode in the first sensor at the geometric center of the four second sensor photodiodes. This implementation, by arranging the photodiodes of the first and second sensors in a centrally symmetrical manner, helps to proportionally align event information and pixel value information in space. In this context, the top left, bottom left, top right, and bottom right corners of the photodiode refer to the outer left, bottom left, top right, and bottom right regions of the photodiode, respectively, excluding the internal regions of the photodiode.

[0106] Furthermore, in some embodiments of this disclosure, the distance between photodiodes in any two adjacent first sensors among a plurality of macro pixels in the macro pixel block array 210 is equal, and the distance between photodiodes in any two adjacent second sensors among the plurality of macro pixels is equal. Figure 7 A first exemplary arrangement diagram of a plurality of macropixels according to an embodiment of the present disclosure is shown. Figure 7 The example shows 4 Figure 6 The macro pixels shown include macro pixel (1,1), macro pixel (1,2), macro pixel (2,1), and macro pixel (2,2).

[0107] As an example, we can see that the distance from DVS_DS in macro pixel (2,1) to DVS_DS in macro pixel (1,1) is equal to DVS_Pitch. The distance from DVS_DS in macro pixel (2,1) to DVS_DS in macro pixel (2,2) is also equal to DVS_Pitch. Similarly, the distance between any other two adjacent DVS_DS is also equal.

[0108] As an example, it can be seen that the distance from APS1 in macro pixel (1,2) to APS2 in macro pixel (1,1) is equal to APS_Pitch. The distance from APS4 in macro pixel (1,2) to APS2 in macro pixel (2,2) is also equal to APS_Pitch. Similarly, the distance between any two adjacent photodiodes of the other second sensors is also equal. By arranging the photodiodes of the first sensor and the photodiodes of the second sensor at equal intervals, the resolution of the image sensing circuit 200 can be proportionally adjusted.

[0109] Figure 8 A second exemplary arrangement diagram of a plurality of macropixels according to an embodiment of the present disclosure is shown. Figure 8 The example illustrates a single macropixel comprising a photodiode DVS_DS for a first sensor and a photodiode APS for a second sensor. The DVS_DS and APS can be arranged symmetrically along the central axis of the macropixel. This ensures that the distance between any two adjacent photodiodes in the first sensors of the plurality of macropixels is equal, and the distance between any two adjacent photodiodes in the second sensors of the plurality of macropixels is also equal.

[0110] Figure 9A A third exemplary arrangement diagram of a plurality of macro pixels according to an embodiment of the present disclosure is shown. Figure 9A The example shows a single macropixel comprising a first sensor photodiode DVS_DS and two second sensor photodiodes APS1 and APS2. Figure 9A In the example, APS1 is placed at the bottom left corner of DVS_DS, and APS2 is placed at the top right corner of DVS_DS. The distance from APS1 in macro pixel (1,1) to APS1 in macro pixel (1,2) is equal to the distance from APS2 in macro pixel (1,1) to APS2 in macro pixel (1,2). The distance from APS1 in macro pixel (1,1) to APS1 in macro pixel (2,1) is equal to the distance from APS2 in macro pixel (1,1) to APS2 in macro pixel (2,1). The distance from DVS_DS in macro pixel (1,1) to DVS_DS in macro pixel (1,2) is equal to the distance from DVS_DS in macro pixel (1,1) to DVS_DS in macro pixel (2,1).

[0111] As Figure 9A Variations of the example, Figure 9B A fourth exemplary arrangement diagram of a plurality of macropixels according to embodiments of the present disclosure is shown. Figure 9BIn the example, APS1 is placed at the bottom right corner of DVS_DS, and APS2 is placed at the top left corner of DVS_DS. The distance from APS1 in macro pixel (1,1) to APS1 in macro pixel (1,2) is equal to the distance from APS2 in macro pixel (1,1) to APS2 in macro pixel (1,2). The distance from APS1 in macro pixel (1,1) to APS1 in macro pixel (2,1) is equal to the distance from APS2 in macro pixel (1,1) to APS2 in macro pixel (2,1). The distance from DVS_DS in macro pixel (1,1) to DVS_DS in macro pixel (1,2) is equal to the distance from DVS_DS in macro pixel (1,1) to DVS_DS in macro pixel (2,1).

[0112] Figure 10 A fifth exemplary arrangement diagram of a plurality of macropixels according to embodiments of the present disclosure is shown. Figure 10 In the example, APS1 is positioned directly above DVS_DS, APS2 is positioned at the lower left corner of DVS_DS, and APS3 is positioned at the lower right corner of DVS_DS. The distance from APS1 in macro pixel (1,1) to APS2 in macro pixel (1,2) is equal to the distance from APS3 in macro pixel (1,1) to APS1 in macro pixel (1,2). The distance from DVS_DS in macro pixel (1,1) to DVS_DS in macro pixel (1,2) is equal to the distance from DVS_DS in macro pixel (1,1) to DVS_DS in macro pixel (2,1). To align DVS_DS, the boundaries of macro pixels can be misaligned. For example... Figure 10 As shown, the top edge of macro pixel (1,1) and the top edge of macro pixel (1,2) may not be aligned, but the DVS_DS in macro pixel (1,1) and the DVS_DS in macro pixel (1,2) are aligned.

[0113] Figure 11 A sixth exemplary arrangement diagram of a plurality of macropixels according to an embodiment of the present disclosure is shown. A second sensor's photodiodes APS1-APS8 are arranged at the upper left, upper right, upper right, left, right, lower left, lower right, and lower right corners of the photodiode DVS_DS in the first sensor, respectively. Thus, the photodiode in the first sensor is located at the geometric center of the eight second sensor photodiodes. The distance from APS3 in macropixel (1,1) to APS1 in macropixel (1,2) is equal to the distance from APS8 in macropixel (1,1) to APS3 in macropixel (2,1). The distance from DVS_DS in macropixel (1,1) to DVS_DS in macropixel (1,2) is equal to the distance from DVS_DS in macropixel (1,1) to DVS_DS in macropixel (2,1).

[0114] exist Figures 6 to 11In the example, the photodiodes in the first and second sensors are shown as rectangular. Setting the photodiodes as rectangular facilitates the overall layout and planning of the image sensing circuit 200. Alternatively, the photodiodes in the first and second sensors can be set in other shapes. Figure 12 An example is shown where the photodiodes in the first and second sensors are regular polygons. Setting the photodiodes as regular polygons allows for easier carrier flow by utilizing the obtuse angles of the polygon's boundaries. This is beneficial for the flow of photoelectric charge. Figure 12 In the example, the distance from APS4 in macro pixel (1,1) to APS3 in macro pixel (1,2) is equal to the distance from APS3 in macro pixel (1,1) to APS1 in macro pixel (2,1). The distance from DVS_DS in macro pixel (1,1) to DVS_DS in macro pixel (1,2) is equal to the distance from DVS_DS in macro pixel (1,1) to DVS_DS in macro pixel (2,1).

[0115] In some embodiments of this disclosure, where a capacitor is present in the pixel circuit DVS_CIR of the first sensor, if the capacitor used in the pixel circuit DVS_CIR needs to span the position of the photodiode DVS_DS in the first sensor, then the photodiode DVS_DS needs to make way for the capacitor. This is because front exposure requires the photodiode DVS_DS to be completely exposed outside other metal, so the photodiode DVS_DS is not vertically compatible with either the MIM capacitor or the MOM capacitor composed of the metal above it. Therefore, these embodiments of this disclosure propose dividing the photodiode DVS_DS in the first sensor within a single macropixel into separate portions. Figure 13 An arrangement diagram of such an embodiment is shown. The photodiode DVS_DS can be divided into multiple parts, including a first part DVS_DS_1 and a second part DVS_DS_2. The geometric center of these multiple parts is determined as the center point of the photodiode DVS_DS. (Reference) Figure 13For example, the geometric centers of DVS_DS_1 and DVS_DS_2 are determined as the center points of DVS_DS. For instance, the geometric center C11 of DVS_DS_1 and DVS_DS_2 in macropixel (1,1) is determined as the center point of DVS_DS in macropixel (1,1). The geometric center C12 of DVS_DS_1 and DVS_DS_2 in macropixel (1,2) is determined as the center point of DVS_DS in macropixel (1,2). The geometric center C21 of DVS_DS_1 and DVS_DS_2 in macropixel (2,1) is determined as the center point of DVS_DS in macropixel (2,2). The geometric center C22 of DVS_DS_1 and DVS_DS_2 in macropixel (2,2) is determined as the center point of DVS_DS in macropixel (2,2). The distance between the center points of any two adjacent photodiodes in the first sensor among these macropixels is equal. The distance from the center point of DVS_DS in macro pixel (1,1) to the center point of DVS_DS in macro pixel (1,2) is equal to DVS_Pitch. The distance from the center point of DVS_DS in macro pixel (1,1) to the center point of DVS_DS in macro pixel (2,1) is equal to DVS_Pitch. Similarly, the distance between the center points of any other two adjacent DVS_DS is also equal.

[0116] exist Figures 6 to 13 In the example, the pixel circuitry in the first sensor within a single macropixel is shown arranged on the same layer as the photodiode of the first sensor. Alternatively, the pixel circuitry in the first sensor within a single macropixel can be arranged in a three-dimensional stack below the photodiodes of the first and second sensors. Figure 14 An exemplary schematic diagram of a macropixel of this embodiment is shown. The photodiode DVS_DS in the first sensor and the photodiodes APS1, APS2, APS3, and APS4 of the four second sensors are arranged in the first layer L1. The pixel circuit DVS_CIR in the first sensor is arranged in the second layer L2. It is evident that the embodiments of this disclosure are applicable not only to two-dimensional chips but also to three-dimensional chips, demonstrating a wide range of applications and being unrestricted by chip package type.

[0117] Figure 15 A comparison diagram is shown between the amount of data that an image sensing circuit 200 according to an embodiment of this disclosure needs to process and another image sensing circuit. This other image sensing circuit processes event data in single-row macropixel groups (single-row groups). Figure 15The example uses datasets with different sparsities (e.g., 0.27, 0.13, 0.05) (where sparsity is simply defined as the ratio of the number of pixels with events to the total number of pixels) to calculate event data for all frames. The data volume results are illustrated using the median and mean processed per frame. Figure 15 The diagram illustrates the block segmentation method using a 2×k macro-pixel block as an example, with a single-row grouping of 1×k. The AER results are shown by processing 512 blocks under a single-row grouping. Figure 15 As can be seen, in the block-based approach, for scenarios with relatively dense event information, the block-based reading method effectively reduces the data volume. Even when the data is very sparse, the block-based reading method effectively reduces the data volume. Secondly, the embodiments of this disclosure can achieve spatially uniform simultaneous output of event information and grayscale information, achieving spatial completeness and continuity of information. For a sparsity of 0.13, events are concentrated in the lower half of the chart. For a sparsity of 0.05, events are randomly distributed throughout the chart. The data volume reduction achieved by event block reading is significantly better in scenarios where events are more concentrated than in scenarios where events are more dispersed. This indicates that the improvement effect of the embodiments of this disclosure is more significant in scenarios with more concentrated events.

[0118] In summary, in the image sensing circuit according to embodiments of this disclosure, changes in illumination intensity are characterized on a macropixel block basis. Only event information for macropixel blocks where an event occurs (a change in illumination intensity exceeding a threshold) is output, thus reducing the amount of data output for event information. If the first information for a macropixel block indicates that the change in illumination intensity does not exceed a threshold, then there is no need to output second information for each macropixel within that macropixel block. Compared to a scheme where event readout and pixel value scanning are implemented by two separate image sensing circuits, this implementation integrates event readout and pixel value scanning into a single image sensing circuit, reducing the size of the image sensing circuit. Achieving a centrally symmetrical and equidistant arrangement of event pixels and grayscale pixels allows the resolution of the image sensing circuit to be proportional, facilitating proportional spatial alignment of event information and pixel value information.

[0119] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0120] A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium can be, for example—but not limited to—an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), and any suitable combination thereof. The computer-readable storage medium as used herein is not to be construed as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0121] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0122] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0123] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0124] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0125] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0127] Unless otherwise expressly indicated by the context, the singular form of words used herein and in the appended claims includes the plural form, and vice versa. Thus, when referring to the singular, the plural form of the corresponding term is generally included. Where the term “example” is used herein, particularly when it follows a set of terms, the “example” is merely exemplary and illustrative and should not be considered exclusive or pervasive.

[0128] Further aspects and scope of adaptation become apparent from the description provided herein. It should be understood that various aspects of this application may be implemented individually or in combination with at least one other aspect. It should also be understood that the descriptions and specific embodiments herein are for illustrative purposes only and are not intended to limit the scope of this application.

[0129] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An image sensing circuit, comprising: A macropixel block array includes multiple macropixel blocks arranged in an array, wherein each macropixel block includes multiple macropixels arranged in an array, and each macropixel includes a first sensor for sensing changes in light intensity. Column data cache circuit; The row scanning circuit is configured such that when the first sensor in the first row of macropixel blocks senses a change in light intensity exceeding a threshold, first information and second information of each macropixel block in the first row are transmitted from the macropixel array to the column data buffer circuit. The first information indicates the change in light intensity sensed by a single macropixel block, and the second information indicates the change in light intensity sensed by each macropixel in the single macropixel block. as well as The column data selection circuit is configured to acquire the first information and the second information stored by the column data cache circuit, and, if it is determined that the change in illumination intensity of the first information of the first macro pixel block exceeds a threshold, output the second information of each macro pixel in the first macro pixel block, as well as the position information and time information of the first macro pixel block.

2. The image sensing circuit according to claim 1 further includes a plurality of row control circuits, wherein the row control circuit of the plurality of row control circuits is configured to determine third information for the row based on the first information of each macro pixel block of the corresponding row, and to send a scan request signal indicating scanning the row to the row scanning circuit when the third information indicates that the change in illumination intensity exceeds a threshold, wherein the third information indicates that the change in illumination intensity exceeds a threshold when the first information of at least one macro pixel block in the row indicates that the change in illumination intensity exceeds a threshold; The row scanning circuit is further configured to, upon receiving a scan request signal from the row control circuit corresponding to the first row, determine that the first sensor in the macro-pixel block of the first row senses a change in light intensity exceeding a threshold.

3. The image sensing circuit according to claim 1 or 2 further includes a pixel value output circuit; The macro pixels among the plurality of macro pixels also include a second sensor that converts light intensity into pixel values; The row scanning circuit is further configured to scan each row of macro pixels in the macro pixel block array line by line and provide the pixel values ​​converted by each second sensor in the scanned row to the pixel value output circuit. The pixel value output circuit is configured to store and output the pixel values ​​converted by each of the second sensors in each row.

4. The image sensing circuit of claim 3, wherein the first sensor and the second sensor in a single macropixel include respective photodiodes.

5. The image sensing circuit according to claim 4, wherein the distance between any two adjacent photodiodes in the first sensor among the plurality of macro pixels is equal, and the distance between any two adjacent photodiodes in the second sensor among the plurality of macro pixels is equal.

6. The image sensing circuit according to claim 5, wherein a single macropixel includes P second sensors, where P is a positive integer, and P photodiodes of the P second sensors are arranged with the photodiodes of the first sensors in the macropixel as the center.

7. The image sensing circuit according to claim 4, wherein the shape of the photodiode in the first sensor and the second sensor comprises: Rectangles or regular polygons.

8. The image sensing circuit according to claim 4, wherein the pixel circuit of the first sensor in a single macropixel is arranged on the same layer as the photodiode of the first sensor, or the pixel circuit of the first sensor in a single macropixel is arranged on the lower layer of the photodiodes of the first sensor and the second sensor in a three-dimensional stacking manner.

9. The image sensing circuit of claim 4, wherein the photodiode in the first sensor of a single macropixel comprises a plurality of separate portions, the geometric center of the plurality of portions of the photodiode is determined as the center point of the photodiode, and the distance between the center points of any two adjacent photodiodes in the first sensor of the plurality of macropixels is equal.

10. A chip, comprising: Image sensing circuit according to any one of claims 1 to 9; as well as An encapsulation structure surrounds the image sensing circuit.

11. An electronic device, comprising: Image sensing circuit according to any one of claims 1 to 9; as well as A power supply circuit is configured to supply power to the image sensing circuit.

12. A method for an image sensing circuit, the image sensing circuit comprising: Macro-pixel block array, row scanning circuit, column data buffer circuit, and column data selection circuit; The macropixel block array includes multiple macropixel blocks arranged in an array, and each macropixel block includes multiple macropixels arranged in an array. Each macropixel includes a first sensor for sensing changes in light intensity. The method includes: Generate first information and second information for each macropixel block, wherein the first information indicates the light intensity change sensed by a single macropixel block, and the second information indicates the light intensity change sensed by each macropixel in the single macropixel block; In response to the first information indicating that the change in illumination intensity of the macro-pixel block in the first row exceeds a threshold, a scan request signal indicating that the row should be scanned is sent to the row scanning circuit. When the row scanning circuit receives the scan request signal, it causes the first information and the second information of each macro pixel block of the row corresponding to the scan request signal to be transmitted from the macro pixel array to the column data cache circuit. The column data cache circuit stores the first information and the second information; and The column data selection circuit obtains the first information and the second information from the column data cache circuit, and when it determines that the change in illumination intensity of the first information of the first macro pixel block exceeds a threshold, it outputs the second information of each macro pixel in the first macro pixel block, as well as the position information and time information of the macro pixel block.

Images