Method for correcting dynamic vision sensor events and image sensor for performing the same

Abstract

A method for correcting dynamic vision sensor (DVS) events is described. The method generates event data including events representing motion information of an object included in an image. In addition, the method generates image data capturing the image, and generates edge data representing edge information of the image based on the image data. The method also generates omitted events for the event data based on the edge data. By supplementing omitted events for event data provided from the DVS with edge information, the accuracy of the event data and the performance of machine vision devices and systems operating based on the event data are enhanced.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the priority of Korean Patent Application No. 10-2019-0081962 filed on July 8, 2019, in the Korean Intellectual Property Office (KIPO), the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] Example embodiments relate generally to semiconductor integrated circuits, and more particularly, to a method of correcting an event of a dynamic vision sensor (DVS) and an image sensor performing the method. Background Art

[0004] Image sensors detect visual information to create images. Numerous applications use image sensors, such as smartphones, tablets, and cameras. Frame-based image sensors are used for machine tasks, such as object recognition. Because they use static image sequencing to create motion, frame-based image sensors often create redundant and unusable data for motion imaging tasks.

[0005] Dynamic Vision Sensors (DVS) reduce data usage by preprocessing visual information from multiple pixels rather than capturing multiple static images. DVS provides the imaging system with pixel-level changes in light intensity, called event data.

[0006] DVS provides event data, such as the horizontal and vertical coordinates of the event on the image plane, the polarity of the event, and the timestamp of the event. Sometimes, event data is omitted from the event data set, which may cause malfunction of the imaging system. Therefore, there is a need in the art for a method of effectively capturing motion data while compensating for omitted events. Summary of the invention

[0007] Some example embodiments may provide a method of correcting events to enhance the accuracy of event data provided from a dynamic vision sensor (DVS).

[0008] Some example embodiments may provide an image sensor performing a method of correcting an event of DVS.

[0009] According to an example embodiment, a method for correcting a DVS event includes: generating event data using DVS pixels, the event data including an event representing motion information of an object included in an image; generating image data of a captured image using image pixels; generating edge data representing edge information of the image based on the image data; and generating an omitted event of the event data based on the edge data.

[0010] According to an example embodiment, an image sensor includes a DVS pixel, an image pixel, and a DVS event compensation circuit. The DVS pixel generates event data including events representing motion information of an object included in an image. The image pixel generates image data capturing an image. The DVS event compensation circuit generates edge data representing edge information of the image based on the image data, and generates an omitted event of the event data based on the edge data.

[0011] According to an example embodiment, a method for correcting a DVS event includes: sensing event data representing a change of an image; sensing image data of the image; generating edge data representing edge information of the image based on the image data, wherein each pixel of the edge data corresponds to a pixel of the event data; obtaining a line sum value by summing values ​​of the edge data multiplied by a value of the event data in a pixel direction relative to a target pixel; comparing the line sum value with a reference value; and generating an omission event based on the comparison.

[0012] According to example embodiments, the described systems and methods may enhance the accuracy of event data and the performance of machine vision devices and systems operating based on the event data by supplementing omitted events of event data provided from a DVS using edge information. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Example embodiments of the present disclosure will be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings.

[0014] Figure 1 is a flow chart illustrating a method of correcting a DVS event according to an example embodiment.

[0015] Figure 2 is a block diagram illustrating an image processing system according to example embodiments.

[0016] Figure 3 is a block diagram illustrating an image sensor according to example embodiments.

[0017] Figure 4 is a block diagram illustrating a DVS event compensation circuit according to an example embodiment.

[0018] Figure 5 is used to describe the Figure 4 FIG. 1 is a diagram of the operation of the brightness converter in the DVS event compensation circuit.

[0019] Fig. 6A and Figure 6B is used to describe the Figure 4 A diagram of the operation of the filter in the DVS event compensation circuit.

[0020] Figure 7is a flow chart illustrating a method of generating an omission event according to an example embodiment.

[0021] Figure 8 It is shown Figure 7 Schematic diagram of an example embodiment of a method.

[0022] Fig. 9 is a diagram illustrating a pixel array included in an image sensor according to example embodiments.

[0023] FIG. 10A to FIG. 10D is a circuit diagram illustrating an example embodiment of an image pixel included in a pixel array.

[0024] Fig.11 is a circuit diagram illustrating an example embodiment of a DVS pixel included in a pixel array.

[0025] Fig.12 It is shown Fig.11 A diagram of the operation of a DVS pixel.

[0026] Fig.13 is a diagram illustrating an example embodiment of a layout of a pixel array included in an image sensor according to example embodiments.

[0027] Fig.14 It is shown that it has the Fig.13 Schematic diagram of the layout of a stacked structure pixel array.

[0028] Fig.15 It is shown that Fig.14 Schematic diagram of an example embodiment of an image pixel implemented in a pixel array of a stacked structure.

[0029] Fig.16 It is shown that Fig.14 Schematic diagram of an example embodiment of a DVS pixel implemented in a pixel array of a stacked structure.

[0030] Fig.17 It is shown that Fig.14 Schematic diagram of an example embodiment of an image pixel implemented in a pixel array of a stacked structure.

[0031] Fig.18 It shows that Fig.14 Schematic diagram of an example embodiment of a pixel array with a stacked structure.

[0032] Fig.19 is a block diagram illustrating an image sensor according to example embodiments.

[0033] Fig. 20 and Fig.21is a diagram illustrating an example embodiment of a layout of a pixel array included in an image sensor according to example embodiments.

[0034] Fig. 22 is a block diagram illustrating an example embodiment of a DVS event compensation apparatus included in an image sensor according to example embodiments.

[0035] Fig.23 is used to describe the Fig. 22 A diagram of the operation of the synchronizer in the DVS event compensation circuit.

[0036] Fig.24 is a block diagram illustrating an example embodiment of a DVS event compensation apparatus included in an image sensor according to example embodiments.

[0037] Fig.25 and Fig.26 is used to describe the Fig.24 A diagram of the operation of the mode controller in the DVS event compensation circuit.

[0038] Fig. 27 is a block diagram illustrating a computing system including an image sensor according to example embodiments. DETAILED DESCRIPTION

[0039] The present disclosure relates to a dynamic vision sensor (DVS) and a method for correcting DVS events by supplementing omitted events of event data provided from the DVS. The accuracy of event data can be enhanced by supplementing event data with omitted events, which can improve the performance of devices and systems that operate based on enhanced event data. Embodiments of the present disclosure can be applied to improve the performance of various systems including video capture devices and simultaneous localization and mapping (SLAM) systems.

[0040] Embodiments of the present disclosure generate edge data representing edge information of an image based on image data, and generate omitted events based on the edge data. In some cases, the event data is adjusted to ensure that each pixel of the edge data corresponds to a pixel of the event data. Embodiments of the present disclosure synchronize the edge data and the event data spatially, temporally, or both. In some cases, DVS event compensation may be selectively performed.

[0041] Various example embodiments will be described more fully below with reference to the accompanying drawings in which some example embodiments are shown. In the accompanying drawings, the same numerals refer to the same elements from beginning to end. Repeated descriptions may be omitted.

[0042] Figure 1is a flow chart illustrating a method of correcting a DVS event according to an example embodiment.

[0043] refer to Figure 1 , using DVS pixels to generate event data, the event data including events representing motion information of an object included in an image (S100). Fig.11 and Fig.12 Describes the use of DVS pixels to generate event data.

[0044] Image data of the captured image is generated using the image pixels (S200). In some example embodiments, image data (e.g., color pixel values) may be generated using image pixels including color pixels corresponding to different colors. FIG. 10A to FIG. 10D Describes the generation of image data using image pixels.

[0045] Edge data indicating edge information of an image is generated from the image data (S300). An omitted event of event data is generated from the edge data (S400).

[0046] Each pixel of the edge data may correspond to a pixel of the event data. For a one-to-one correspondence, resolution conversion, time synchronization and / or spatial synchronization may be performed between the edge data and the event data.

[0047] When a pixel of edge data corresponds to a pixel of event data, the determination as to whether to generate an omission event may be based on a line sum value that can be obtained by summing the multiplied values ​​of edge data and event data in units of pixels, as will be described below with reference to Figure 7 and Figure 8 Described.

[0048] Thus, according to example embodiments, the method of correcting the events of the DVS can enhance the accuracy of the event data and the performance of the machine vision device and system operating based on the event data. Correcting the events of the DVS can be performed by supplementing or compensating for the omitted events of the event data provided from the DVS using the edge information of the object of the image.

[0049] Figure 2 is a block diagram illustrating an image processing system according to example embodiments.

[0050] refer to Figure 2 , the image processing system 10 may include an image sensor (SEN) 100, an image signal processor (ISP) 200, a processor 300, and a peripheral circuit (PERI) 400. The image processing system 10 may be implemented as a system-on-chip (SoC).

[0051] The image sensor 100 may perform the functions of an image sensor to generate image data of a captured image using image pixels. In addition, the image sensor 100 may also perform the functions of a dynamic vision sensor (DVS). The combination of the image sensor and DVS functions of the image sensor 100 may use DVS pixels to provide event data, including events representing motion information of an object included in the image.

[0052] In some example embodiments, each DVS pixel may include a signal storage element (e.g., a capacitor). The DVS pixel may detect a voltage change of the signal storage element to generate event data including an event. In some example embodiments, the image pixels may be color pixels corresponding to different colors, and the image data may be color image data.

[0053] The image sensor 100 may provide sensing data SNDT to the image signal processor 200 in a digital form. The sensing data SNDT may include color image data output from the image data and / or event data output from the DVS pixel. The overall operation of the image sensor 100 may be controlled by a control signal CTRL provided from the processor 300. The processor 300 may be a general-purpose processor such as a central processing unit (CPU).

[0054] According to example embodiments, the image sensor 100 may further include at least one depth sensor to provide depth image data in addition to the color image data. The image sensor 100 may be implemented as a separate chip. The image sensor 100 may be a complementary metal oxide semiconductor (CMOS) image sensor.

[0055] The image sensor 100 may include a DVS event compensation circuit (DVSEC) 500 according to an example embodiment. The DVS event compensation circuit 500 may generate edge data representing edge information of an image based on image data, and generate omitted events of event data based on the edge data. The DVS event compensation circuit 500 may provide compensated event data by combining events included in the original event data and omitted events generated in the DVS event compensation circuit 500.

[0056] The image signal processor 200 may receive sensing data SNDT including image data and compensated event data, and process the sensing data SNDT to provide processed data PCDT. For example, the image signal processor 200 may process the sensing data SNDT frame by frame. The sensing data SNDT may be processed frame by frame.

[0057] The image signal processor 200 may include a frame memory to store the sensing data SNDT and / or the processed data PCDT. The image signal processor 200 may also adjust the brightness, contrast, and color saturation of the image data included in the sensing data SNDT. Figure 2 In the embodiment shown in FIG. 1 , the image signal processor 200 is implemented outside the image sensor 100 , but example embodiments are not limited. According to example embodiments, the image signal processor 200 may be implemented inside the image sensor 100 .

[0058] The processor 300 may receive the processed data PCDT from the image signal processor 200 and perform various functions by comparing the processed data PCDT with an image code or a motion code stored in advance. For example, the image code may be a two-dimensional image, a three-dimensional (3D) image, and a motion code. The two-dimensional image may be a fingerprint or face of a person. The three-dimensional (3D) image may include depth or spatial information. The motion code may be a reference motion of an image corresponding to a continuous motion such as a gesture.

[0059] In some example embodiments, the processor 300 may perform a machine vision function based on the processed data PCDT. For example, when the motion code is a motion image of a finger drawing a circle, the processor 300 may determine whether the processed data PCDT corresponds to the motion image of the finger drawing a circle. The processor 300 may then perform a machine vision function based on the determined result. The processor 300 may activate a system connected to the image processing system 10, such as a power system, an audio system, a specific application, etc., based on the comparison result of the processed data PCDT and the stored motion code.

[0060] The peripheral circuit 400 may provide a signal or data to the processor 300. The signal or data is generated based on the system state and / or various inputs. The peripheral circuit 400 may be implemented as an input / output (I / O) interface. In this case, the peripheral circuit 400 may send a signal generated according to the user's input to the processor 300. The I / O interface may be any type of I / O device, such as an external input button, a touch screen, or a mouse. The peripheral circuit 400 may include a display device or an interface to be connected to an external display device. The display device may display an image based on the processed image data PCDT.

[0061] Figure 3 is a block diagram illustrating an image sensor according to example embodiments.

[0062] refer to Figure 3, the image sensor 100 may include a pixel array 110 , a control logic (CLOG) (or a control circuit) 120 , a row driver (RDEC) 130 , a pixel signal processing circuit 150 , and a DVS event compensation circuit (DVSEC) 500 .

[0063] The pixel array 110 includes a plurality of color pixels C that obtain color image data of an object and a plurality of DVS pixels M that sense motion of the object. The pixel array 110 may further include a color filter array (not shown) including a plurality of color filter layers (not shown) that transmit light having a predetermined wavelength.

[0064] The pixel array 110 may further include a depth sensor pixel (not shown) for obtaining depth information of an object. When the pixel signal of the depth sensor pixel is processed using a time-of-flight (TOF) scheme, the image sensor 100 may further include an infrared pass filter (not shown). The infrared pass filter filters light other than infrared light emitted by an infrared light source (not shown) controlled by the control logic 120 and infrared light reflected from an object after being emitted by the infrared light source.

[0065] The control logic 120 may be based on Figure 2 The control logic 120 may control the row driver 130 and the pixel signal processing circuit 150 by using the control signal CTRL of the image signal processor 200 in the image sensor 100.

[0066] The row driver 130 may enable the color pixels C and the DVS pixels M row by row according to the control of the control logic 120 .

[0067] The pixel signal processing circuit 150 may process the pixel signal from the pixel array 110 to output the sensing data SNDT. The sensing data SNDT may include the image data IMDT and the event data EVDT. The pixel signal processing circuit 150 may include a readout circuit (ROC) 154, a column AER (Address Event Representation) AER_C 156, and a row AER AER_R 158.

[0068] Each of the DVS pixels M included in the pixel array 110 may output an event signal according to a change in the amount of light. Fig.11 and Fig.12 Description of the event signal The column AER 156 may receive the event signal and output a column address value of each DVS pixel M that has generated the event signal based on the event signal.

[0069] The readout circuit 154 may receive a pixel signal output from each of the color pixels C included in the pixel array 110, and process the pixel signal to output the image data IMDT. According to example embodiments, the readout circuit 154 may include a column decoder (not shown), a column driver (not shown), a Correlated Double Sampling (CDS) block (not shown), an Analog-to-Digital Converter (ADC) block (not shown), an output buffer (not shown), and the like.

[0070] The row AER 158 may receive the event signal output from each DVS pixel M and output a row address value of each DVS pixel M that has generated the event signal. Outputting the row address value of each DVS pixel M may be based on the event signal. The row address value may be sent to the column AER 156. The column AER 156 may generate event data EVDT including an event.

[0071] The DVS event compensation circuit 500 can generate edge data representing edge information of an image based on the image data IMDT, and generate an omitted event of event data based on the edge data. The DVS event compensation circuit 500 can provide compensated event data EVDT' by combining an event included in the original event data EVDT and an omitted event generated in the DVS event compensation circuit 500.

[0072] Thus, according to example embodiments, the method of correcting events of a DVS and the image sensor can enhance the accuracy of event data and the performance of machine vision devices and systems operating based on event data. Correcting events of a DVS can be performed by supplementing omitted events of event data provided from a DVS using edge information of an object of an image.

[0073] Figure 4 is a block diagram illustrating a DVS event compensation circuit according to an example embodiment, Figure 5 is used to describe the Figure 4 A diagram of the operation of the DVS event compensation circuit in the brightness converter, and Fig. 6A and Figure 6B is used to describe the Figure 4 A diagram of the operation of the filter in the DVS event compensation circuit.

[0074] refer to Figure 4 , the DVS event compensation circuit 500 may include a brightness converter (LCNV) 510 , a filter (FLT) 520 , a scaler (SCL) 530 , an event generator (EVGEN) 540 , and an event combiner (EVCMB) 550 .

[0075] The brightness converter 510 may generate brightness data YDT based on the image data IMDT. Figure 5 As shown, the first image data IMDT1 may correspond to a pattern of a repeating unit pixel group including two green pixels G, one red pixel R, and one blue pixel B, and the second image data IMDT2 may correspond to a pattern of a repeating unit pixel group including one green pixel G, one red pixel R, one blue pixel B, and one white pixel W. The pattern of arranging image pixels is not limited to Figure 5 Those patterns.

[0076] For example, the first image data IMDT1 corresponding to four pixel values ​​of four image pixels may be represented by one pixel value Y of the brightness data YDT as in Equation 1 (Eq. 1).

[0077] Y=Gr·R+2·Gg·G+Gb·B (1)

[0078] In Equation 1, variables Gr, Gg, and Gb indicate gains of color pixels, and variables R, G, and B indicate pixel values ​​of color pixels.

[0079] In the same manner, the second image data IMDT2 corresponding to four pixel values ​​of four image pixels may be represented by one pixel value Y of the brightness data YDT as in Equation 2 (Eq. 2).

[0080] Y=Gr·R+Gg·G+Gb·B+Gw·W (2)

[0081] In Equation 2, variables Gr, Gg, Gb, and Gw indicate gains of color pixels, and variables R, G, B, and W indicate pixel values ​​of color pixels.

[0082] The filter 520 may generate edge data EGDT based on the brightness data YDT. The filter 520 may be implemented with an edge detector such as a high-pass filter (HPF), wherein the edge detector is configured to extract edge information from the brightness data YDT. The edge detector may generate edge data EGDT indicating the edge of an image in the brightness data YDT using a scheme such as Canny edge detection. Fig. 6A An example image of image data is shown, and Figure 6B Shown with Fig. 6A The portion of the image RT corresponding to the edge data is the edge image.

[0083] The scaler 530 may spatially synchronize the edge data EGDT and the event data EVDT so that pixels of the edge data EGDT and pixels of the event data EVDT correspond to each other. For example, the scaler 530 may adjust the resolution of the edge data EGDT so that the edge data EGDT′ has the same resolution as that of the event data EVDT.

[0084] In some example embodiments, event data EVDT of a first resolution may be generated using DVS pixels arranged at a first resolution, and image data IMDT of a second resolution may be generated using image pixels arranged at a second resolution higher than the first resolution. In this case, the brightness converter 510 and the filter 520 may generate raw edge data EGDT of a second resolution indicating edge information of an image based on the image data IMDT of the second resolution, and the scaler 530 may convert the raw edge data EGDT of the second resolution into edge data EGDT' of the first resolution. If the image data IMDT and the event data EVDT have the same resolution, the scaler 530 may be omitted.

[0085] The event generator 540 may generate an omission event OMEV based on the edge data EGDT'. Figure 7 and Figure 8 An example embodiment of generating an omission event OMEV is described.

[0086] The event combiner 550 may generate compensated event data EVDT′ by combining events included in the original event data EVDT and the omission event OMEV generated by the event generator 540 .

[0087] Figure 7 is a flow chart illustrating a method of generating an omission event according to an example embodiment.

[0088] Refer to Figure 6 and Figure 7 , the event generator 540 may initialize the address or position of the target pixel (S10). The pixel address may be represented by (i, j), where i is the row number of the pixel and j is the column number of the pixel. After spatially synchronizing the edge data EGDT' and the event data EVDT, the target pixel may indicate a DVS pixel corresponding to a possible omission event, so that the pixel of the edge data EGDT' and the pixel of the event data EVDT may correspond to each other (e.g., have the same resolution).

[0089] The event generator 540 may set a window WIN(i, j) centered on the target pixel (S11). The window WIN(i, j) may include the target pixel and the neighboring pixels of the target pixel. For example, the window WIN(i, j) may have a size of 3*3, such as Figure 8 As shown, the size can be 5*5, 7*7, etc.

[0090] With respect to the target pixel and the adjacent pixels in the window WIN(i, j), the event generator 540 may calculate at least one line sum value LS by summing the multiplied values ​​of the edge data EGDT′ and the event data EVDT in units of pixels in one direction ( S12 ).

[0091] The event generator 540 may compare the line sum value LS with the reference value THS (S13), and generate an omission event based on the result of the comparison. For example, when the line sum value LS is greater than the reference value THS (S13: Yes), the event generator 540 may generate an omission event OMEV (i, j) corresponding to the target pixel (S14).

[0092] When the line sum value LS is less than the reference value THS (S13: No), the event generator 540 may not generate the omission event OMEV (i, j) corresponding to the target pixel. The event generator 540 may repeat the above process (S11, S12, S13, S14) by changing the address (i, j) of the target pixel (S16) until the process is completed for the last pixel among the DVS pixels.

[0093] In this way, when the target pixel is sequentially changed from among the DVS pixels (S16), generating a line sum value (S12), comparing the line sum value with a reference value (S13), and generating an omission event based on the comparison result (S14) may be repeated.

[0094] Therefore, according to an example embodiment, the DVS pixel senses event data EVDT representing a change of an image; the image pixel senses image data IMDT of the image; the filter 520 can generate edge data EGDT representing edge information of the image based on the image data; the calibrator 530 can generate modified edge data EGDT', wherein each pixel of the modified edge data EGDT' corresponds to a pixel of the event data; and the event generator 540 can obtain a line sum value by summing the values ​​of the edge data multiplied by the value of the event data in a pixel direction relative to the target pixel, compare the line sum value with a reference value, and generate an omission event based on the comparison.

[0095] Figure 8 It is shown Figure 7 Schematic diagram of an example embodiment of a method.

[0096] For ease of illustration and description, Figure 8 An example of a window of 3*3 size is shown, and the size of the window can be changed variously. Figure 8 In , as an example, the address of the target pixel is (1, 1).

[0097] refer to Figure 8 , regarding the target pixel and the adjacent pixels in the window, the horizontal line sum value LSX, the vertical line sum value LSY, the first diagonal line sum value LSD1 and the second diagonal line sum value LSD2 can be calculated by summing the multiplied values ​​of the edge data EGDT and the event data EVDT in units of pixels in each direction.

[0098] Regarding the target pixel (1, 1) and the adjacent pixels (1, 0) and (1, 2) adjacent to the target pixel (1, 1) in the horizontal direction DX, the horizontal line sum value LSX can be obtained by summing the multiplication values ​​Y10×E10, Y11×E11 and Y12×E12 of the edge data EGDT and the event data EVDT in units of pixels in the horizontal direction DX.

[0099] Regarding the target pixel (1, 1) and the adjacent pixels (0, 1) and (2, 1) adjacent to the target pixel (1, 1) in the vertical direction DY, the vertical line sum value LSY can be obtained by summing the multiplication values ​​Y01×E01, Y11×E11 and Y21×E21 of the edge data EGDT and the event data EVDT in units of pixels in the vertical direction DY.

[0100] Regarding the target pixel (1, 1) and the adjacent pixels (2, 0) and (0, 2) adjacent to the target pixel (1, 1) in the first diagonal direction DD1, the first diagonal sum value LSD1 can be obtained by summing the multiplication values ​​Y20×E20, Y11×E11 and Y02×E02 of the edge data EGDT and the event data EVDT in units of pixels in the first diagonal direction DD1.

[0101] Regarding the target pixel (1, 1) and the adjacent pixels (0, 0) and (2, 2) adjacent to the target pixel (1, 1) in the second diagonal direction DD2, the second diagonal sum value LSD2 can be obtained by summing the multiplication values ​​Y00×E00, Y11×E11 and Y22×E22 of the edge data EGDT and the event data EVDT in units of pixels in the second diagonal direction DD2.

[0102] in this case, Figure 7 The comparison between the center line sum value LS and the reference value THS ( S13 ) may include comparing each of the horizontal line sum value LSX, the vertical line sum value LSY, the first diagonal line sum value LSD1 , and the second diagonal line sum value LSD2 with the first reference value TH1 .

[0103] In some example embodiments, when at least one of the horizontal line sum value LSX, the vertical line sum value LSY, the first diagonal line sum value LSD1, and the second diagonal line sum value LSD2 is greater than the first reference value TH1, the event generator 540 may generate an omission event OMEV(1,1) corresponding to the target pixel (1,1).

[0104] In some example embodiments, when the count value is greater than the second reference value TH2, the event generator 540 may generate an omission event OMEV(1, 1) corresponding to the target pixel (1, 1), where the count value indicates the number of values ​​greater than the first reference value TH1 from the horizontal line sum value LSX, the vertical line sum value LSY, the first diagonal line sum value LSD1, and the second diagonal line sum value LSD2.

[0105] According to an example embodiment, the event generator 540 may determine whether the omission event corresponding to the target pixel is an on-event or an off-event based on a first number of on-events and a second number of off-events corresponding to the neighboring pixels of the target pixel. For example, when the first number is greater than the second number, the event generator 540 may determine the omission event as an on-event. When the second number is greater than the first number, the event generator 540 may also determine the omission event as an off-event. For another example, when the first number is greater than the reference number, the event generator 540 may determine the omission event as an on-event, and when the second number is greater than the reference number, the omission event is determined as an off-event.

[0106] Fig. 9 is a diagram illustrating a pixel array included in an image sensor according to example embodiments.

[0107] refer to Fig. 9 , the pixel array may include a plurality of pixels PX arranged in a matrix form of multiple rows and columns. The pixel array may include image pixels, DVS pixels, or both image pixels and DVS pixels. Each pixel PX may be connected to a corresponding column line COL and a corresponding row line ROW. Fig. 9 An example is shown in which each pixel PX is connected to one column line COL and one row line ROW. Depending on the configuration of the pixel, each pixel may be connected to two or more column lines and / or two or more row lines.

[0108] FIG. 10A to FIG. 10D is a circuit diagram illustrating an example embodiment of an image pixel included in a pixel array.

[0109] refer to Fig. 10A, the unit pixel 600a may include a photosensitive element and a readout circuit. An example of the photosensitive element may be a photodiode PD. An example of the readout circuit may be a transfer transistor TX, a reset transistor RX, a drive transistor DX, and a selection transistor SX.

[0110] For example, the photodiode PD may include an n-type region in a p-type substrate, so that the n-type region and the p-type substrate form a pn junction diode. The photodiode PD receives incident light and generates photocharges based on the incident light. In some example embodiments, instead of or in addition to the photodiode PD, the unit pixel 600a may further include a phototransistor, a photogate, a pinned photodiode, etc.

[0111] The photocharge generated in the photodiode PD can be transferred to the floating diffusion node FD through the transfer transistor TX. The transfer transistor TX can be turned on in response to the transfer control signal TG. The drive transistor DX is used as a source follower amplifier that amplifies the signal corresponding to the charge on the floating diffusion node FD. The selection transistor SX can transmit the amplified signal VOUT to the column line COL in response to the selection signal SEL. The floating diffusion node FD can be reset by the reset transistor RX. For example, the reset transistor RX can discharge the floating diffusion node FD in response to the reset signal RS for correlated double sampling (CDS).

[0112] Fig. 10A 6 shows a unit pixel 600a of a four-transistor configuration including four transistors TX, RX, DX, and SX. Fig. 10B , Fig. 10C and Fig. 10D As shown, the configuration of the unit pixel may be variously changed. Power may be supplied via a voltage supply terminal VDD and a ground.

[0113] refer to Fig. 10B , the unit pixel 600b may have a three-transistor configuration including a photosensitive element such as a photodiode PD and a readout circuit including a reset transistor RX, a drive transistor DX, and a select transistor SX. Fig. 10A Compared with the unit pixel 600a, Fig. 10B The transfer transistor TX is omitted in the unit pixel 600b.

[0114] refer to Fig. 10C, the unit pixel 600c may have a five-transistor configuration including a photosensitive element such as a photodiode PD and a readout circuit including a transfer transistor TX, a gate transistor GX, a reset transistor RX, a drive transistor DX, and a select transistor SX. The gate transistor GX may selectively apply a transfer control signal TG to the transfer transistor TX in response to a select signal SEL. Fig. 10A Compared with the unit pixel 600a, the gate transistor GX is also included in Fig. 10C In the unit pixel 600c.

[0115] refer to Fig. 10D , the unit pixel 600d may have a five-transistor configuration including a photosensitive element such as a photodiode PD and a readout circuit including a phototransistor PX, a transfer transistor TX, a reset transistor RX, a drive transistor DX, and a selection transistor SX. The phototransistor PX may be turned on or off in response to a photogate signal PG. When the phototransistor PX is turned on, the unit pixel 600d may be enabled, and when the phototransistor PX is turned off, the unit pixel 600d may be disabled. Fig. 10A Compared with the unit pixel 600a, the phototransistor PX is also included in Fig. 10D In the unit pixel 600d. Fig. 10D In addition to the configuration of Fig. 10C The gate transistor GX (or bias transistor) of the six-transistor configuration.

[0116] The phototransistor PX may be turned on and off in response to the photo gate signal PG. When the phototransistor PX is turned on, the photodiode PD may detect incident light to generate photo charges. Conversely, when the phototransistor PX is turned off, the photodiode PD may not detect incident light.

[0117] Fig.11 is a circuit diagram illustrating an example embodiment of a DVS pixel included in a pixel array.

[0118] refer to Fig.11 , the DVS pixel 700 may include a photodiode PD, a current-to-voltage (I / V) converter 710 , an amplifier circuit 720 , a comparator circuit 730 , and digital logic 740 .

[0119] The photodiode PD is an example of a photoelectric conversion element. The photodiode PD may be a phototransistor, a photogate, a pinned photodiode (PPD), or a combination thereof. The photodiode PD may generate a photocurrent IP according to the intensity of incident light.

[0120] The I / V converter 710 may include a conversion transistor FT and an inversion amplifier LA. The inversion amplifier LA may convert and amplify an input voltage to output a first voltage VLOG. As a result, the I / V converter 710 may sense a photocurrent IP flowing through the photodiode PD to output a first current VLOG corresponding to the photocurrent IP.

[0121] The amplifier circuit 720 may include a first capacitor C1, a second capacitor C2, an amplifier AMP, and a reset switch SW. The reset switch SW may be switched in response to a reset signal RST provided from the digital logic 740. When the reset signal RST is activated, the reset switch SW may be turned on to discharge the second capacitor C2, and the second voltage VACC may be reset to a specific voltage level. When the reset signal RST is deactivated, the reset switch SW is turned off, and the input voltage of the amplifier circuit 720 may be maintained at a constant level due to the feedback path of the second capacitor C2. The input terminal of the amplifier AMP may be a virtual ground. Therefore, the amplifier circuit AMP may output a second voltage VACC based on the first voltage VLOG, so that the second voltage VACC may be related to the change of the first voltage VLOG over time.

[0122] The comparator circuit 730 may include a first comparator CMP1 and a second comparator CMP2. The first comparator CMP1 may compare the second voltage VACC with the first reference voltage VR1 to generate a turn-on event signal SON. The second comparator CMP2 may compare the second voltage VACC with the second reference voltage VR2 to generate a turn-off event signal SOFF.

[0123] For example, when the second voltage VACC is lower than the first reference voltage VR1, the first comparator CMP1 can activate the turn-on event signal SON. When the second voltage VACC is higher than the second reference voltage VR2, the second comparator CMP2 can activate the turn-off event signal SOFF. In this way, when the change in incident light is greater than a predetermined level, the comparator circuit 730 can activate the turn-on event signal SON and the turn-off event signal SOFF. When the shading of the DVS pixel 700 is brighter than a predetermined level, the turn-on event signal SON can be at a high level, and when the shading of the DVS pixel 700 is darker than a predetermined level, the turn-off event signal SOFF can be at a high level. The turn-on event signal SON and the turn-off event signal SOFF can be sent to the digital logic 740 via the column line COL and sent to the digital logic 740. Figure 3 Column AER 156.

[0124] The digital logic 740 may generate an on-off event signal SEVN based on the on-event signal SON and the off-event signal SOFF output from the comparator circuit 730. For example, the digital logic 740 may include an OR gate OR1 to receive the on-event signal SON and the off-event signal SOFF, and generate the on-off event signal SEVN when one of the on-event signal SON and the off-event signal SOFF is activated at a high level. The on-off event signal SEVN may be transmitted to the row AER 158 through the row line ROW. In addition, the digital logic 740 may include another OR gate OR2, wherein the OR gate OR2 activates the reset signal RST when one of the on-off event signal SEVN and the global reset signal GRST is activated at a high level.

[0125] Fig.12 It is shown Fig.11 A diagram of the operation of a DVS pixel.

[0126] exist Fig.12 , the vertical axis represents the intensity of incident light, and the horizontal axis represents time.

[0127] refer to Fig.12 During the first period t1, the intensity of the incident light remains constant, and thus the turn-on event signal SON and the turn-off event signal SOFF may not be activated.

[0128] During the second period t2 , the intensity of the incident light increases with the first slope, and the on-event signal SON may be activated three times.

[0129] During the third period t3 , the intensity of the incident light decreases at a second slope that is smaller than the first slope, and the off-event signal SOFF may be activated twice.

[0130] During the fourth period t2 , the intensity of the incident light increases with the second slope, and the on-event signal SON may be activated twice.

[0131] The intensity of the incident light is maintained constant during the fifth period t5. Therefore, the first voltage VLOG and the second voltage VACC may be maintained constant so that the turn-on event signal SON and the turn-off event signal SOFF may not be activated.

[0132] Fig.13 is a diagram illustrating an example embodiment of a layout of a pixel array included in an image sensor according to example embodiments.

[0133] refer to Fig.13 , the pixel array ARR_CM may include DVS pixels M arranged with image pixels including red pixels R, green pixels G, and blue pixels B. Fig.13 It is illustrated that one DVS pixel M is four times the size of one image pixel, and example embodiments are not limited thereto. In addition, the arrangement of the DVS pixel M and the color pixels R, G, and B may be variously changed.

[0134] Fig.14 It is shown that it has the Fig.13 Schematic diagram of the layout of a stacked structure pixel array.

[0135] refer to Fig.14 , the pixel array may have a stacked structure including a first semiconductor die 1211 and a second semiconductor die 1311. An image pixel (e.g., a CIS pixel) including a photosensitive element (e.g., a photodiode PD) may be formed in the first semiconductor die 1211, and a DVS pixel may be formed in the second semiconductor die 1311. For example, the photodiode PD formed in the first semiconductor die 1211 may be electrically connected to the DVS pixel formed in the second semiconductor die 1311 through a conductive path IC, so the image pixel and the DVS pixel may share the photodiode PD.

[0136] Fig.15 It is shown that Fig.14 A diagram of an example embodiment of an image pixel implemented in a pixel array of a stacked structure, and Fig.16 It is shown that Fig.14 Schematic diagram of an example embodiment of a DVS pixel implemented in a pixel array of a stacked structure.

[0137] The image pixels formed in the first semiconductor die 1211 are connected to the reference FIG. 10A to FIG. 10D The DVS pixel formed in the second semiconductor die 1311 is the same as the reference pixel. Fig.11 The descriptions are the same. The duplicate descriptions are omitted. Fig.15 The DVS pixel backend circuit 1315 may include the reference Fig.11 The amplifier circuit 720, comparator circuit 730 and digital logic 740.

[0138] Fig.15 The configuration in can be operated as an image pixel or a DVS pixel by the first switch SW1 and the second switch SW2 switched based on the switching control signal SWC.

[0139] When the first switch SW1 is turned on and the second switch SW2 is turned off, a voltage corresponding to the photocharges generated by the photodiode PD may be applied to the gate electrode of the driving transistor DX. Thus, four pixels in the first semiconductor die 1211 may operate as four image pixels.

[0140] When the first switch SW1 is turned off and the second switch SW2 is turned on, a voltage corresponding to the photocharges generated by the photodiode PD may be applied to the gate electrode of the conversion transistor FT. Therefore, one pixel in the second semiconductor die 1311 may operate as a DVS pixel.

[0141] Fig.16 A DVS pixel is shown using a photodiode PD formed in a first semiconductor die 1211 and other elements formed in a second semiconductor die 1311 .

[0142] Fig.17 It is shown that Fig.14 A diagram of an example embodiment of an image pixel implemented in a pixel array of a stacked structure, and Fig.18 It shows that Fig.14 Schematic diagram of an example embodiment of a pixel array with a stacked structure.

[0143] refer to Fig.17 , the first to fourth image pixels 1211a to 1211d may include first to fourth transfer transistors TX1 to TX4 that operate in response to first to fourth transfer control signals TG1 to TG4, respectively. The first to fourth transfer transistors TX1 to TX4 may be commonly connected to one floating diffusion node FD, and the first to fourth image pixels 1211a to 1211d may share readout elements RX, DX, and SX. The first to fourth transfer control signals TG1 to TG4 may be sequentially activated by a time division scheme. Data corresponding to photocharges in the first to fourth photodiodes PD1 to PD4 may be sequentially read out.

[0144] Apart from Fig.17 The configuration of the image pixels is applied, Fig.18 Configuration and reference Fig.14 , Fig.15 and Fig.16 The descriptions are basically the same, and duplicate descriptions are omitted.

[0145] Fig.19 is a block diagram illustrating an image sensor according to example embodiments. Fig.19 A first pixel array ARR_C including image pixels and a second pixel array ARR_M including DVS pixels are shown, which are physically separated. Fig.19 , the image sensor may include a first pixel array ARR_C in which a plurality of image pixels are arranged and a second pixel array ARR_M in which a plurality of DVS pixels are arranged.

[0146] Fig. 20 and Fig.21is a diagram illustrating an example embodiment of a layout of a pixel array included in an image sensor according to example embodiments. Fig. 20 An example layout of the first pixel array ARR_C is shown. Fig.21 An example layout of the second pixel array ARR_M is shown.

[0147] As reference Figure 3 The row driver RDEC and the readout circuit ROC may be placed adjacent to the first pixel array ARR_C to provide image data IMDT. Figure 3 The column AER AER_C and the row AER AER_R may be placed adjacent to the second pixel array ARR_M to provide event data EVDT.

[0148] In this way, the image sensor and the DVS distinct from the image sensor can be independently operated by separately implementing an element for the image data IMDA and an element for the event data EVDT.

[0149] Fig. 22 is a block diagram illustrating an example embodiment of a DVS event compensation apparatus included in an image sensor according to example embodiments. Fig.23 is used to describe the Fig. 22 A diagram of the operation of the synchronizer in the DVS event compensation circuit.

[0150] refer to Fig. 22 , the DVS event compensation device 501 may include a synchronizer SYNC and a DVS event compensation circuit DVSEC. Fig. 22 The DVS event compensation circuit DVSEC in Figure 4 The DVS event compensation circuit 500 in FIG. 1 is substantially the same, and repeated description is omitted.

[0151] The synchronizer SYNC can synchronize the original image data IMDT and the original event data EVDT in time to provide synchronized image data SIMDT and synchronized event data SEVDT. Fig.23 As shown, at time points T1, T2, and T3 corresponding to a specific frame rate, the raw image data IMDT is provided as frame image data CF1, CF2, and CF3, and the raw event data EVDT includes irregularly generated events. Fig.23 , events are represented by solid circles, the vertical axis indicates where the event occurs, and the horizontal axis indicates time.

[0152] For example, the synchronizer SYNC can synchronize the event during the first frame period PFR1 with the first frame image data CF1, and synchronize the event during the second frame period PFR2 with the second frame image data CF2. The synchronizer SYNC can also synchronize the event during the third frame period PFR3 with the third frame image data CF3. The synchronization method of the synchronizer SYNC is not limited thereto and can be determined differently.

[0153] Fig.24 is a block diagram illustrating an example embodiment of a DVS event compensation apparatus included in an image sensor according to an example embodiment, and Fig.25 and Fig.26 is used to describe the Fig.24 A diagram of the operation of the mode controller in the DVS event compensation circuit.

[0154] refer to Fig.24 , the DVS event compensation device 502 may include a mode controller MDCON and a DVS event compensation circuit DVSEC. Fig.24 The DVS event compensation circuit DVSEC in Figure 4 The DVS event compensation circuit 500 in FIG. 1 is substantially the same, and repeated description is omitted.

[0155] The mode controller MDCON may determine whether to perform the DVS event compensation operation as described above and generate an enable signal EN. When the enable signal EN is activated, the DVS event compensation circuit DVSEC may perform the DVS event compensation operation in response to the activated enable signal EN. In addition, the multiplexer MUX may provide compensated event data EVDT' as selected event data SEVDT in response to the activated enable signal EN. On the contrary, when the enable signal EN is deactivated, the DVS event compensation circuit DVSEC may not perform the DVS event compensation operation in response to the deactivated enable signal EN. In addition, the multiplexer MUX may provide original event data EVDT as selected event data SEVDT in response to the deactivated enable signal EN.

[0156] Fig.25 An example of an on-off event signal SEVN with irregular events is shown. In some example embodiments, the mode controller MDCON may determine whether to correct the DVS event based on the occurrence rate of the event. Fig.25 As shown, the mode controller MDCON may activate the enable signal EN when the occurrence rate of the event is higher than the reference value, and deactivate the enable signal EN when the occurrence rate of the event is lower than the reference value. According to example embodiments, the mode controller MDCON may generate the enable signal EN based on the on-off event signal SEVN or based on the event data EVDT.

[0157] refer to Fig.26 , event data corresponding to one frame may be divided into a plurality of subframes SBFR11 ˜ SBFR34 . In this case, as described above, the mode controller MDCON may determine whether to correct a DVS event for each of the plurality of subframes SBFR11 ˜ SBFR34 .

[0158] Typically, a captured frame image includes a fixed background and a moving object that is relatively smaller in size than the background. The above-described event compensation operation may be performed on a subframe including the moving object. By omitting the event compensation operation on a subframe that does not include the moving object, processing time and power consumption may be reduced, and the performance of an image sensor and a system including the image sensor according to an example embodiment may be enhanced.

[0159] Fig. 27 is a block diagram illustrating a computing system including an image sensor according to example embodiments.

[0160] refer to Fig. 27 , the computing system 2000 may include a processor 2010, a memory device 2020, a storage device 2030, an input / output (I / O) device 2040, a power supply 2050, and an image sensor 100. Although not shown in Fig. 27 , but the computing system 2000 may also include ports for communicating with video cards, sound cards, memory cards, Universal Serial Bus (USB) devices, and / or other electronic devices.

[0161] The processor 2010 may perform various calculations or tasks. According to some embodiments, the processor 2010 may be a microprocessor or a central processing unit (CPU). The processor 2010 may communicate with the memory device 2020, the storage device 2030, and the input / output device 2040 via an address bus, a control bus, and / or a data bus. In some example embodiments, the processor 2010 may be coupled to an expansion bus, such as a peripheral component interconnect (PCI) bus. The memory device 2020 may store data for operating the computing system 2000. For example, the memory device 2020 may be implemented with a dynamic random access memory (DRAM) device, a mobile DRAM device, a static random access memory (SRAM) device, a phase random access memory (PRAM) device, a ferroelectric random access memory (FRAM) device, a resistive random access memory (RRAM) device, and / or a magnetic random access memory (MRAM) device. The storage device 2030 may include a solid-state drive (SSD), a hard disk drive (HDD), a compact disc read-only memory (CD-ROM), etc. The input / output device 2040 may include an input device (e.g., a keyboard, a keypad, a mouse, etc.) and an output device (e.g., a printer, a display device, etc.). The power supply 2050 supplies an operating voltage to the computing system 2000.

[0162] The image sensor 100 may communicate with the processor 2010 via a bus or other communication link. The image sensor 100 may include a DVS event compensation circuit according to the example embodiment described above. The DVS event compensation circuit may generate edge data representing edge information of an image based on the image data, and generate an omitted event of the event data based on the edge data.

[0163] In this way, the method of correcting the event of the DVS and the image sensor according to the example embodiment can enhance the accuracy of the event data and the performance of the machine vision device and system operating based on the event data. Correcting the event of the DVS can be performed by supplementing the omitted events of the event data provided from the DVS using the edge information of the object of the image.

[0164] The inventive concept can be applied to any electronic device and system using a dynamic vision sensor. For example, the inventive concept can be applied to systems such as mobile phones, smart phones, personal digital assistants (PDAs), portable multimedia players (PMPs), digital cameras, camcorders, personal computers (PCs), server computers, workstations, laptop computers, digital TVs, set-top boxes, portable game consoles, navigation systems, wearable devices, Internet of Things (IoT) devices, Internet of Everything (IoE) devices, e-books, virtual reality (VR) devices, augmented reality (AR) devices, etc.

[0165] The foregoing is illustrative of example embodiments and is not to be construed as limiting thereof. Although a few example embodiments have been described, those skilled in the art will readily appreciate that many modifications are possible in the example embodiments without materially departing from the inventive concept.

Claims

1. A method for correcting a dynamic vision sensor DVS event, comprising: generating event data using the DVS pixels, the event data including events representing motion information of an object included in the image; generating image data capturing the image using image pixels; generating edge data representing edge information of the image based on the image data; as well as generating an omission event for the event data based on the edge data, Wherein, generating the omission event comprises: With respect to a target pixel among the DVS pixels and adjacent pixels of the target pixel, obtaining a line sum value by summing multiplied values ​​of the edge data and the event data in units of pixels in one direction; comparing the line sum value to a reference value; and The omission event is generated based on a result of the comparing.

2. The method according to claim 1, wherein: Each pixel of the edge data corresponds to a pixel of the event data.

3. The method according to claim 1, wherein: Generating the image data comprises: generating color pixel values ​​using color pixels corresponding to different colors, and Wherein, generating the edge data comprises: generating brightness data including brightness values ​​based on the color pixel values ​​of the image data and The edge data is generated based on the brightness data.

4. The method according to claim 1, further comprising: The edge data and the event data are spatially synchronized such that each pixel of the edge data corresponds to a pixel of the event data.

5. The method according to claim 1, wherein: Generating the event data includes: generating event data at a first resolution using the DVS pixels arranged at the first resolution, Wherein, generating the image data comprises: generating image data of a second resolution higher than the first resolution using the image pixels arranged at the second resolution, and Wherein, generating the edge data comprises: generating raw edge data of the second resolution indicating edge information of the image based on the image data of the second resolution; and The original edge data at the second resolution is converted into edge data at the first resolution.

6. The method according to claim 1, further comprising: The image data and the event data are synchronized in time.

7. The method according to claim 1, wherein: Obtaining the line and value includes: With respect to the target pixel and adjacent pixels adjacent to the target pixel in the horizontal direction, obtaining a horizontal line sum value by summing multiplied values ​​of the edge data and the event data in units of pixels in the horizontal direction; With respect to the target pixel and adjacent pixels adjacent to the target pixel in the vertical direction, obtaining a vertical line sum value by summing multiplied values ​​of the edge data and the event data in units of pixels in the vertical direction; With respect to the target pixel and adjacent pixels adjacent to the target pixel in a first diagonal direction, obtaining a first diagonal sum value by summing multiplied values ​​of the edge data and the event data in units of pixels in the first diagonal direction; and With respect to the target pixel and adjacent pixels adjacent to the target pixel in a second diagonal direction, a second diagonal sum value is obtained by summing the multiplied values ​​of the edge data and the event data in units of pixels in the second diagonal direction.

8. The method according to claim 7, wherein: Comparing the line sum value with the reference value comprises: Each of the horizontal line sum value, the vertical line sum value, the first diagonal line sum value, and the second diagonal line sum value is compared with a first reference value.

9. The method according to claim 8, wherein: Generating the omission event based on the result of the comparison includes: When at least one of the horizontal line sum value, the vertical line sum value, the first diagonal line sum value, and the second diagonal line sum value is greater than the first reference value, the omission event corresponding to the target pixel is generated.

10. The method according to claim 8, wherein: Generating the omission event based on the result of the comparison includes: When the count value is greater than a second reference value, the omission event corresponding to the target pixel is generated, wherein the count value indicates the number of values ​​greater than the first reference value from the horizontal line sum value, the vertical line sum value, the first diagonal line sum value and the second diagonal line sum value.

11. The method according to claim 1, wherein: Generating the omission event based on the result of the comparison includes: Based on a first number of on events and a second number of off events corresponding to neighboring pixels of the target pixel, it is determined whether the omitted event corresponding to the target pixel is an on event or an off event.

12. The method according to claim 11, wherein: Determining whether the omission event corresponding to the target pixel is the on event or the off event comprises: When the first number of on events is greater than the second number of off events, determining that the omitted event is the on event; and When the second number of closing events is greater than the first number of opening events, the omitted event is determined to be the closing event.

13. The method according to claim 11, wherein: Determining whether the omission event corresponding to the target pixel is the on event or the off event comprises: When the first number is greater than a reference number, determining that the omission event is the activation event; and When the second number is greater than the reference number, it is determined that the omission event is the closing event.

14. The method according to claim 1, wherein: When the target pixel is sequentially changed, generating the line sum value, comparing the line sum value with the reference value, and generating the omission event based on a result of the comparison are repeated.

15. The method according to claim 1, further comprising: A determination is made whether to correct the DVS event based on the occurrence rate of the event.

16. The method according to claim 1, further comprising: Dividing the event data into a plurality of subframes; as well as A determination is made whether to correct a DVS event for each of the plurality of subframes.

17. An image sensor comprising: A dynamic vision sensor DVS pixel configured to generate event data representing motion information of an object included in the image; image pixels configured to generate image data capturing said image; as well as a DVS event compensation circuit configured to generate edge data representing edge information of the image based on the image data, and to generate an omitted event of the event data based on the edge data, Wherein, generating the omission event comprises: With respect to a target pixel among the DVS pixels and adjacent pixels of the target pixel, obtaining a line sum value by summing multiplied values ​​of the edge data and the event data in units of pixels in one direction; comparing the line sum value to a reference value; and The omission event is generated based on a result of the comparing.

18. The image sensor according to claim 17, wherein: Each pixel of the edge data corresponds to a pixel of the event data.

19. A method for correcting a dynamic vision sensor (DVS) event, comprising: sensing event data representing a change of the image; sensing image data of the image; generating edge data representing edge information of the image based on the image data, wherein each pixel of the edge data corresponds to a pixel of the event data; obtaining a line sum value by summing the values ​​of the edge data multiplied by the value of the event data in a pixel direction relative to the target pixel; comparing the line sum value to a reference value; as well as An omit event is generated based on the comparison.

Images