Vision sensor and operation method thereof

Abstract

A vision sensor according to an embodiment of the present invention may comprise: a pixel array which includes a plurality of pixels arranged in the form of a matrix; an event detection circuit which includes a column AER circuit and a row AER circuit; and an event calculation circuit which receives an event signal provided from the event detection circuit to calculate an event value corresponding to a target pixel, wherein the event calculation circuit determines a mode on the basis of a context for the target pixel and calculates the event value corresponding to the target pixel according to the determined mode.

Description

Vision sensor and its operating method

[0001] The present invention relates to a vision sensor and its operating method. Specifically, it relates to a vision sensor employing a method for predicting events and its operating method.

[0002] The Dynamic Vision Sensor (DVS) system detects changes in light intensity and outputs information (event signals) about events based on these changes. Based on these event signals, it can recognize the shape and movement of objects. Dynamic Vision Sensor systems are utilized in various fields to detect and track moving objects.

[0003] Meanwhile, each pixel within the dynamic vision sensor can individually determine whether an event has occurred based on changes in light intensity. To effectively determine whether an event has occurred, ongoing research into the operation of the dynamic vision sensor is necessary.

[0004] The present invention aims to provide a vision sensor and its operating method for predicting an event of a target pixel by utilizing a previously confirmed event.

[0005] The technical tasks to be achieved by this embodiment are not limited to the technical tasks described above, and other technical tasks can be inferred from the following embodiments.

[0006] A vision sensor according to one embodiment of the present invention includes: a pixel array including a plurality of pixels arranged in a matrix form; an event detection circuit including a column AER circuit and a row AER circuit; and an event calculation circuit that receives an event signal from the event detection circuit and calculates an event value corresponding to a target pixel, wherein the event calculation circuit can determine a mode based on a context for the target pixel and calculate an event value corresponding to the target pixel according to the determined mode.

[0007] According to one embodiment of the present invention, the context may include at least one of: an event value calculation completed before calculating an event value for the target pixel; an event value for a pixel that is identical to the column in which the target pixel is located and is located in the previous row based on the row in which the target pixel is located; an event value for each of at least one pixel that is different from the column in which the target pixel is located and is located in the previous row based on the row in which the target pixel is located; an event value for each of at least one pixel that is different from the column in which the target pixel is located and is located in the same row as the row in which the target pixel is located; and an event value for each of at least one pixel that is different from the column in which the target pixel is located and is located in the next row based on the row in which the target pixel is located.

[0008] According to one embodiment of the present invention, when the number of contexts is 1 or less, the event generation circuit can be set to a coding mode to generate an event value corresponding to the target pixel, and the event value corresponding to the target pixel can be generated based on the event signal.

[0009] According to one embodiment of the present invention, when the number of contexts is two or more, the event generation circuit utilizes the contexts. Calculate the value, and If the absolute value of the value is greater than the preset threshold value, the prediction mode is set to produce the event value corresponding to the target pixel, and Based on the value, the event value corresponding to the target pixel can be predicted.

[0010] According to one embodiment of the present invention, the event generation circuit assigns different weights to each of the plurality of contexts by considering the physical distance of the target pixel. It can calculate the value.

[0011] In an operating method of a vision sensor according to an embodiment of the present invention, the method may include: a step of storing a context for a target pixel; a step of determining a mode based on the context; and a step of calculating an event value corresponding to the target pixel based on the determined mode.

[0012] According to one embodiment of the present invention, when the number of contexts is 1 or less, the step of determining the mode may set the event generating circuit to a coding mode to generate an event value corresponding to the target pixel, and the step of generating the event value corresponding to the target pixel may generate the event value corresponding to the target pixel based on the event signal.

[0013] According to one embodiment of the present invention, when the number of contexts is two or more, the step of determining the mode utilizes the contexts. Calculate the value, and If the absolute value of the value is greater than a preset threshold value, the step of setting the prediction mode to calculate the event value corresponding to the target pixel is as follows: Based on the value, the event value corresponding to the target pixel can be predicted.

[0014] According to one embodiment of the present invention, the step of determining the mode includes assigning different weights to each of the plurality of contexts by considering the physical distance of the target pixel. It can calculate the value.

[0015] According to the vision sensor and its operating method according to the present invention, the speed of operation of the vision sensor can be improved by predicting the event value of a target pixel using a previously confirmed event value.

[0016] FIG. 1 is a block diagram illustrating an image processing system according to an embodiment of the present invention.

[0017] FIG. 2 is a block diagram showing the configuration of a vision sensor according to one embodiment of the present invention.

[0018] FIG. 3 is a conceptual diagram showing a schematic configuration of a pixel array according to one embodiment.

[0019] Figure 4a is a conceptual diagram showing the scanning operation of a vision sensor according to one embodiment of the present invention.

[0020] Figure 4b is a conceptual diagram showing the scanning operation of a vision sensor according to another embodiment of the present invention.

[0021] Figure 5 is a flowchart showing the operation of a vision sensor according to one embodiment of the present invention.

[0022] Figure 6a is a flowchart showing the operation of a vision sensor according to one embodiment of the present invention.

[0023] Figure 6b is a conceptual diagram illustrating the operation of a vision sensor according to one embodiment of the present invention.

[0024] Figure 7 is a flowchart showing the operation of a vision sensor according to one embodiment of the present invention.

[0025] FIGS. 8A to 8E are conceptual diagrams showing the operation of a vision sensor according to one embodiment of the present invention.

[0026] Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. It should be understood that the present invention is not limited to specific embodiments, but rather encompasses various modifications, equivalents, and / or alternatives of the embodiments of the present invention. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0027] In this document, the expressions “has”, “may have”, “includes”, or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), but do not exclude the presence of additional features.

[0028] In this document, the expressions "A or B," "at least one of A and / or B," or "one or more of A and / or B" can include all possible combinations of the listed items. For example, "A or B," "at least one of A and B," or "at least one of A or B" can all refer to cases where (1) at least one A is included, (2) at least one B is included, or (3) at least one A and at least one B are included.

[0029] The terms "first," "second," "first," or "second" used herein may describe various components, regardless of order and / or importance, and are used only to distinguish one component from another, without limiting the components. For example, without departing from the scope of the rights set forth in this document, the first component may be renamed the second component, and similarly, the second component may be renamed the first component.

[0030] The expression "configured to" as used herein can be used interchangeably with, for example, "suitable for", "having the capacity to", "designed to", "adapted to", "made to", or "capable of". The term "configured to" does not necessarily mean "specifically designed to".

[0031] In this document, the words "command", "instruction", "control information", "message", "information", "data", "packet", "data packet", "intent" and / or "signal" transmitted and received between the first electronic device(s) and the second electronic device(s) may include or refer to human-perceivable ideas or specific electrical expressions (e.g., digital codes / analog physical quantities) regardless of their expressions. It will be apparent to those skilled in the art to which the invention disclosed in this document pertains that the exemplary expressions listed above may be interpreted in various ways depending on the context in which they are used. In this document, "is greater than B" not only simply means "is greater than B" but also includes the meaning of "is equal to or greater than B."

[0032] The terms used in this document are used only to describe specific embodiments and may not be intended to limit the scope of other embodiments. The singular expression may include the plural expression unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as commonly understood by those of ordinary skill in the art described in this document. Terms defined in general dictionaries among the terms used in this document may be interpreted as having the same or similar meaning in the context of the relevant technology, and shall not be interpreted in an idealized or overly formal sense unless explicitly defined in this document. In some cases, even if a term is defined in this document, it cannot be interpreted to exclude the embodiments of this document.

[0033] Fig. 1 is a block diagram for explaining an image processing system (1) according to an embodiment of the present invention.

[0034] An image processing system (1) may include a light source (10) and an electronic device (20).

[0035] According to an embodiment of the present invention, the light source (10) may include at least one light-emitting element. For example, the light source (10) may include an LED, a laser diode, and a VCSEL (Vertical Cavity Surface Emitting Laser).

[0036] According to an embodiment of the present invention, the electronic device (20) may include a vision sensor (21) and a processor (25).

[0037] The vision sensor (21) can collect light incident from the light source (10). The vision sensor (21) can detect a change in the intensity of the incident light and output an event signal. For example, when an event occurs in which the intensity of the light increases, the vision sensor (21) can output a corresponding positive event. Conversely, when an event occurs in which the intensity of the light decreases, the vision sensor (21) can output a negative event.

[0038] According to an embodiment of the present invention, the vision sensor (21) may be a dynamic vision sensor. For example, the vision sensor (21) may access a pixel where a change in light intensity is detected and output an event signal. For example, the change in light intensity may be generated from the movement of a subject being photographed by the vision sensor (21) or from the movement of the vision sensor (21) itself. The event signal may be mainly generated from the outline of the subject.

[0039] Since the vision sensor (21) outputs only values ​​corresponding to light whose intensity changes, the amount of processing data can be reduced compared to other sensors (e.g., CMOS image sensors, etc.).

[0040] The vision sensor (21) can detect changes in the intensity of the collected light and provide an event signal to the processor (25).

[0041] The processor (25) can receive an event signal from the vision sensor (21). The processor (25) can respond appropriately based on the event signal. For example, the processor (25) can execute or change the operation of the electronic device (20) based on the event signal.

[0042] Figure 2 is a block diagram showing the configuration of a vision sensor (21) according to one embodiment of the present invention.

[0043] According to one embodiment of the present invention, the vision sensor (21) may include an event detection circuit (210), an event generation circuit (220), and an output buffer (230).

[0044] The event detection circuit (220) may include a pixel array (211), a column AER (Address Event Representation) circuit (223), and a row AER circuit (225). According to an embodiment of the present invention, the event detection circuit (220) may detect an event by utilizing the pixel array (211), etc.

[0045] The pixel array (210) may include at least one pixel that detects an event based on a change in incident light brightness. The pixel array (210) may include at least one pixel in which at least one row and at least one column are arranged in a matrix form.

[0046] According to an embodiment of the present invention, a pixel may include a photoelectric conversion element that generates a charge according to the brightness of incident light. When the pixel detects a change in the brightness of the incident light based on the photocurrent flowing from the photoelectric conversion element, the pixel may provide a request for reading from the corresponding pixel to a column AER (Address Event Representation) circuit (213) and a row AER circuit (215). The pixel may output an event signal indicating that an event has been detected by the column AER circuit (213) and the row AER circuit (215).

[0047] Specifically, a signal notifying that an event in which the light intensity increases or decreases in a pixel included in the pixel array (210) has occurred may be provided from the pixel to the column AER circuit (220). For example, the pixel array (210) may detect a change in light from each pixel, output an output voltage, and compare the output voltage with a preset threshold value to determine whether an event has occurred. When an event occurs, the pixel array (210) may provide an event signal to the column AER circuit (220).

[0048] The column AER circuit (220) can provide a response signal (ACK) to the pixel in response to a signal received from a pixel that has detected an event. The pixel that has received the response signal (ACK) can provide polarity information of the occurred event to the row AER circuit (230).

[0049] The column AER circuit (220) can generate a column address of a pixel that detected an event based on a signal received from the pixel that detected the event.

[0050] The low AER circuit (230) can receive polarity information from a pixel that detects an event. Based on the polarity information, the low AER circuit (230) can generate a timestamp containing information about the time at which the event occurred.

[0051] The low AER circuit (230) can provide a reset signal to the pixel where the event occurred in response to polarity information. The reset signal can reset the pixel where the event occurred. Furthermore, the low AER circuit (230) can generate a row address of the pixel where the event occurred.

[0052] According to an embodiment of the present invention, a pixel can detect the presence or absence of an event by comparing a photocurrent according to the luminance of incident light with a preset threshold value. For example, if the amount of luminance change is greater than the preset threshold value, the pixel can detect the amount of change as a positive event. In another example, if the amount of luminance change is less than the preset threshold value, the pixel can detect the amount of change as a negative event.

[0053] The event detection circuit (210) can provide the generated event signal to the event generation circuit (220).

[0054] The event generation circuit (220) can generate event data based on the event signal provided from the event detection circuit (210). According to an embodiment of the present invention, the event data can include information such as a value corresponding to an event (hereinafter, event value), a row address of a pixel where an event occurred, a column address of a pixel where an event occurred, polarity, a timestamp, etc. According to an embodiment of the present invention, an event value corresponding to a positive event can be set to '1', and an event value corresponding to a negative event can be set to '-1'.

[0055] The event generation circuit (220) can set a mode based on context information (hereinafter, “context”). The context refers to an event value corresponding to at least one pixel (hereinafter, “target pixel”) that is close to a pixel that is considered for generating the current event value among at least one pixel for which an event value has already been generated.

[0056] For example, the context means that the event value calculation is completed before the event value for the target pixel is calculated, and at least one of the event value for a pixel that is identical to the column where the target pixel is located and is located in the previous row based on the row where the target pixel is located, the event value for at least one pixel that is different from the column where the target pixel is located and is located in the previous row based on the row where the target pixel is located, the event value for at least one pixel that is different from the column where the target pixel is located and is located in the same row as the row where the target pixel is located, and the event value for at least one pixel that is different from the column where the target pixel is located and is located in the next row based on the row where the target pixel is located.

[0057] According to one embodiment of the present invention, when there is one context, the event generation circuit (220) can set a coding mode to generate event data. In the coding mode, the event generation circuit (220) can generate event data corresponding to a target pixel based on an event signal provided from the event detection circuit (210).

[0058] According to another embodiment of the present invention, when there are four contexts, the event generation circuit (220) can set a prediction mode to generate event data. In the prediction mode, the event generation circuit (220) can generate event data corresponding to the target pixel based on the context.

[0059] According to an embodiment of the present invention, the event generation circuit (220) may include a buffer (hereinafter, "context buffer") that stores context for generating event data later. The event generation circuit (220) may utilize the context stored in the context buffer to set a mode and generate event data corresponding to a target pixel.

[0060] The output buffer (250) can generate a packet based on column address, row address, and polarity information. The output buffer (250) can add a header indicating the start of the packet to the front of the packet and a tail indicating the end of the packet to the back of the packet.

[0061] Although not shown in the drawing, the vision sensor (21) may further include a controller and memory, etc.

[0062] The controller can control the event detection circuit (210), the event generation circuit (220), and the output buffer. Based on the received event, the controller can identify pixels that detect movement, characteristics, etc. of the subject, and identify information about time, movement direction, etc.

[0063] According to an embodiment of the present invention, the controller can control the reading order of pixels included in the pixel array. The reading control order can be set by the designer.

[0064] The memory can store various data to support the operation of the vision sensor (21) according to an embodiment of the present invention.

[0065] FIG. 3a is a conceptual diagram showing a schematic configuration of a pixel array (211) according to one embodiment.

[0066] A plurality of unit pixels (PX) arranged in an 'M x N' array form can be connected to a plurality of row lines (ROW 0 to ROW M-1) and a plurality of column lines (COL 0 to COL N-1).

[0067] A plurality of unit pixels (PX) included in the pixel array (210) can be scanned in units of rows (ROW) or columns (COL). In one embodiment of the present invention, while sequentially scanning a plurality of columns (COL), pixel signals can be detected from the unit pixels (PX) and an event signal can be generated using the pixel signals.

[0068] Fig. 4a is a conceptual diagram illustrating a scanning operation of a vision sensor (21) according to one embodiment of the present invention. Fig. 4b is a conceptual diagram illustrating a scanning operation of a vision sensor (21) according to another embodiment of the present invention.

[0069] According to one embodiment of the present invention, referring to FIG. 4A, after scanning from the pixels located in the first row (ROW[1]) to the pixels located in the fifth row (ROW[5]) among the pixels included in the first column (COL[1]), the scanning operation may be sequentially performed in the order of the second column (COL[2]), the third column (COL[3]), the fourth column (COL[4]), and the fifth column (COL[5]).

[0070] According to another embodiment of the present invention, referring to FIG. 4b, after scanning from the pixels located in the first row (ROW[1]) to the pixels located in the fifth row (ROW[5]) among the pixels included in the first column (COL[1]), the scanning operation may be performed randomly in the order of the fourth column (COL[4]), the second column (COL[2]), the fifth column (COL[5]), and the third column (COL[3]).

[0071] Fig. 5 is a flowchart illustrating the operation of a vision sensor (21) according to one embodiment of the present invention. In particular, Fig. 5 illustrates the operation of a vision sensor (21) that calculates an event value corresponding to a target pixel according to an embodiment of the present invention.

[0072] In step S501, the vision sensor (21) can detect an event signal corresponding to a target pixel. Specifically, an event detection circuit (210) included in the vision sensor (21) can detect an event signal corresponding to the target pixel.

[0073] In step S503, the vision sensor (21) can determine a mode to produce an event value corresponding to the target pixel.

[0074] According to an embodiment of the present invention, an event detection circuit (210) provides an event signal to an event generation circuit (220), and the event generation circuit (220) can determine a mode to generate an event value corresponding to a target pixel based on at least one pre-stored context.

[0075] According to one embodiment of the present invention, if there is 0 or 1 pre-stored context, the event generation circuit (220) can determine the mode for generating an event value corresponding to a target pixel as a coding mode.

[0076] According to another embodiment of the present invention, if there are two or more pre-stored contexts, the event generation circuit (220) may determine a mode for generating an event value corresponding to a target pixel based on the context as a prediction mode.

[0077] For example, the event generation circuit (220) may determine a mode for generating an event value corresponding to a target pixel as a prediction mode when the absolute value of the context sum is greater than a preset threshold value.

[0078] According to one embodiment of the present invention, when there are two or more contexts, the event generation circuit (220) can calculate the absolute value of the sum of contexts by adding a weight to a context corresponding to a pixel having a physical distance closer to a target pixel among the plurality of contexts, and can determine the mode as a prediction mode based thereon.

[0079] In step S505, the vision sensor (21) can calculate or predict an event value corresponding to a target pixel according to preset criteria based on the determined mode and context. This is specifically described in FIGS. 6A to 8B.

[0080] According to one embodiment of the present invention, when there is 0 or 1 context, the event generating circuit (220) can generate an event value corresponding to the target pixel.

[0081] According to another embodiment of the present invention, when there are two or more contexts, the event generation circuit (220) can predict an event value corresponding to a target pixel based on the contexts according to a preset method. The event generation circuit (220) can predict an event value corresponding to a target pixel by adding a weight to a context corresponding to a pixel that is physically closer to the target pixel among the plurality of contexts.

[0082] In step S507, the vision sensor (21) can output the generated event value (event data).

[0083] According to one embodiment of the present invention, when there is 0 or 1 context, the vision sensor (21) can output a calculated event value corresponding to the target pixel.

[0084] According to another embodiment of the present invention, when there are two or more contexts, the vision sensor (21) can output a predicted event value corresponding to the target pixel.

[0085] Fig. 6a is a flowchart illustrating the operation of a vision sensor (21) according to one embodiment of the present invention. Fig. 6b is a conceptual diagram illustrating the operation of a vision sensor (21) according to one embodiment of the present invention. In particular, Figs. 6a and 6b illustrate the operation of an event generation circuit (220) according to an embodiment of the present invention.

[0086] In step S601, when the event generation circuit (220) receives an event signal from the event detection circuit (210), it can check whether the number of contexts stored in the context buffer is 1 or less.

[0087] According to an embodiment of the present invention, the context means that the event value calculation is completed before the event value for the target pixel is calculated, and the event value for the pixel located in the previous row based on the row where the target pixel is located and the event value for the pixel located in the next row based on the row where the target pixel is located.

[0088] Referring to Fig. 6b, the order of calculating the event value is sequentially performed from the first row (ROW[1]) for the pixels included in the first column (COL[1]). The target pixel (C) is located in the third row (ROW[3]), and the event values ​​corresponding to the pixels located in the first row (ROW[1]) and the pixels located in the second row (ROW[2]) have been calculated.

[0089] In this case, the context utilized to calculate the event value of the target pixel is the event value of the pixel located in the second row (ROW[2]). The event calculation circuit (220) can confirm that the number of contexts utilized to calculate the event value of the target pixel (C) is 1.

[0090] If the number of confirmed contexts is greater than 1 ('NO' in step S601), the event generation circuit (220) operation described in FIG. 7a and FIG. 8a may proceed.

[0091] On the other hand, if the number of confirmed contexts is 1 or less ('YES' in step S601), in step S603, the event generation circuit (220) can be set to coding mode.

[0092] In step S605, the event generation circuit (220) can generate an event value based on an event signal provided from the event detection circuit (210).

[0093] Fig. 7 is a flowchart illustrating the operation of a vision sensor (21) according to one embodiment of the present invention. In particular, Fig. 7 illustrates the operation of an event generation circuit (220) when there are two or more pre-stored contexts according to one embodiment of the present invention.

[0094]

[0095] In step S701, the event generation circuit (220) utilizes a preset method. A value can be calculated. According to an embodiment of the present invention, The value can be used as a prediction tool that utilizes pixels close to the target pixel to predict the event value of the target pixel. In addition, The value can be calculated by assigning a weight set by considering the physical distance from the target pixel to each pixel close to the target pixel.

[0096] Produced If the absolute value of the value is less than or equal to a preset threshold value ('NO' in step S701), the event generation circuit (220) can set the mode for generating the event value corresponding to the target pixel to the coding mode in step S703.

[0097] Furthermore, in step S705, the event generation circuit (220) can generate an event value of a target pixel according to a set coding mode, and in step S709, the vision sensor (21) can output the generated event value.

[0098] On the other hand, the produced If the absolute value of the value is greater than the preset threshold value ('YES' in step S701), in step S707, the event generation circuit (220) can be set to the prediction mode to generate the event value corresponding to the target pixel. At the same time, the event generation circuit (220) Based on the value, an event value corresponding to the target pixel can be estimated. According to one embodiment of the present invention, the event generation circuit (220) The event value corresponding to the target pixel can be estimated by the sign of the value.

[0099] In step S709, the vision sensor (21) can output an event value corresponding to the target pixel estimated in step S707.

[0100] Figures 8a to 8f are conceptual diagrams illustrating the operation of a vision sensor (21) according to an embodiment of the present invention. In particular, Figures 8a to 8f illustrate the operation of a vision sensor (21) that estimates an event value corresponding to a target pixel according to a context.

[0101] Referring to Fig. 8a, the order of calculating the event value is performed in the order of the first column (COL[1]), the second column (COL[2]), and sequentially from the first row (ROW[1]) to the fifth row (ROW[5]) for the pixels included in the same column. The target pixel (C) is located in the third row (ROW[3]) among the pixels included in the second column (COL[2]).

[0102] The contexts utilized to calculate the event value of the target pixel are the event value (R0) of the pixel located in the third row (ROW[3]) among the pixels included in the first column (COL[1]) and the event value (R1) of the pixel located in the second row (ROW[2]) among the pixels included in the second column (COL[2]). The number of contexts to be utilized by the event calculation circuit (220) to calculate the event value of the target pixel (C) can be confirmed as two.

[0103] The event generation circuit (220) utilizes R0 and R1 (i.e., context). The value can be calculated. Specifically, the event calculation circuit (220) is calculated using the following mathematical expression 1. It can calculate the value.

[0104] [Mathematical Formula 1]

[0105] However, the above mathematical expression 1 is only an example and is not limited thereto.

[0106] The event generation circuit (220) is generated Compare the absolute value of the value with a preset threshold value. If the absolute value of the value is greater than a preset threshold, the prediction mode can be set to produce an event value corresponding to the target pixel. At the same time, the event production circuit (220) Based on the value, the event value corresponding to the target pixel can be estimated.

[0107] Referring to Fig. 8b, the order of calculating the event value is performed in the order of the first column (COL[1]), the second column (COL[2]), and sequentially from the first row (ROW[1]) to the fifth row (ROW[5]) for the pixels included in the same column. The target pixel (C) is located in the third row (ROW[3]) among the pixels included in the second column (COL[2]).

[0108] The contexts utilized to calculate the event value of the target pixel are the event value (R2) of the pixel located in the second row (ROW[2]) among the pixels included in the first column (COL[1]), the event value (R0) of the pixel located in the third row (ROW[3]) among the pixels included in the first column (COL[1]), and the event value (R1) of the pixel located in the second row (ROW[2]) among the pixels included in the second column (COL[2]). The number of contexts to be utilized by the event calculation circuit (220) to calculate the event value of the target pixel (C) can be confirmed to be three.

[0109] The event generation circuit (220) utilizes R0, R1, and R2 (i.e., context). The value can be calculated. Specifically, the event calculation circuit (220) is calculated using the following mathematical expression 2. It can calculate the value.

[0110] [Equation 2]

[0111] However, the above mathematical formula 2 is only an example and is not limited thereto.

[0112] The event generation circuit (220) assigns weights to R0 and R1 because they are event values ​​of the pixels closest to the target pixel. It can calculate the value.

[0113] The event generation circuit (220) is generated Compare the absolute value of the value with a preset threshold value. If the absolute value of the value is greater than a preset threshold, the prediction mode can be set to produce an event value corresponding to the target pixel. At the same time, the event production circuit (220) Based on the value, the event value corresponding to the target pixel can be estimated.

[0114] Referring to Fig. 8c, the order of calculating the event value is performed in the order of the first column (COL[1]), the second column (COL[2]), and sequentially from the first row (ROW[1]) to the fifth row (ROW[5]) for the pixels included in the same column. The target pixel (C) is located in the third row (ROW[3]) among the pixels included in the second column (COL[2]), and the event values ​​corresponding to all pixels included in the first column (COL[1]), the pixels located in the first row (ROW[1]) among the pixels included in the second column (COL[2]), and the pixels located in the second row (ROW[2]) among the pixels included in the second column (COL[2]) have been calculated.

[0115] The contexts utilized to calculate the event value of the target pixel are the event value (R2) of the pixel located in the second row (ROW[2]) among the pixels included in the first column (COL[1]), the event value (R0) of the pixel located in the third row (ROW[3]) among the pixels included in the first column (COL[1]), the event value (R3) of the pixel located in the fourth row (ROW[4]) among the pixels included in the first column (COL[1]), and the event value (R1) of the pixel located in the second row (ROW[2]) among the pixels included in the second column (COL[2]). The number of contexts utilized by the event calculation circuit (220) to calculate the event value of the target pixel (C) can be confirmed to be 4.

[0116] The event generation circuit (220) utilizes R0, R1, R2, and R3 (i.e., context). The value can be calculated. Specifically, the event calculation circuit (220) is calculated using the following mathematical formula 3. It can calculate the value.

[0117] [Equation 3]

[0118] However, the above mathematical expression 3 is only an example and is not limited thereto.

[0119] The event generation circuit (220) assigns weights to R0 and R1 because they are event values ​​of the pixels closest to the target pixel. It can calculate the value.

[0120] The event generation circuit (220) is generated Compare the absolute value of the value with a preset threshold value. If the absolute value of the value is greater than a preset threshold, the prediction mode can be set to produce an event value corresponding to the target pixel. At the same time, the event production circuit (220) Based on the value, the event value corresponding to the target pixel can be estimated.

[0121] Referring to FIG. 8d, the order of calculating the event value is performed in the order of the second column (COL[2]), the first column (COL[1]), and sequentially from the first row (ROW[1]) to the fifth row (ROW[5]) for the pixels included in the same column. The target pixel (C) is located in the third row (ROW[3]) among the pixels included in the first column (COL[1]), and the event values ​​corresponding to all pixels included in the first column (COL[2]), the pixels located in the first row (ROW[1]) among the pixels included in the first column (COL[1]), and the pixels located in the second row (ROW[2]) among the pixels included in the first column (COL[1]) have completed calculation.

[0122] In this case, the contexts utilized to calculate the event value of the target pixel are the event value (R2) of the pixel located in the second row (ROW[2]) among the pixels included in the second column (COL[2]), the event value (R0) of the pixel located in the third row (ROW[3]) among the pixels included in the second column (COL[2]), the event value (R3) of the pixel located in the fourth row (ROW[4]) among the pixels included in the second column (COL[2]), and the event value (R1) of the pixel located in the second row (ROW[2]) among the pixels included in the first column (COL[1]). The number of contexts utilized by the event calculation circuit (220) to calculate the event value of the target pixel (C) can be confirmed to be 4.

[0123] The event generation circuit (220) utilizes R0, R1, R2, and R3 (i.e., context). The value can be calculated. Specifically, the event calculation circuit (220) is calculated using the following mathematical formula 4. It can calculate the value.

[0124] [Equation 4]

[0125] However, the above mathematical expression 4 is only an example and is not limited thereto.

[0126] The event generation circuit (220) assigns weights to R0 and R1 because they are event values ​​of the pixels closest to the target pixel. It can calculate the value.

[0127] The event generation circuit (220) is generated Compare the absolute value of the value with a preset threshold value. If the absolute value of the value is greater than a preset threshold, the prediction mode can be set to produce an event value corresponding to the target pixel. At the same time, the event production circuit (220) Based on the value, the event value corresponding to the target pixel can be estimated.

[0128] Referring to FIG. 8e, the order of calculating the event value is performed in the order of the first column (COL[1]), the third column (COL[3]), the second column (COL[2]), and sequentially from the first row (ROW[1]) to the fifth row (ROW[5]) for the pixels included in the same column. The target pixel (C) is located in the third row (ROW[3]) among the pixels included in the second column (COL[2]), and the event values ​​corresponding to all pixels included in the first column (COL[1]), all pixels included in the third column (COL[3]), and the pixels located in the first row (ROW[1]) among the pixels included in the second column (COL[2]), and the pixels located in the second row (ROW[2]) among the pixels included in the second column (COL[2]) have been calculated.

[0129] In this case, the context used to calculate the event value of the target pixel is the event value (R3) of the pixel located in the second row (ROW[2]) among the pixels included in the first column (COL[1]), the event value (R0) of the pixel located in the third row (ROW[3]) among the pixels included in the first column (COL[1]), the event value (R5) of the pixel located in the fourth row (ROW[4]) among the pixels included in the first column (COL[1]), the event value (R4) of the pixel located in the second row (ROW[2]) among the pixels included in the second column (COL[2]), the event value (R2) of the pixel located in the third row (ROW[3]) among the pixels included in the second column (COL[2]), the event value (R6) of the pixel located in the fourth row (ROW[4]) among the pixels included in the second column (COL[2]), and the event value (R7) of the pixel located in the second row (ROW[4]) among the pixels included in the second column (COL[2]). This is the event value (R1) of the pixel located in row (ROW[2]). The number of contexts to be utilized by the event generation circuit (220) to generate the event value of the target pixel (C) can be confirmed to be 7.

[0130] The event generation circuit (220) is composed of R0, R1, R2, R 3, R 4, R 5, By leveraging R6 (i.e. context) The value can be calculated. Specifically, the event calculation circuit (220) is calculated using the following mathematical expression 5. It can calculate the value.

[0131] [Equation 5]

[0132] However, the above mathematical expression 5 is only an example and is not limited thereto.

[0133] The event generation circuit (220) is R0, R 1, and R2 is the event value of the pixel closest to the target pixel, so it is weighted. It can calculate the value.

[0134] The event generation circuit (220) is generated Compare the absolute value of the value with a preset threshold value. If the absolute value of the value is greater than a preset threshold, the prediction mode can be set to produce an event value corresponding to the target pixel. At the same time, the event production circuit (220) Based on the value, the event value corresponding to the target pixel can be estimated.

[0135] Referring to Fig. 8c, the order of calculating the event value is the first column (COL[1]), the fifth column (COL[5]), the third column (COL[3]), and is sequentially calculated from the first row (ROW[1]) to the fifth row (ROW[5]) for the pixels included in the same column. The target pixel (C) is located in the third row (ROW[3]) among the pixels included in the second column (COL[2]), and the event values ​​corresponding to all pixels included in the first column (COL[1]), all pixels included in the fifth column (COL[5]), and the pixels located in the first row (ROW[1]) among the pixels included in the third column (COL[3]), and the pixels located in the second row (ROW[2]) among the pixels included in the third column (COL[3]) have been calculated.

[0136] In this case, the context used to calculate the event value of the target pixel is the event value (R3) of the pixel located in the second row (ROW[2]) among the pixels included in the first column (COL[1]), the event value (R0) of the pixel located in the third row (ROW[3]) among the pixels included in the first column (COL[1]), the event value (R5) of the pixel located in the fourth row (ROW[4]) among the pixels included in the first column (COL[1]), the event value (R4) of the pixel located in the second row (ROW[2]) among the pixels included in the fifth column (COL[5]), the event value (R2) of the pixel located in the third row (ROW[3]) among the pixels included in the fifth column (COL[5]), the event value (R6) of the pixel located in the fourth row (ROW[4]) among the pixels included in the fifth column (COL[5]), and the event value (R6) of the pixel located in the second row (ROW[4]) among the pixels included in the third column (COL[3]). This is the event value (R1) of the pixel located in row (ROW[2]). The number of contexts to be utilized by the event generation circuit (220) to generate the event value of the target pixel (C) can be confirmed to be 7.

[0137] The event generation circuit (220) is composed of R0, R1, R2, R 3, R 4, R 5, By leveraging R6 (i.e. context) The value can be calculated. Specifically, the event calculation circuit (220) is calculated using the following mathematical expression 6. It can calculate the value.

[0138] [Equation 6]

[0139] However, the above mathematical expression 6 is only an example and is not limited thereto.

[0140] The event generation circuit (220) assigns the greatest weight to R0 because it is the closest pixel to the target pixel, and R 1, and R2 is the closest pixel to the target pixel, so it is an event value and is given a weighting factor. It can calculate the value.

[0141] The event generation circuit (220) is generated Compare the absolute value of the value with a preset threshold value. If the absolute value of the value is greater than a preset threshold, the prediction mode can be set to produce an event value corresponding to the target pixel. At the same time, the event production circuit (220) Based on the value, the event value corresponding to the target pixel can be estimated.

[0142] Although all components constituting the embodiments of the present invention have been described as being combined or operating in combination, the present invention is not necessarily limited to such embodiments. That is, within the scope of the present invention, all components may be selectively combined and operated one or more times.

[0143] Meanwhile, the various embodiments described herein may be implemented by hardware, middleware, microcode, software, and / or a combination thereof. For example, the various embodiments may be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions presented herein, or a combination thereof.

[0144] Additionally, for example, various embodiments may be embodied or encoded in a computer-readable medium containing instructions. Instructions embodied or encoded in the computer-readable medium may cause a programmable processor or other processor to perform a method when the instructions are executed, for example. The computer-readable medium includes a computer storage medium, which may be any available medium that can be accessed by a computer. For example, such a computer-readable medium may include a RAM, a ROM, an EEPROM, a CD-ROM or other optical disk storage medium, a magnetic disk storage medium, or other magnetic storage devices.

[0145] Such hardware, software, firmware, etc. may be implemented within the same device or within separate devices to support the various operations and functions described herein. Additionally, components, units, modules, components, etc. described as “units” in the present invention may be implemented together or individually as separate but interoperable logic devices. The depiction of different features for modules, units, etc. is intended to highlight different functional embodiments and does not necessarily imply that they must be realized by separate hardware or software components. Rather, the functionality associated with one or more modules or units may be performed by separate hardware or software components, or integrated into common or separate hardware or software components.

[0146] Although operations are depicted in the drawings in a particular order, this should not be construed as requiring that these operations be performed in the particular order depicted, or in any sequential order, or that all depicted operations be performed to achieve the desired results. In certain circumstances, multitasking and parallel processing may be advantageous. Furthermore, the distinction between various components in the embodiments described above should not be construed as requiring such distinction in all embodiments, and it should be understood that the components depicted may generally be integrated together into a single software product or packaged into multiple software products.

[0147] The electronic device, server, or external device according to the various embodiments of the present document described above may include, for example, at least one of a smartphone, a tablet PC, a mobile phone, a video phone, a desktop PC, a laptop PC, a personal digital assistant (PDA), a portable multimedia player (PMP), an MP3 player, a mobile medical device, a camera, or a wearable device.

[0148] According to various embodiments, the wearable device may include at least one of an accessory type (e.g., a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD)), a fabric or clothing-integrated type (e.g., an electronic garment), a body-attached type (e.g., a skin pad or tattoo), or a bio-implant type (e.g., an implantable circuit).

[0149] In some embodiments, the electronic device or external device may be a home appliance. The home appliance may include, for example, at least one of a television, a digital video disk player (DVD player), an audio device, a refrigerator, an air conditioner, a vacuum cleaner, an oven, a microwave oven, a washing machine, an air purifier, a set-top box, a home automation control panel, a security control panel, a TV box, a game console, an electronic dictionary, an electronic key, a camcorder, or an electronic picture frame.

[0150] In another embodiment, the electronic device, external device, or wearable device may include at least one of various medical devices (e.g., various portable medical measuring devices (such as a blood glucose meter, a heart rate meter, a blood pressure meter, or a body temperature meter), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), computed tomography (CT), a camera, or an ultrasound machine), a navigation device, a satellite navigation system (Global Navigation Satellite System (GNSS)), an event data recorder (EDR), a flight data recorder (FDR), an automobile infotainment device, a home robot, or an internet of things device (e.g., a light bulb, various sensors, an electric or gas meter, a sprinkler device, a fire alarm, a thermostat, a streetlight, an exercise machine, a hot water tank, a heater, a boiler, or the like).

[0151]

[0152] As described above, the best practice embodiments have been disclosed in the drawings and specifications. While specific terminology has been used herein, it is solely for the purpose of describing the present invention and is not intended to limit the scope of the invention as defined in the claims. Therefore, those skilled in the art will understand that various modifications and equivalent embodiments are possible. Therefore, the true technical protection scope of the present invention should be determined by the technical spirit of the appended claims.

Claims

1. An event detection circuit including a pixel array including a plurality of pixels arranged in a matrix form, a column AER circuit, and a row AER circuit; and An event generation circuit that receives an event signal from the above event detection circuit and generates an event value corresponding to the target pixel. Including, The above event generation circuit is, Determine a mode based on the context for the target pixel, and calculate an event value corresponding to the target pixel according to the determined mode. Vision sensor.

2. In claim 1, The above context is, Event value calculation is completed before calculating the event value for the above target pixel, An event value for a pixel that is identical to the column where the target pixel is located and is located in the previous row based on the row where the target pixel is located, an event value for each of at least one or more pixels that is different from the column where the target pixel is located and is located in the previous row based on the row where the target pixel is located, an event value for each of at least one or more pixels that is different from the column where the target pixel is located and is located in the same row as the row where the target pixel is located, and an event value for each of at least one or more pixels that is different from the column where the target pixel is located and is located in the next row based on the row where the target pixel is located. Vision sensor.

3. In claim 1, If the number of the above contexts is 1 or less, The above event generating circuit is set to a coding mode to generate an event value corresponding to the target pixel, and generates an event value corresponding to the target pixel based on the event signal. Vision sensor.

4. In claim 1, If the number of the above contexts is 2 or more, The above event generation circuit utilizes the above context. Calculate the value, and If the absolute value of the value is greater than the preset threshold value, the prediction mode is set to produce the event value corresponding to the target pixel, and Predicting the event value corresponding to the target pixel based on the value Vision sensor.

5. In claim 4, The above event generation circuit assigns different weights to each of the multiple contexts by considering the physical distance of the target pixel. Calculating the value Vision sensor.

6. In the operation method of the vision sensor, Step of saving context for target pixel; A step of determining a mode based on the above context; and A step of calculating an event value corresponding to the target pixel according to the determined mode; An operating method of a vision sensor including:

7. In claim 6, The above context is, Event value calculation is completed before calculating the event value for the above target pixel, An event value for a pixel that is identical to the column where the target pixel is located and is located in the previous row based on the row where the target pixel is located, an event value for each of at least one or more pixels that is different from the column where the target pixel is located and is located in the previous row based on the row where the target pixel is located, an event value for each of at least one or more pixels that is different from the column where the target pixel is located and is located in the same row as the row where the target pixel is located, and an event value for each of at least one or more pixels that is different from the column where the target pixel is located and is located in the next row based on the row where the target pixel is located. How the vision sensor works.

8. In claim 6, If the number of the above contexts is 1 or less, The steps for determining the above mode are: The above event generation circuit is set to coding mode to generate an event value corresponding to the target pixel, The step of calculating the event value corresponding to the target pixel is: Calculating an event value corresponding to the target pixel based on the above event signal How the vision sensor works.

9. In claim 6, If the number of the above contexts is 2 or more, The steps for determining the above mode are: Using the above context Calculate the value, and If the absolute value of the value is greater than a preset threshold, the prediction mode is set to produce an event value corresponding to the target pixel. The step of calculating the event value corresponding to the target pixel is: Above Predicting the event value corresponding to the target pixel based on the value How the vision sensor works.

10. In claim 9, The steps for determining the above mode are: By considering the physical distance of the target pixel, different weights are given to each of the multiple contexts. Calculating the value How the vision sensor works.

Images