Method and system for generating DVS frames

Abstract

A method and system for generating a frame for a DVS are disclosed. The method includes the steps of receiving a DVS pixel of an object from a DVS, initializing a timespan to a timespan value, generating a frame by integrating DVS pixels captured within the timespan, evaluating the movement speed of an object, and combining a plurality of consecutive frames into a single combined frame when the evaluated movement speed is lower than a first threshold value. The system includes a DVS configured to capture a DVS pixel of an object, and a processor configured to receive a DVS pixel from the DVS and perform the method.

Description

Technology Field

[0001] The present disclosure generally relates to a method and system for generating DVS frames. Background Technology

[0002] There is a new, state-of-the-art sensor called the DVS (Dynamic Vision Sensor). The DVS captures only changes in luminous intensity and then generates asynchronous DVS pixels (pixel events). Compared to conventional cameras, the DVS has the advantages of low latency, no motion blur, high dynamic range, and low power consumption.

[0003] Unlike conventional cameras that generate an entire image or frame composed of continuous pixels, DVS generates only asynchronous DVS pixels that can be spatiotemporally discrete and discontinuous; therefore, to create an image or frame, it is necessary to integrate the DVS pixels (pixel events) captured by the DVS within a time span. Since DVS captures only changes in luminosity, if the scene captured by the DVS is stable—for example, when the object to be detected is nearly stationary or moving slowly—it may not be able to capture enough pixels for a satisfactory image or frame within a specific time span. On the other hand, when the scene changes rapidly—for example, when the object to be detected moves quickly within the scene—the DVS may capture an excessive number of redundant DVS pixels within a specific time span.

[0004] Therefore, a method or system is needed to acquire satisfactory images or frames for DVS regardless of the movement speed of the object to be detected.

[0005] Furthermore, there may be situations where parts of the scene change rapidly while others change slowly; for instance, some objects to be detected in the scene may be moving quickly, while others are moving slowly or are stationary. In such situations, DVS may have captured too many redundant DVS pixels from fast-moving objects but failed to capture enough DVS pixels from slow-moving or stationary objects. Consequently, there may be some difficulty in obtaining satisfactory images or frames for both fast and slow objects.

[0006] Therefore, a method or system is needed to acquire satisfactory DVS images or frames for all objects to be detected when objects within a scene have different movement speeds. means of solving the problem

[0007] According to one of the embodiments of the present disclosure, a method for generating a frame for a DVS is provided, the method comprising: receiving DVS pixels of an object from the DVS; initializing a timespan to a timespan value; generating a frame by integrating DVS pixels captured within the timespan; evaluating the movement speed of an object; and combining a plurality of consecutive frames into a single combined frame when the evaluated movement speed is lower than a first threshold value.

[0008] According to one embodiment of the present invention, a system for generating a DVS frame is provided, the system comprises a DVS configured to capture a DVS pixel of an object, and a processor configured to receive a DVS pixel from the DVS and perform the aforementioned method.

[0009] According to one of the embodiments of the present disclosure, a computer-readable storage medium for storing a computer program is provided, and the computer program can be executed by a processor to perform the previously mentioned method.

[0010] Other systems, methods, features, and advantages of the present invention will be or will be obvious to a person skilled in the art upon reviewing the following drawings and detailed description. All such additional systems, methods, features, and advantages are incorporated within this description, are within the scope of this disclosure, and are intended to be protected by the following claims. Brief explanation of the drawing

[0011] The present invention may be better understood by referring to the following drawings and description. The components of the drawings are not necessarily in scale, but rather focus on explaining the principles of the invention. Additionally, the same reference numerals in the drawings indicate corresponding parts in different drawings. FIGS. 1a-1c illustrates schematic diagrams of images or frames of objects having different movement speeds, wherein the object (100) in FIG. 1a moves at a moderate speed in the direction indicated by the arrow, the object (100) in FIG. 1b moves very fast in the direction indicated by the arrow, and the object (100) in FIG. 1c moves very slowly or is substantially stationary. FIGS. 2a-2b and 3a-3b are images or frames showing person detection in a water drinking scenario, FIGS. 2a-2b shows an image or frame when a person (200) lifts a cup to drink water, and FIGS. 3a-3b shows an image or frame when a person (200) finishes drinking water and puts the cup down on the table. Figure 4 illustrates an example of combining continuous DVS frames into a single combined frame. FIG. 5 illustrates a flowchart of a method according to some embodiments of the present invention. FIG. 6 illustrates an exemplary process of step (S05) of FIG. 5 according to some embodiment of the present invention. FIG. 7 illustrates an exemplary process of step (S53) of FIG. 6 according to some embodiment of the present invention. FIG. 8 illustrates a flowchart of a method according to some other embodiment of the present invention. Specific details for implementing the invention

[0012] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

[0013] As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise. As used herein, the terms “include,” “including,” “include,” and / or “including” specify the presence of the specified features, integers, steps, actions, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, actions, elements, components, and / or groups thereof. As used herein, the terms “and / or” and the symbol “ / ” mean including any one or more of the associated listed items and any combination thereof. Additionally, while terms such as first, second, etc., may be used herein to describe various elements, components, steps, or calculations, these elements, components, steps, or calculations should not be limited by these terms; rather, these terms are used only to distinguish one element, component, step, or calculation from another. For example, within the scope of the present disclosure, the first component may be named the second component, and likewise, the first calculation may be named the second calculation, and likewise, the first step may be named the second step.

[0014] As used herein, the terms “DVS pixel,” “pixel event,” and “event” may be used interchangeably and refer to a change in luminosity captured by the DVS. A DVS pixel or pixel event or event may generally be represented as pixel or event data taking the form [x, y, t, p], where x and y represent the x-coordinate and y-coordinate, t represents the timestamp of the event, and p represents the polarity indicating whether the change in luminosity becomes brighter or darker. The terms “image,” “frame,” or “DVS frame” of the DVS refer to an image or frame in which all pixels taken by the DVS within a timespan are integrated. The term “combined frame” refers to a frame created by combining several consecutive DVS frames.

[0015] In order to clarify the use of the claims in the application and thereby provide notice to the public, " , , ... and <n>" or "< / n> , , ... <n>Unless otherwise clearly expressed by the applicant, the phrase “, or a combination thereof” is defined by the applicant in the broadest sense to mean one or more elements selected from the group consisting of A, B, ... and N, that is, any combination of one or more of elements A, B, ... or N, such as any one element alone or in combination with one or more other elements that may include additional elements not listed.

[0016] DVS captures only changes in luminosity and then generates asynchronous DVS pixels (pixel events). To generate an image or frame for viewing or further processing, such as object recognition or motion detection, it is necessary to integrate the DVS pixels (pixel events) captured by DVS within a timespan into a single image or frame.

[0017] Since DVS captures only changes in luminosity, the number of DVS pixels captured by DVS within a time span can vary depending on the rate of change in the scene, such as the movement speed of the object to be recognized or detected in the scene. For instance, if the object to be recognized or detected in the scene is substantially stationary or moving very slowly, DVS may capture only a very small number of the object's DVS pixels (pixel events) within a specific time span, which may be insufficient to generate a satisfactory image or frame. On the other hand, if the object to be recognized or detected is moving rapidly in the scene, DVS may capture too many redundant pixels along the object's path within the time span, resulting in a blurry image or frame.

[0018] FIGS. 1a-1c illustrates schematic diagrams of images or frames of objects (100) having different movement speeds, wherein the object (100) in FIG. 1a moves at a moderate speed in the direction indicated by the arrow, the object (100) in FIG. 1b moves very fast in the direction indicated by the arrow, and the object (100) in FIG. 1c moves very slowly or is substantially stationary. In FIG. 1a, since the object (100) moves at a moderate speed, there is a sufficient event of a change in luminosity in the time span due to the movement of the object (100), so the DVS can capture sufficient DVS pixels of the object (100) in the time span. Meanwhile, since the object (100) moves only a short distance (D1) during the time span, the image or frame generated by integrating all DVS pixels in the time span can be substantially sharp and clearly show the shape (circle) of the object.

[0019] In FIG. 1b, the object (100) moves rapidly in the time span and travels a relatively large distance of D2. Because the object (100) travels a relatively large distance within the time span, the DVS can capture many duplicate pixels in the object's path of movement within the time span, so the image or frame generated by integrating all DVS pixels within the time span is blurry and does not clearly show the shape of the object (100), which is undesirable or unsatisfactory.

[0020] In FIG. 1c, the object (100) is moving slowly or is almost stationary. Due to the slow movement of the object (100), there are not many events of change in luminosity of the object (100) within the time span. Therefore, since the DVS does not sufficiently capture the DVS pixels of the object (100) within the time span, the image or frame generated by integrating all DVS pixels within the time span does not contain enough DVS pixels to clearly show the shape (circle) of the object shown as a dotted circle in FIG. 1c.

[0021] Therefore, according to FIG. 1a-1c, in order to obtain a satisfactory image or frame, the time span used to integrate the DVS pixels must match the movement speed of the object.

[0022] FIGS. 2a-2b and 3a-3b are images or frames showing person detection in a water drinking scenario, FIGS. 2a-2b shows an image or frame when a person (200) lifts a cup to drink water, and FIGS. 3a-3b shows an image or frame when a person (200) finishes drinking water and puts the cup down on the table.

[0023] At the moment shown in FIGS. 2a-2b, that is, when a person (200) lifts a cup (212) to drink water, the cup (212) and parts of the person (200), such as the arm (202) and head (204), move at a relatively fast speed. The image or frame shown in FIG. 2a is generated by integrating DVS pixels captured by the DVS in a first time span, and the image or frame shown in FIG. 2b is generated by integrating DVS pixels captured by the DVS in a second time span, and the first time span is longer than the second time span. Because the cup (212), head (204), and arm (202) move relatively fast and the first time span is relatively long, the cup (212), head (204), and arm (202) travel a relatively large distance in the first time span. Therefore, since the DVS can capture many redundant pixels in the movement paths of the cup (212), head (204), and arm (202) in the first time span, the image or frame generated by integrating the DVS pixels in the first time span may be blurry, which is undesirable or unsatisfactory. Additionally, among the cup (212), head (204), and arm (202), the cup (212) can move the fastest, so the cup is the blurriest and least recognizable in FIG. 2a. For comparison, a second time span smaller than the first time span is used to integrate the DVS pixels into the image or frame of FIG. 2b. As shown in FIG. 2b, the cup (212), head (204), and arm (202) are much less blurry and more recognizable than shown in FIG. 2a. Therefore, for the moment shown in FIG. 2a-2b, that is, when a person (200) lifts a cup (212) to drink water, a relatively short time span is more appropriate due to the relatively fast movement speed of the cup (212), head (204), and arm (202), and an image or frame generated by using a relatively short second time span can be much more recognizable and satisfactory.

[0024] At the moment shown in FIGS. 3a-3b, that is, when the person (200) has finished drinking water and is placing the cup down on the table, the cup (212) and the person (200), such as his arm (202) and his head (204), are moving at a relatively slow speed. The image or frame shown in FIG. 3a is generated by using the same time span as the first time span of FIG. 2a, and the image or frame shown in FIG. 3b is generated by using the same time span as the second time span of FIG. 2b. Although the cup (212) and the person (200), such as his arm (202) and his head (204), are moving at a relatively slow speed, a sufficient event of light intensity change still exists due to the relatively long first time span. Meanwhile, since the cup (212) and the person (200) move only a short distance during the first time span due to the relatively slow speed, the image or frame generated by integrating all DVS pixels in the first time span can clearly show the shape of the cup (212) and the person (200). In comparison, there are not many events of light intensity change due to the relatively slow movement speed and the relatively short second time span. Therefore, in FIG. 3b, there are not enough DVS pixels to clearly show the shape of the cup (122) and the person (200), which is undesirable or unsatisfactory.

[0025] Accordingly, according to Figures 2a-2b and 3a-3b, in order to obtain a satisfactory frame during the entire drinking process, a relatively long time span may be used when the cup and the person move at a relatively slow speed, and a relatively short time span may be used when the cup and the person move at a relatively fast speed.

[0026] The inventors of the present invention discovered that, to the human eye, pixels appear continuous for a stable object in a series of DVS frames with a short time span, as shown in FIG. 3b, but in reality, the pixels are discontinuous. By analyzing the pixels of a stable or slow object in a series of DVS frames with a short time span, it was discovered that among the series of DVS frames, if one of the frames has a DVS pixel at a specific coordinate position, the probability of another DVS pixel at the same coordinate position in neighboring frames or frames is very small. That is, the DVS pixels of a series of DVS frames are generally located at different coordinate positions. Although two or more series of DVS frames may appear very similar, each series of DVS frames mainly contains DVS pixels located at different coordinate positions. Therefore, it is possible to combine a series of DVS frames containing insufficient information or DVS pixels so that the DVS pixels included in the series of DVS frames complement each other to provide a combined frame containing sufficient information or DVS pixels.

[0027] FIG. 4 illustrates an example of combining consecutive DVS frames into a single combined frame. The frames (402, 404, 406) illustrated in FIG. 4 are consecutive DVS frames similar to the frame illustrated in FIG. 3b. The inventors of the present invention have discovered that although the frames (402, 404, 406) appear very similar to each other, when one of the consecutive frames (402, 404, 406) has a DVS pixel at one specific coordinate position, the probability that the other two of the frames (402, 404, 406) will have another DVS pixel at the same coordinate position is very small. That is, although frames (402, 404, 406) appear similar to each other, the coordinate position of a DVS pixel in one of frames (402, 404, 406) may be substantially or almost completely different from the coordinate position in the other two of frames (402, 404, 406), which may be unexpected and counterintuitive. Therefore, when frames (402, 404, 406) are combined into a single combined frame (400), the DVS pixels of each frame (402, 404, 406) may be complementary to each other, resulting in a satisfactory combined frame having sufficient information or DVS pixels.

[0028] The present disclosure provides a method and system for generating a frame for a DVS. The method of the present disclosure includes the steps of receiving DVS pixels of an object from a DVS, initializing a timespan to a small timespan value, generating a frame by integrating DVS pixels captured within the timespan, evaluating the movement speed of the object, and combining a plurality of consecutive frames into a single combined frame when the evaluated movement speed is lower than a first threshold value. Since the timespan is set to a small value, the object moves a short distance, so there are not many duplicate pixels of the moving object that could blur the frame even if the object moves at a relatively fast speed. Meanwhile, in the method of the present invention, when the movement speed of the object is slow, a plurality of consecutive frames are combined into a single frame. By combining a plurality of consecutive frames into a single combined frame, there will be enough DVS pixels in the combined frame to clearly show the shape of the object even when the object is moving very slowly or is substantially stationary.

[0029] FIG. 5 illustrates a flowchart of a method according to some embodiment of the present disclosure. The process starts at (S01) and then proceeds to (S02). At (S02), a relatively small time span is set. A relatively small time span is set to prevent the frame from becoming blurred due to duplicate DVS pixels even when the object's movement speed is relatively large. Then the process proceeds to (S03), where a frame is generated by integrating DVS pixels captured by the DVS within the time span. Then the process proceeds to (S04), where the object's movement speed is evaluated. If the evaluated movement speed of the object is sufficiently fast, the process returns to (S03), where the next frame is generated by integrating DVS pixels captured by the DVS within the next time span. If the evaluated movement speed is not sufficiently fast, the process proceeds to (S05), where a plurality of consecutive frames can be combined into a single combined frame. The process then proceeds to (S06), where an evaluation is performed to determine whether the combined frame has achieved the desired effect. If the combined frame has not achieved the desired effect (No), the process returns to (S05), where more consecutive frames are combined into the combined frame. If the combined frame has already achieved the desired effect (Yes), the process returns to (S03), where the next frame is generated by integrating DVS pixels captured by the DVS within the next timespan.

[0030] The specific value of the small time span set in (S02) may vary depending on the specific application scenario and may have any appropriate value as long as the time span is small enough to avoid blurry frames due to duplicate DVS pixels even when the movement speed of the object in the scenario is relatively large. In some embodiments of the present invention, the small time span set in (S02) may be 500 microseconds or less, 1 millisecond or less, 2 milliseconds or less, 3 milliseconds or less, or 4 milliseconds or less.

[0031] The evaluation of the movement speed of an object in (S04) may be performed in various ways. In some embodiments of the present invention, the movement speed of an object may be evaluated based on the number or density of DVS pixels within a frame generated in (S03). If the number or density of DVS pixels in part or all of the frame exceeds a specified value, the movement speed of the object may be evaluated as fast. Otherwise, the movement speed of the object may be evaluated as stable or slow. In some other embodiments of the present disclosure, the movement speed of an object may be detected or evaluated by a suitable speed sensor. In other embodiments of the present invention, the evaluation or estimation of the movement speed of an object may be performed using a deep learning algorithm or model. Since deep learning algorithms or models are known in the art, a detailed description is omitted.

[0032] According to some embodiments of the present invention, in (S05), two, three, four, or five consecutive frames may be combined into one combined frame. However, the present invention is not limited thereto, and the number of consecutive frames combined into one frame in (S05) may be any suitable number. In some other embodiments, the number of consecutive frames combined into one frame in (S05) may vary depending on the evaluation result from (S04). For example, when the movement speed evaluated in (S04) is very low, a relatively large number of consecutive frames, e.g., four or five or more consecutive frames, may be combined into one frame in (S05). Otherwise, a relatively small number of consecutive frames, e.g., two or more consecutive frames, may be combined into one frame in (S05).

[0033] Evaluation of whether the combined frame in (S06) has achieved the desired effect may be performed in various ways. In some embodiments of the present disclosure, the evaluation may be performed based on the number or density of DVS pixels within the combined frame generated in (S05). If the number or density of DVS pixels in part or all of the combined frame exceeds a specified value, it is determined that the desired effect has been achieved. Otherwise, it is determined that the desired effect has not been achieved. In some other embodiments, regions having high-density DVS pixels may be identified, and the evaluation may be performed based on the proportion of high-density DVS pixel regions within the frame. For example, in some embodiments, the frame may be divided into a plurality of regions, e.g., 100 regions, and each region having more than a specified number of DVS pixels may be identified as a high-density DVS pixel region, and the evaluation may be performed based on the number of high-density DVS pixel regions. For example, if the number of high-density DVS pixel regions within the combined frame exceeds a specified number, it is determined that the desired effect has been achieved. Otherwise, it is determined that the desired effect has not been achieved. In some other embodiments of the present invention, an evaluation of whether the combined frame has achieved the desired effect may be performed using a deep learning algorithm or model. A deep learning algorithm or model may be used to detect the desired object. If detection is successful, that is, if the deep learning algorithm or model can recognize the desired object, it is determined that the desired effect has been achieved. Otherwise, it is determined that the desired effect has not been achieved.

[0034] FIG. 6 illustrates an exemplary process of step (S05) of FIG. 5 according to some embodiment of the present disclosure. The process begins at (S51) and proceeds to (S52), where the output frame is initialized by setting all pixels of the output frame to a background color (gray level). The process then proceeds to (S53), where one of a plurality of consecutive frames is combined into an output frame. Then, the process proceeds to (S54), where it is determined whether all of the plurality of consecutive frames have been combined into an output frame. If it is determined that all of the plurality of consecutive frames have been combined into an output frame, the process proceeds to (S55), where the output frame is output as the combined frame. Otherwise, the process returns to (S53), where the next frame among the plurality of consecutive frames is combined into an output frame.

[0035] FIG. 6 illustrates an exemplary process of step (S05) of FIG. 5, but the invention is not limited thereto. For example, in some other embodiments, one of a plurality of consecutive frames may be set as an output frame and each of the remaining frames among the plurality of consecutive frames may be combined as an output frame.

[0036] FIG. 7 illustrates an exemplary process of step (S53) of FIG. 6 according to some embodiments of the present disclosure. The process starts at (S531) and proceeds to (S532), where one DVS pixel from the current frame is combined into an output frame. Then the process proceeds to (S533), where it is determined whether all DVS pixels of the current frame have been combined into an output frame. If it is determined that all DVS pixels of the current frame have been combined into an output frame, the process terminates. Otherwise, the process returns to (S532), where the next DVS pixel in the current frame is combined into an output frame.

[0037] In the embodiment illustrated in FIG. 6, at (S53), the entire frame of a plurality of consecutive frames is combined into an output frame. However, the invention is not limited thereto. In some other embodiments, only a portion of the frames is combined into a combined frame. This is particularly beneficial and advantageous in situations where some objects to be detected may be moving quickly and some other objects to be detected may be moving slowly or even stationary in the scene. In this case, a portion of the frame containing the slow object may be combined into a combined frame, but a portion of the frame containing the fast object may not be combined into a combined frame. That is, for the portion of the frame containing the slow object, more consecutive frames are combined into a combined frame, and for the portion of the frame containing the fast object(s), fewer consecutive frames are combined into a combined frame. Thus, satisfactory frames can be obtained for both the fast object and the slow object.

[0038] In some embodiments of the present invention, a DVS frame or combined frame may be visualized in grayscale, and each pixel of the DVS frame or combined frame has multiple gray levels. Generally, each pixel of the grayscale may have 256 gray levels, namely gray level 0 to gray level 255, where gray level 0 is white, gray level 255 is black, and gray levels 1–254 are gray levels between white and black. However, the present invention is not limited thereto, and the DVS frame or combined frame of the present disclosure may be a grayscale having any suitable gray levels. In some embodiments of the present disclosure, the background color is an intermediate gray level between gray level 0 and gray level 255. In some embodiments of the present disclosure, the background color may be gray level 128. In some embodiments of the present invention, when p of [x, y, t, p] of a DVS pixel indicates that the luminance is darkening, the gray level of the corresponding DVS pixel within the frame may be set to gray level 255, and when p of [x, y, t, p] of a DVS pixel indicates that the luminance is brightening, the gray level of the corresponding DVS pixel within the frame may be set to gray level 0. In these embodiments, any pixel within the DVS frame or combined frame may be one of three possible gray levels, namely gray level 0, gray level 128, and gray level 255. In other embodiments of the present invention, as long as the three possible gray levels are easily distinguishable from one another, the three possible gray levels may be gray levels other than gray level 0, gray level 128, and gray level 255.

[0039] In some embodiments of the present invention, the step (S532) illustrated in FIG. 7 comprises copying the gray level of the current DVS pixel to the corresponding pixel in the output frame (combined frame), that is, replacing the gray level of the pixel in the output frame (combined frame) with the gray level of the corresponding DVS pixel in the current frame. In these embodiments, the gray level of the DVS pixel in the combined frame is independent of the sequential position of the source frame in the consecutive frames. In some other embodiments of the present invention, the gray level of the DVS pixel in the combined frame may vary depending on the sequential position of the source frame in the consecutive DVS frames. The later the source frame is in the consecutive DVS frames, the further the gray level of the resulting DVS pixel in the combined frame is from the background color (gray level). For example, when consecutive DVS frames include three frames, namely Frame 1, Frame 2, and Frame 3, if the source DVS pixel is in Frame 3, the resulting DVS pixel will have a gray level furthest from the background color, and if the source DVS pixel is in Frame 1, the resulting DVS pixel will have a gray level closest to the background color.

[0040] FIG. 8 illustrates a flowchart of a method according to some other embodiment of the present invention. The method illustrated in FIG. 8 is similar to that illustrated in FIG. 5, except for steps (S04' and S07). In (S04'), the movement speed of an object is evaluated. If the evaluated movement speed of the object is too fast, the process proceeds to (S07), where the time span is shortened, for example, set to half of the current value. Then, the process returns to (S03), where the next frame is generated by integrating DVS pixels captured by DVS within the shortened time span, or the current frame is played back by integrating DVS pixels captured by DVS within the shortened time span. If the evaluated movement speed is too slow, the process proceeds to (S05), where a plurality of consecutive frames can be combined into a single combined frame. If the evaluated movement speed is normal or an intermediate value, the process returns to (S03), where the next frame is generated by integrating the DVS pixels captured by the DVS within the time span. Other steps of the method illustrated in FIG. 8 are similar to those illustrated in FIG. 5, so a detailed description thereof is omitted.

[0041] According to some embodiments of the present disclosure, the present disclosure may be implemented as follows.

[0042] Item 1: A method for generating a frame for DVS, comprising the following:

[0043] Step of receiving the object's DVS pixels from the DVS,

[0044] Step of initializing the timespan to the timespan value,

[0045] A step of generating a frame by integrating DVS pixels captured within a timespan,

[0046] A step of evaluating the movement speed of the above object,

[0047] A step of combining multiple consecutive frames into a single combined frame when the evaluated movement speed is lower than the first threshold.

[0048] Item 2: A method for generating a frame according to Item 1, wherein the step of combining a plurality of consecutive frames into a single combined frame includes the step of combining each of the plurality of consecutive frames into a combined frame one by one.

[0049] Item 3: A method for generating a frame, wherein, in any one of Items 1-2, the step of combining a frame into a single combined frame includes the step of combining each DVS pixel within the frame into a combined frame.

[0050] Item 4: A method for generating a frame, comprising, in any one of Items 1-3, the step of initializing each pixel within the combined frame to a background color.

[0051] Item 5: A method for generating a frame, wherein in any one of Items 1-4, combining DVS pixels into a combined frame comprises copying the gray levels of the DVS pixels to corresponding pixels within the combined frame.

[0052] Item 6: A method for generating a frame, wherein, in any one of Items 1-5, the movement speed is evaluated based on the number or density of DVS pixels within at least a portion of the frame.

[0053] Item 7: A method for generating a frame, wherein, in any one of Items 1-6, the movement speed is evaluated based on the ratio of high-density DVS pixel areas within the frame.

[0054] Item 8: A method for generating a frame, wherein in any one of Items 1-7, the step of combining a plurality of consecutive frames into one combined frame comprises combining consecutive frames into one combined frame until the number or density of pixels in at least a portion of the combined frame is equal to a threshold value.

[0055] Item 9: A method for generating a frame, wherein, in any one of Items 1-8, the method further comprises the step of resetting the timespan to a smaller timespan value when the evaluated movement speed is higher than the second threshold that is higher than the first threshold.

[0056] Item 10: A method for generating a frame, wherein, in any one of Items 1-9, the step of combining a plurality of consecutive frames into a single frame comprises the step of combining a portion of at least one frame among the plurality of consecutive frames into the combined frame.

[0057] Item 11: A system for generating DVS frames, comprising:

[0058] DVS configured to capture the object's DVS pixels,

[0059] A processor configured to perform the method of any one of items 1-10.

[0060] Item 12: A computer-readable storage medium that stores a computer program that can be executed by a processor to perform the method of any one of items 1-10.

[0061] Aspects of the present invention may take the form of an entire hardware embodiment, an entire software embodiment (including, e.g., firmware, resident software, micro-code, etc.), or an embodiment combining software and hardware embodiments, all of which may generally be referred to as "circuit," "module," or "system" in this specification.

[0062] The present disclosure may be a system, a method, and / or a computer program product. A computer program product may include a computer-readable storage medium (or medium) having computer-readable program instructions for a processor to perform an aspect of the present disclosure.

[0063] A computer-readable storage medium may be a type of device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof, but is not limited thereto. A non-limiting list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, RAM (Random Access Memory), ROM (Read-Only Memory), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable CD-ROM (compact disc read-only memory), DVD (digital versatile disk), memory sticks, floppy disks, mechanically encoded devices, such as punched cards or grooved structures with instructions written thereon, and any suitable combination of the foregoings. As used herein, a computer-readable storage medium is not interpreted as a transient signal itself, e.g., radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission medium (e.g., optical pulses passing through a fiber optic cable), or an electrical signal transmitted through a wire.

[0064] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine that produces a means for implementing a function / operation specified in a flowchart and / or a block or block diagram of a block diagram, such that the instructions executed through the processor of the computer or other programmable data processing device produce a machine. Since these computer-readable program instructions may also be stored in a computer-readable storage medium capable of instructing a computer, a programmable data processing device, and / or other device to function in a specific manner, a computer-readable storage medium having the stored instructions comprises a manufactured article containing instructions that implement a function / operation specified in a flowchart and / or an aspect of a block or block diagram of a block diagram.

[0065] The flowcharts and block diagrams in the drawings illustrate the architecture, function, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block of a flowchart or block diagram may represent a module, segment, or part of an instruction containing one or more executable instructions for implementing a specified logical function(s). In some alternative implementations, the functions mentioned in the blocks may occur out of the order mentioned in the drawings. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or blocks may be executed in reverse order from time to time depending on the related functions. Each block of the block diagram and / or flowchart, and combinations of blocks of the block diagram and / or flowchart, may be implemented by a special-purpose hardware-based system that performs a specified function or operation or performs a combination of special-purpose hardware and computer instructions.

[0066] Systems and methods have been described in general terms to aid in understanding the details of the disclosure. In some examples, well-known structures, materials, and / or behaviors have not been specifically shown or described in detail to avoid obscuring aspects of the disclosure. In other cases, specific details have been provided to provide a complete understanding of the disclosure. Those skilled in the art will recognize that the disclosure may be embodied in other specific forms to adapt, for example, to a particular system or device or situation or material or component, without departing from its spirit or essential nature. Accordingly, the disclosure and description in this specification are intended to explain, not to limit, the scope of the disclosure. Accordingly, the disclosure is not limited in light of the appended claims and their equivalents.< / n>

Claims

Claim 1 A method for generating a frame for a DVS, wherein the method is performed by a processor, and the method comprises the steps of receiving DVS pixels of an object from the DVS, initializing a time span to a time span value, generating a frame by integrating DVS pixels captured within the time span, evaluating the movement speed of the object, and combining a plurality of consecutive frames into a single combined frame when the evaluated movement speed is lower than a first threshold. Claim 2 A method for generating a frame according to claim 1, wherein the step of combining a plurality of consecutive frames into a single combined frame includes the step of combining each of the plurality of consecutive frames into a combined frame one by one. Claim 3 A method for generating a frame according to paragraph 2, wherein the step of combining a frame into a single combined frame includes the step of combining each DVS pixel within the frame into a combined frame. Claim 4 A method for generating a frame according to paragraph 3, further comprising the step of initializing each pixel within the combined frame to a background color. Claim 5 A method for generating a frame according to claim 4, wherein combining DVS pixels into a combined frame includes copying the gray level of the DVS pixels to a corresponding pixel within the combined frame. Claim 6 A method for generating a frame, wherein, in any one of claims 1 to 5, the movement speed is evaluated based on the number or density of DVS pixels within at least a portion of the frame. Claim 7 A method for generating a frame, wherein, in any one of claims 1 to 5, the movement speed is evaluated based on the ratio of high-density DVS pixel areas within the frame. Claim 8 A method for generating a frame according to any one of claims 1 to 5, wherein the step of combining a plurality of consecutive frames into one combined frame comprises the step of combining consecutive frames into one combined frame until the number or density of pixels in at least a portion of the combined frame is equal to a threshold value. Claim 9 A method for generating a frame, comprising further including the step of resetting the timespan to a smaller timespan value when the evaluated movement speed is higher than the second threshold value which is higher than the first threshold value in any one of claims 1 to 5. Claim 10 A method for generating a frame, wherein, in any one of claims 1 to 5, the step of combining a plurality of consecutive frames into a single frame includes the step of combining a portion of at least one frame among the plurality of consecutive frames into the combined frame. Claim 11 A system for generating a DVS frame, comprising a DVS configured to capture DVS pixels of an object, and a processor configured to perform the method of any one of claims 1 to 5. Claim 12 A computer-readable storage medium storing a computer program that can be executed by the processor to perform the method of any one of claims 1 to 5.

Images