Method and system for generating DVS frames

Abstract

A method and system for generating a frame for a DVS is disclosed. The method includes receiving DVS pixels of an object from the DVS, initializing a time span to a time span value, generating a frame by integrating the DVS pixels captured within the time span, evaluating a movement speed of the object, and combining multiple consecutive frames into one combined frame if the evaluated movement speed is lower than a first threshold. The system includes a DVS configured to capture DVS pixels of the object, and a processor configured to receive the DVS pixels from the DVS and perform the method.

Description

[Technical Field]

[0001] The present disclosure generally relates to methods and systems for generating DVS frames. [Background technology]

[0002] There is a new cutting edge sensor called DVS (Dynamic Vision Sensor). DVS only captures changes in light intensity and creates asynchronous DVS pixels (pixel events). Compared to traditional cameras, DVS has the advantages of low latency, no motion blur, high dynamic range, and low power consumption.

[0003] Unlike conventional cameras, which create complete images or frames containing contiguous pixels, DVSs create only asynchronous DVS pixels, which may be spatially and temporally discrete and discontinuous. Therefore, DVS pixels (pixel events) captured by a DVS must be integrated within a time span to create an image or frame. Because a DVS captures only changes in light intensity, if the scene being captured by the DVS is stable, such as when the object being detected is essentially stationary or moving slowly within the scene, it may not be possible to capture enough pixels for a satisfactory image or frame in a given time span. On the other hand, if the scene is rapidly changing, such as when the object being detected is moving quickly within the scene, the DVS may capture excessive redundant DVS pixels in a given time span.

[0004] Therefore, there is a need for a method or system that can obtain satisfactory images or frames for DVS regardless of the speed at which the detected object is moving.

[0005] Furthermore, there may be situations where some parts of a scene change rapidly and other parts change slowly. For example, some objects to be detected may move quickly in the scene, while other objects to be detected may move slowly or may be stationary. In this situation, the DVS may have captured too many redundant DVS pixels from fast-moving objects but not enough DVS pixels from slow-moving or stationary objects. Therefore, there may be some difficulty in obtaining satisfactory images or frames for both fast and slow objects.

[0006] Therefore, there is a need for a method or system that can obtain satisfactory DVS images or frames for all detected objects when the objects in the scene are moving at different speeds. Summary of the Invention [Means for solving the problem]

[0007] According to one aspect of the present disclosure, there is provided a method for generating a frame for a DVS, the method including: receiving DVS pixels of an object from the DVS, initializing a time span to a time span value, generating a frame by integrating the DVS pixels captured within the time span, evaluating a movement speed of the object, and combining multiple consecutive frames into one combined frame if the evaluated movement speed is lower than a first threshold.

[0008] According to one aspect of the present disclosure, there is provided a system for generating DVS frames, the system including a DVS configured to capture DVS pixels of an object, and a processor configured to receive the DVS pixels from the DVS and perform the method described above.

[0009] According to one aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program, which can be executed by a processor to perform the above-described method.

[0010] Other systems, methods, features, and advantages of the present disclosure will be or become apparent to one with skill in the art upon examination of the following figures and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the following claims. The present specification also provides, for example, the following items: (Item 1) 1. A method for generating a frame for DVS, comprising: receiving DVS pixels of the object from the DVS; initializing a timespan to a timespan value; generating a frame by integrating the captured DVS pixels within a time span; assessing a speed at which the object is moving; If the estimated moving speed is lower than a first threshold, combining the consecutive frames into one combined frame; The method comprising: (Item 2) 2. The method for generating frames according to claim 1, wherein combining a plurality of consecutive frames into one combined frame comprises combining each of the plurality of consecutive frames into the combined frame one by one. (Item 3) 3. The method for generating frames according to claim 2, wherein combining frames into one combined frame includes combining each of the DVS pixels in the frames into the combined frame. (Item 4) 4. The method for generating a frame according to item 3, further comprising initializing each pixel in the combined frame to a background color. (Item 5) 5. The method for generating a frame according to item 4, wherein combining DVS pixels into the combined frame includes copying the gray levels of the DVS pixels into corresponding pixels in the combined frame. (Item 6) 10. A method of generating a frame according to any preceding item, wherein the speed of movement is assessed based on the number or density of DVS pixels in at least a portion of the frame. (Item 7) 10. A method of generating a frame according to any preceding item, wherein the rate of movement is estimated based on a proportion of high density DVS pixel areas within the frame. (Item 8) 10. The method of generating frames of any of the preceding items, wherein combining a plurality of consecutive frames into a single combined frame comprises combining consecutive frames into a single combined frame until a number or density of pixels in at least a portion of the combined frame is greater than or equal to a threshold. (Item 9) 2. The method of generating frames according to any preceding item, further comprising resetting the time span to a smaller time span value if the evaluated movement speed is higher than a second threshold that is higher than the first threshold. (Item 10) 10. The method of generating frames according to any of the preceding items, wherein combining a plurality of consecutive frames into one frame includes combining a portion of at least one frame of the plurality of consecutive frames into the combined frame. (Item 11) 1. A system for generating a DVS frame, comprising: a DVS configured to capture DVS pixels of an object; A processor configured to perform the method of any of the preceding items; The system comprising: (Item 12) A computer-readable storage medium storing a computer program that can be executed by a processor to perform the method according to any one of items 1 to 10.

[0011] The present disclosure may be better understood with reference to the following drawings and description. The components in the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the present disclosure. Moreover, in the figures, like reference numerals indicate corresponding parts throughout the different views. [Brief explanation of the drawings]

[0012] [Figure 1] A-C show schematic images or frames of objects having different moving speeds, where the object 100 in A is moving at a moderate speed in the direction indicated by the arrow, the object 100 in B is moving very quickly in the direction indicated by the arrow, and the object 100 in C is moving very slowly or is substantially stationary. [Figure 2] Image or frame A shows person detection in a drinking water scenario, where person 200 is raising his cup to drink water. Image or frame B shows person detection in a drinking water scenario, where person 200 is raising his cup to drink water. [Figure 3] Image A shows a person detection in a drinking water scenario, where person 200 finishes drinking water and places the cup on the table. Image B shows a person detection in a drinking water scenario, where person 200 finishes drinking water and places the cup on the table. [Figure 4] An example of combining consecutive DVS frames into one combined frame is shown. [Figure 5]1 illustrates a flowchart of a method according to some embodiments of the present disclosure. [Figure 6] 6 illustrates an example process for step S05 of FIG. 5 according to some embodiments of the present disclosure. [Figure 7] 7 illustrates an exemplary process for step S53 of FIG. 6 according to some embodiments of the present disclosure. [Figure 8] 10 shows a flowchart of a method according to some other embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Preferred embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings.

[0014] As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. The terms "comprises," "comprising," "includes," and / or "including," when used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude or add to the presence of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. As used herein, the term "and / or" and the symbol " / " mean the inclusion of any and all combinations of one or more of the associated listed items. Additionally, although terms such as "first," "second," etc. may be used herein to describe various elements, components, steps, or calculations, these elements, components, steps, or calculations should not be limited by these terms; rather, these terms are used only to distinguish certain elements, components, steps, or calculations from one another. For example, a first component could be referred to as a second component, a first calculation could be referred to as a second calculation, and a first step could be referred to as a second step, all without departing from the scope of the present disclosure.

[0015] As used herein, the terms "DVS pixel," "pixel event," and "event" can be used interchangeably and refer to light intensity changes captured by a DVS. A DVS pixel or pixel event can be represented by pixel or event data, typically in the form [x,y,t,p], where x and y represent the x and y coordinates, t represents the timestamp of the event, and p represents the polarity, meaning whether the light intensity change is brightening or dimmer. The terms "DVS image" or "frame," or "DVS frame," refer to an image or frame in which all pixels acquired by a DVS within a time span are integrated. The term "combined frame" refers to a frame generated by combining several consecutive DVS frames.

[0016] For clarity of use in the pending claims and to inform the public hereby, " 、 , ...and <n> At least one of" or "< / n> 、 、... <n>The phrase "at least one of, or a combination thereof" supersedes any other implied definition, whether before or after the present application, unless expressly asserted to the contrary by the applicant, and is intended to mean one or more elements selected from the group A, B, . . . and N, i.e., any combination of one or more of the elements A, B, . . . or N, including any one element alone or in combination with one or more of the other elements, which may also include additional unlisted elements in combination.

[0017] The DVS only captures light intensity changes and then creates asynchronous DVS pixels (pixel events). To generate an image or frame for display or further processing such as object recognition or motion detection, the DVS pixels (pixel events) captured by the DVS within a time span need to be integrated into one image or frame.

[0018] Because the DVS captures only light intensity changes, the number of DVS pixels captured by the DVS within a time span may depend on the rate of change in the scene, such as the speed at which an object being recognized or detected in the scene moves. For example, if an object being recognized or detected in the scene is substantially stationary or moves very slowly, the DVS may capture only a very small number of DVS pixels (pixel events) for that object within a particular time span, which may be insufficient to generate a satisfactory image or frame. On the other hand, if the object being recognized or detected moves quickly within the scene, the DVS may capture too many redundant pixels in the object's path of movement within the time span, resulting in a blurry image or frame.

[0019] 1A-1C show schematic diagrams of images or frames of an object 100 having different moving speeds. In FIG. 1A, the object 100 moves at a moderate speed in the direction indicated by the arrow. In FIG. 1B, the object 100 moves very quickly in the direction indicated by the arrow. In FIG. 1C, the object 100 moves very slowly or is substantially stationary. In FIG. 1A, the object 100 moves at a moderate speed. Therefore, there are sufficient events of light intensity change in the time span due to the movement of the object 100, and therefore the DVS can capture sufficient DVS pixels of the object 100 within the time span. On the other hand, because the object 100 moves only a short distance D1 during the time span, the image or frame generated by integrating all DVS pixels within the time span is substantially clear and can clearly show the shape (circle) of the object.

[0020] 1B, object 100 moves at a high speed and travels a relatively long distance D2 within the time span. Because object 100 travels a relatively long distance within the time span, the DVS may capture many redundant pixels within the object's path of movement within the time span, and therefore, an image or frame generated by integrating all DVS pixels within the time span may be blurred and may not clearly depict the shape of object 100, which may be undesirable or unsatisfactory.

[0021] In Figure 1C, object 100 is moving slowly or is essentially stationary. Because the object 100 is moving slowly, there are not many light intensity change events for object 100 in a given time span. Therefore, the DVS cannot capture enough DVS pixels of object 100 within the time span, and therefore the image or frame generated by integrating all DVS pixels within the time span does not have enough DVS pixels to clearly show the shape of the object (circle), as shown by the dotted circle in Figure 1C.

[0022] Therefore, according to Figures 1A-1C, to obtain a satisfactory image or frame, the time span used to integrate the DVS pixels needs to match the object's moving speed.

[0023] 2A-2B and 3A-3B are images or frames illustrating person detection in a drinking water scenario, where FIGS. 2A-2B show images or frames when person 200 is raising a cup to drink water, and FIGS. 3A-3B show images or frames when person 200 has finished drinking water and is placing the cup on the table.

[0024] 2A-2B, i.e., when person 200 is lifting cup 212 to drink water, cup 212 and parts of person 200, such as arm 202 and head 204, are moving at a relatively fast speed. The image or frame shown in FIG. 2A is generated by integrating DVS pixels captured by the DVS over a first time span, while the image or frame shown in FIG. 2B is generated by integrating DVS pixels captured by the DVS over a second time span, the first time span being longer than the second time span. Because cup 212, head 204, and arm 202 are moving relatively fast and the first time span is relatively long, cup 212, head 204, and arm 202 travel a relatively long distance in the first time span. Therefore, the DVS may capture many redundant pixels in the movement paths of the cup 212, head 204, and arm 202 in the first time span, and therefore, the image or frame generated by integrating the DVS pixels in the first time span may be blurred and undesirable or unsatisfactory. In addition, among the cup 212, head 204, and arm 202, the cup 212 may move the fastest, so in FIG. 2A, the cup is the blurriest and least recognizable. For comparison, a second time span smaller than the first time span is used to integrate the DVS pixels into the image or frame of FIG. 2B. As shown in FIG. 2B, the cup 212, head 204, and arm 202 are much less blurred and more recognizable than those shown in FIG. 2A. Therefore, at the moment shown in Figures 2A-2B, i.e., when person 200 is raising cup 212 to drink water, because the movement speed of cup 212, head 204, and arm 202 is relatively fast, and the image or images have a relatively short time span, a frame generated using a second, relatively short time span may be much more recognizable and satisfying.

[0025] At the moment shown in Figures 3A and 3B, when person 200 finishes drinking water and places the cup on the table, cup 212 and person 200, e.g., their arm 202 and head 204, move at a relatively slow speed. The image or frame shown in Figure 3A is generated using the same time span as the first time span of Figure 2A, and the image or frame shown in Figure 3B is generated using the same time span as the second time span of Figure 2B. Although cup 212 and person 200, e.g., their arm 202 and head 204, move at a relatively slow speed, sufficient events of light intensity change still exist because the first time span is relatively long. Meanwhile, because cup 212 and person 200 move at a relatively slow speed, they only move a short distance in the first time span. Therefore, the image or frame generated by integrating all DVS pixels in the first time span can clearly show the shape of cup 212 and person 200. In comparison, due to the relatively slow movement speed and the relatively short second time span, there are not many light intensity change events, and therefore, in Figure 3B, there are not enough DVS pixels to clearly show the shapes of cup 122 and person 200, which is undesirable or unsatisfactory.

[0026] Therefore, according to Figures 2A-2B and 3A-3B, in order to obtain satisfactory frames throughout the drinking process, a relatively long time span can be used when the cup and person move at a relatively slow speed, while a relatively short time span can be used when the cup and person move at a relatively fast speed.

[0027] The inventors of the present disclosure discovered that for stable objects in consecutive DVS frames over a short time span, such as that shown in FIG. 3B, the pixels appear continuous to the human eye, but are actually discontinuous. By analyzing the pixels of stable or slow objects in consecutive DVS frames over a short time span, they found that if one consecutive frame contains a DVS pixel at a specific coordinate location, the likelihood of another DVS pixel at the same coordinate location in its adjacent frame or frames is extremely slim. That is, the DVS pixels in consecutive DVS frames are usually at different coordinate locations. Although two or more consecutive DVS frames may appear very similar, each consecutive DVS frame primarily contains DVS pixels at different coordinate locations. Therefore, consecutive DVS frames with insufficient information or DVS pixels can be combined together, so that the DVS pixels contained in the consecutive DVS frames complement each other to provide a combined frame with sufficient information or DVS pixels.

[0028] FIG. 4 shows an example of combining consecutive DVS frames into a single combined frame. Frames 402, 404, and 406 shown in FIG. 4 are consecutive DVS frames, each of which is similar to the frame shown in FIG. 3B. The inventors of the present disclosure discovered that, although frames 402, 404, and 406 are very similar to one another, if one of consecutive frames 402, 404, and 406 has a DVS pixel at a particular coordinate location, the likelihood that the other two of frames 402, 404, and 406 have another DVS pixel at the same coordinate location is very small. That is, although frames 402, 404, and 406 appear similar to one another, the coordinate location of a DVS pixel in one of frames 402, 404, and 406 may be substantially or almost completely different from that of the other two of frames 402, 404, and 406, which may seem unexpected and counterintuitive. Therefore, when frames 402, 404, 406 are combined into one combined frame 400, the DVS pixels of each of frames 402, 404, 406 can complement each other, resulting in a satisfactory combined frame with sufficient information or DVS pixels.

[0029] The present disclosure provides a method and system for generating frames for a DVS. The method includes receiving DVS pixels of an object from a DVS, initializing a time span to a short time span value, generating a frame by integrating the DVS pixels captured within the time span, evaluating the object's movement speed, and combining multiple consecutive frames into a single combined frame if the evaluated movement speed is lower than a first threshold. Because the time span is set to a small value, the object moves a short distance, and therefore, even if the object moves at a relatively fast speed, there are not many extra pixels of the moving object that may blur the frame. On the other hand, the method of the present disclosure combines multiple consecutive frames into a single frame if the object moves slowly. By combining multiple consecutive frames into a single combined frame, there are enough DVS pixels in the combined frame to clearly display the shape of the object, even if the object is moving very slowly or is substantially stationary.

[0030] FIG. 5 shows a flowchart of a method according to some embodiments of the present disclosure. The process starts at S01 and proceeds to S02. In S02, a relatively short time span is set. The relatively short time span is set to avoid blurring of the frame due to redundant DVS pixels even when the object's movement speed is relatively high. Next, the process proceeds to S03, where a frame is generated by integrating DVS pixels captured by the DVS within the time span. Next, the process proceeds to S04, where the object's movement speed is evaluated. If the evaluated object's movement speed is sufficiently fast, the process returns to S03, where the next frame is generated by integrating DVS pixels captured by the DVS within the next time span. If the evaluated movement speed is not sufficiently fast, the process proceeds to S05, where multiple consecutive frames can be combined into one combined frame. Next, the process proceeds to S06, where the combined frame is evaluated to see if it has achieved the desired effect. If the combined frame has not achieved the desired effect (No), the process returns to S05, where further consecutive frames are combined into the combined frame. If the combined frame has already reached the desired effect (yes), the process returns to S03, where the next frame is generated by integrating the DVS pixels captured by the DVS within the next time span.

[0031] The particular value of the short time span set in S02 may depend on the particular application scenario and can have any suitable value as long as the time span is small enough to avoid frame blurring due to redundant DVS pixels even if the object's moving speed is relatively fast in the scenario. In some embodiments of the present disclosure, the short time span set in S02 may be 500 microseconds or less, 1 millisecond or less, 2 milliseconds or less, 3 milliseconds or less, or 4 milliseconds or less.

[0032] The evaluation of the object's moving speed in S04 can be performed in various ways. In some embodiments of the present disclosure, the object's moving speed may be evaluated based on the number or density of DVS pixels in the frame generated in S03. If the number or density of DVS pixels in a portion of the frame or the entire frame exceeds a predetermined value, the object's moving speed may be evaluated as fast. Otherwise, the object's moving speed may be evaluated as stable or slow. In some other embodiments of the present disclosure, the object's moving speed may be detected or evaluated by an appropriate speed sensor. In some other embodiments of the present disclosure, the evaluation or estimation of the object's moving speed may be performed by using a deep learning algorithm or model. Deep learning algorithms or models are known in the art, and therefore will not be described in detail.

[0033] According to some embodiments of the present disclosure, two, three, four, or five consecutive frames may be combined into one combined frame in S05. However, the present disclosure is not limited thereto, and the number of consecutive frames combined into one frame in S05 may be any appropriate number. In some other embodiments, the number of consecutive frames combined into one frame in S05 may depend on the evaluation result from S04. For example, if the movement speed evaluated in S04 is very slow, a relatively large number of consecutive frames, for example, four or five or more consecutive frames, may be combined into one frame in S05. Otherwise, a relatively small number of consecutive frames, for example, two or three consecutive frames, may be combined into one frame in S05.

[0034] The evaluation of whether the combined frame in S06 has achieved the desired effect can be performed in various ways. In some embodiments of the present disclosure, the evaluation may be performed based on the number or density of DVS pixels in the combined frame generated in S05. If the number or density of DVS pixels in a portion of the combined frame or the entire combined frame exceeds a predetermined value, it is determined that the desired effect has been achieved. Otherwise, it is determined that the desired effect has not been achieved. In some other embodiments, regions with high-density DVS pixels may be identified, and the evaluation may be performed based on the proportion of high-density DVS pixel regions in the frame. For example, in some embodiments, the frame may be divided into multiple regions, e.g., 100 regions, and each region with more than a predetermined number of DVS pixels may be identified as a high-density DVS pixel region, and the evaluation may be performed based on the number of high-density DVS pixel regions. For example, if the number of high-density DVS pixel regions in the combined frame exceeds a predetermined number, it is determined that the desired effect has been achieved. Otherwise, it is determined that the desired effect has not been achieved. In some other embodiments of the present disclosure, the evaluation of whether the combined frame has achieved the desired effect may be performed by using a deep learning algorithm or model. The deep learning algorithm or model may be used to detect the desired object. If the detection is successful, i.e., the deep learning algorithm or model is able to recognize the desired object, then it is determined that the desired effect has been achieved; otherwise, it is determined that the desired effect has not been achieved.

[0035] 6 shows an exemplary process of step S05 of FIG. 5 according to some embodiments of the present disclosure. The process starts at S51 and then proceeds to S52, where an output frame is initialized by setting all pixels in the output frame to a background color (gray level). Next, the process proceeds to S53, where one of the multiple consecutive frames is combined into the output frame. Next, the process proceeds to S54, where it is determined whether all of the multiple consecutive frames have been combined into the output frame. If it is determined that all of the multiple consecutive frames have been combined into the output frame, the process proceeds to S55, where the output frame is output as a combined frame. Otherwise, the process returns to S53, where the next one of the multiple consecutive frames is combined into the output frame.

[0036] 6 illustrates an example of the process of step S05 in FIG. 5, but the present invention is not limited thereto. For example, in some other embodiments, one of the multiple consecutive frames can be set as an output frame, and each of the other(s) of the multiple consecutive frames can be combined into the output frame.

[0037] 7 shows an example process for step S53 of FIG. 6 according to some embodiments of the present disclosure. The process starts at S531 and then proceeds to S532, where one DVS pixel from the current frame is combined into the output frame. Next, the process proceeds to S533, where it is determined whether all DVS pixels in the current frame have been combined into the output frame. If it is determined that all DVS pixels in the current frame have been combined into the output frame, the process ends. If not, the process returns to S532, where the next one of the DVS pixels in the current frame is combined into the output frame.

[0038] In the embodiment shown in FIG. 6 , in S53, the entirety of the multiple consecutive frames are combined into the output frame. However, the present disclosure is not limited thereto. In some other embodiments, only a portion of the frames are combined into the combined frame. This is particularly useful and advantageous in situations where some of the detected objects are moving at high speed, while other portions of the detected objects are moving slowly or are stationary in the scene. In this situation, portions of the frames including slow objects may be combined into the combined frame, while portions of the frames including fast objects may not be combined into the combined frame. That is, for frame portions including slow object(s), more consecutive frames are combined into the combined frame, while for frame portions including fast object(s), fewer consecutive frames are combined into the combined frame. Therefore, satisfactory frames can be achieved for both fast and slow objects.

[0039] In some embodiments of the present disclosure, the DVS frame or combined frame can be visualized as a grayscale, with each pixel in the DVS frame or combined frame having multiple gray levels. Typically, each pixel in the grayscale can have 256 gray levels, from gray level 0 to gray level 255, with gray level 0 being white, gray level 255 being black, and gray levels 1 to 254 being gray levels between white and black. However, the present disclosure is not limited thereto, and the DVS frame or combined frame of the present disclosure may be a grayscale having any suitable gray level. In some embodiments of the present disclosure, the background color is an intermediate gray level between gray level 0 and gray level 255. In some embodiments of the present disclosure, the background color may be gray level 128. In some embodiments of the present disclosure, if p in [x, y, t, p] of a DVS pixel indicates that the light intensity is getting darker, the gray level of the corresponding DVS pixel in the frame may be set to gray level 255, and if p in [x, y, t, p] of a DVS pixel indicates that the light intensity is getting brighter, the gray level of the corresponding DVS pixel in the frame may be set to gray level 0. In such embodiments, any pixel in the DVS frame or combined frame may be one of three possible gray levels: gray level 0, gray level 128, and gray level 255. In some other embodiments of the present disclosure, the three possible gray levels may be gray levels other than gray level 0, gray level 128, and gray level 255, as long as the three possible gray levels are easily distinguishable from one another.

[0040] In some embodiments of the present disclosure, step S532 shown in FIG. 7 includes copying the gray level of the current DVS pixel to the corresponding pixel in the output frame (combined frame), i.e., replacing the gray level of the pixel in the output frame (combined frame) with the gray level of the corresponding DVS pixel in the current frame. In such embodiments, the gray level of the DVS pixel in the combined frame is independent of the sequential position of the source frame within the consecutive frames. In other embodiments of the present disclosure, the gray level of the DVS pixel in the combined frame may depend on the sequential position of the source frame within the consecutive DVS frames. The later the source frame within the consecutive DVS frames, the farther the gray level of the resulting DVS pixel in the combined frame will be from the background color (gray level). For example, if the consecutive DVS frames include three frames, i.e., frame 1, frame 2, and frame 3, if the source DVS pixel is from frame 3, the resulting DVS pixel will have the gray level farthest from the background color, and if the source DVS pixel is from frame 1, the resulting DVS pixel will have the gray level closest to the background color.

[0041] FIG. 8 shows a flowchart of a method according to some other embodiments of the present disclosure. The method shown in FIG. 8 is similar to the method shown in FIG. 5 except for steps S04′ and S07. In S04′, the moving speed of the object is evaluated. If the evaluated moving speed of the object is too fast, the process proceeds to S07 to shorten the time span, for example, by setting the time span to half the current value. Then, the process returns to S03 to generate the next frame by integrating DVS pixels captured by the DVS within the shortened time span, or the current frame is regenerated by integrating DVS pixels captured by the DVS within the shortened time span. If the evaluated moving speed is slow, the process proceeds to S05 to combine multiple consecutive frames into one combined frame. If the evaluated moving speed is a normal or medium value, the process returns to S03 to generate the next frame by integrating DVS pixels captured by the DVS within the time span. The other steps of the method shown in FIG. 8 are similar to those shown in FIG. 5, and therefore will not be described in detail.

[0042] According to some embodiments of the present disclosure, the present disclosure can be implemented as follows.

[0043] Item 1: A method for generating a frame for DVS, receiving DVS pixels of the object from the DVS; initializing a timespan to a timespan value; generating a frame by integrating the captured DVS pixels within a time span; assessing a speed at which the object is moving; If the estimated moving speed is lower than a first threshold, combining the consecutive frames into one combined frame; The method comprising:

[0044] Item 2: A method for generating a frame according to Item 1, wherein combining a plurality of consecutive frames into one combined frame includes combining each of the plurality of consecutive frames into the combined frame one by one.

[0045] Item 3: A method for generating a frame described in any of items 1 to 2, wherein combining frames into one combined frame includes combining each of the DVS pixels in the frames into the combined frame.

[0046] Item 4: A method for generating a frame according to any one of Items 1 to 3, further comprising initializing each pixel in the combined frame to a background color.

[0047] Item 5: A method for generating a frame described in any of Items 1 to 4, wherein combining DVS pixels into the combined frame includes copying the gray levels of the DVS pixels to corresponding pixels in the combined frame.

[0048] Item 6: A method for generating a frame according to any one of items 1 to 5, wherein the movement speed is evaluated based on the number or density of DVS pixels in at least a portion of the frame.

[0049] Item 7: A method for generating a frame according to any one of items 1 to 6, wherein the movement speed is evaluated based on the proportion of a high-density DVS pixel area within the frame.

[0050] Item 8: The method for generating frames according to any one of Items 1 to 7, wherein combining a plurality of consecutive frames into one combined frame comprises combining consecutive frames into one combined frame until the number or density of pixels in at least a portion of the combined frame is equal to or greater than a threshold.

[0051] Item 9: A method for generating a frame described in any of Items 1 to 8, further comprising resetting the time span to a smaller time span value when the evaluated moving speed is higher than a second threshold that is higher than the first threshold.

[0052] Item 10: A method for generating a frame according to any one of Items 1 to 9, wherein combining a plurality of consecutive frames into one frame includes combining a portion of at least one frame of the plurality of consecutive frames into the combined frame.

[0053] Item 11: A system for generating a DVS frame, a DVS configured to capture DVS pixels of an object; A processor configured to execute the method according to any one of items 1 to 10; The system comprising:

[0054] Item 12: A computer-readable storage medium storing a computer program that can be executed by a processor to perform the method described in any one of items 1 to 10.

[0055] Aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.) or an embodiment combining software and hardware aspects, which may be generally referred to herein as a "circuit," "module," or "system."

[0056] The present disclosure may be a system, a method, and / or a computer program product, which may include computer-readable storage medium(s) having computer-readable program instructions for causing a processor to perform aspects of the present disclosure.

[0057] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples of computer-readable storage media include, but are not limited to, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanical encoding devices such as punch cards or raised structures with instructions stored in grooves, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be construed as a transitory signal, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted over a wire.

[0058] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to manufacture a machine, whereby the instructions, executing via the processor of the computer or other programmable data processing apparatus, create means for performing the functions / acts specified in the flowchart and / or block diagram of the block or blocks. These computer-readable program instructions may also be stored on a computer-readable storage medium that can instruct a computer, programmable data processing apparatus, and / or other device to function in a particular manner, whereby a computer-readable storage medium storing instructions includes an article of manufacture containing instructions that implement aspects of the functions / acts specified in the flowchart and / or block diagram of the block or blocks.

[0059] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, apparatuses, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, including one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on the functionality involved. It should also be noted that each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a dedicated hardware-based system that performs the specified functions or acts, or may be executed by a combination of dedicated hardware and computer instructions.

[0060] The systems and methods have been described in general terms to facilitate a thorough understanding of the present disclosure. In some instances, well-known structures, materials, and / or operations have not been specifically shown or described in detail to avoid obscuring aspects of the present disclosure. In other instances, specific details have been described to provide a thorough understanding of the present disclosure. Those skilled in the relevant art will recognize that the present disclosure may be embodied in other specific forms, for example, to adapt to particular systems, devices, situations, materials, or components, without departing from the spirit or essential characteristics thereof. Therefore, the disclosures and descriptions herein are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Accordingly, the present disclosure should not be limited except in light of the appended claims and their equivalents.< / n>

Claims

1. 1. A method for generating a frame for a DVS, comprising: receiving DVS pixels of an object from a DVS; initializing a timespan to a timespan value; generating a frame by integrating the captured DVS pixels within a time span; assessing a speed at which the object is moving; combining a plurality of consecutive frames into one combined frame if the estimated moving speed is lower than a first threshold; A method comprising:

2. The method of generating frames according to claim 1 , wherein combining a plurality of consecutive frames into a combined frame comprises combining each of the plurality of consecutive frames one by one into the combined frame.

3. The method of generating frames of claim 2 , wherein combining frames into a combined frame includes combining each of the DVS pixels in the frames into the combined frame.

4. The method of generating a frame of claim 3 further comprising initializing each pixel in the combined frame to a background color.

5. The method of generating a frame of claim 4 , wherein combining DVS pixels into the combined frame includes copying a gray level of the DVS pixel to a corresponding pixel in the combined frame.

6. A method for generating a frame as described in any one of claims 1 to 5, wherein the movement speed is evaluated based on the number or density of DVS pixels in at least a portion of the frame, and if the number or density of DVS pixels in a portion of the frame or the entire frame exceeds a predetermined value, the movement speed of the object is evaluated to be fast.

7. 7. The method for generating frames according to claim 1, wherein combining a plurality of consecutive frames into one combined frame comprises combining consecutive frames into one combined frame until a number or density of pixels in at least a portion of the combined frame is equal to or greater than a threshold.

8. The method for generating frames according to any one of claims 1 to 7, further comprising resetting the time span to a smaller time span value when the evaluated moving speed is higher than a second threshold that is higher than the first threshold.

9. 9. The method for generating frames according to claim 1, wherein combining a plurality of consecutive frames into one frame comprises combining a portion of at least one frame of the plurality of consecutive frames into the combined frame.

10. 1. A system for generating a DVS frame, comprising: a DVS configured to capture DVS pixels of an object; a processor; a computer storage medium having a computer program stored thereon, the computer program being configured to perform the method according to any one of claims 1 to 9 when executed by the processor; Including, the system.

11. A computer-readable storage medium storing a computer program executable by a processor to perform the method according to any one of claims 1 to 9.

Images