Flicker noise filtering system

Abstract

The application discloses a flicker noise filtering system. To solve the flicker noise problem caused by the flickering light source, the flicker noise filtering scheme disclosed by the application is as follows: a first event is received, the time difference between the first event and the previous event of the first event is obtained according to the timestamp of the first event and the timestamp of the previous event of the first event; the size relationship between the time difference and a first threshold value is judged, and the result representing the size relationship is written into a first position in a first list corresponding to the first event in a first storage space; whether the first event is flicker noise is judged according to a plurality of results representing the size relationship stored in a plurality of positions of the first list corresponding to the first event in the first storage space. The application takes the list recording time difference discrimination results as a technical means, solves the flicker noise problem, and obtains the low-power-consumption, low-delay and high-performance noise reduction effect. The application is suitable for the field of event cameras and brain-like computing.

Description

[0001] This invention is a divisional application of Chinese invention patent application No. 202211345510.3 (filed on October 31, 2022, entitled "Flicker Noise Filtering Method and Apparatus, Sensor, Chip and Electronic Equipment"). All technical solutions described in that application are incorporated herein by reference. Technical Field

[0002] This invention relates to a flicker noise filtering system, specifically to a method, apparatus, sensor, chip, and electronic device for filtering flicker noise caused by a flickering light source in an event camera. Background Technology

[0003] Traditional frame image sensors suffer from drawbacks such as latency, blurring, and high power consumption, while event cameras can effectively overcome these problems, making them a frontier and hot topic in current academic research. Event cameras can be used in fields such as tracking, VR (eye tracking), obstacle avoidance, optical flow estimation, and driver state detection. They excel at capturing moving objects within the field of view, while stationary objects within the field of view are not imaged. Therefore, imaging is entirely event-driven, just as... Figure 1 As shown.

[0004] Each pixel in an event camera works independently, generating an event indicating the direction of the light change after detecting a change in light. This is its underlying working principle, which conforms to the characteristics of event-driven cameras and has the advantages of low power consumption and low latency, which is completely different from frame image sensors.

[0005] However, in environments such as homes where AC lighting is commonly used, the AC-driven light source will flicker continuously. This change in light intensity (from the light source itself and the light reflected from objects) causes the event camera to continuously generate events (called flicker noise or flicker noise events), producing unwanted false "motion" or resulting in extremely poor image quality, which in turn affects the difficulty and capability of subsequent information processing.

[0006] Figure 2 Subgraphs (a) and (b) show images taken by a low-resolution event camera in two very short time intervals, depicting a scene with a flashing billboard and pedestrians passing by. The moving pedestrians are captured, but the constantly flashing billboard is consistently imaged, which is not the desired effect. The challenge lies in the fact that this unwanted imagery perfectly aligns with the underlying logic of event cameras capturing changing light; it is perfectly normal and reasonable for the event camera itself, but detrimental to practical applications. Therefore, traditional noise reduction schemes (such as those based on spatiotemporal characteristics) are ill-suited for this type of noise.

[0007] Figure 3Sub-figures (a) and (b) show images of a person waving their hand in front of an event camera in two different environments. The strong and flickering background light source makes it difficult to capture the waving motion clearly and reliably, and the poor image quality obviously poses a serious challenge to subsequent information processing.

[0008] Prior art 1: US10248222B2;

[0009] Prior art 2: US11416759B2;

[0010] Prior art 3: US20210067679A1.

[0011] Existing technologies 1-3 are solutions for eliminating flicker noise, but these solutions require caching continuous events, counting events by attenuation, and require a large amount of storage space, complex high-precision calculations, or rely on other sensors to obtain brightness information. These solutions are usually not event-driven or require a clock, and therefore have disadvantages in terms of cost / area, power consumption, and latency.

[0012] Therefore, there is an urgent need in this field for a low-latency, low-cost, low-power, efficient, and reliable noise reduction solution for event camera flicker. Summary of the Invention

[0013] To solve or alleviate some or all of the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0014] A flicker noise filtering method involves receiving a first event, obtaining the time difference between the first event and the previous event based on the timestamp of the first event and the timestamp of the previous event; if the time difference is less than a first threshold, subtracting a non-zero constant from the first value in the first storage unit corresponding to the first event in the first storage space; otherwise, increasing the first value by a non-zero constant to obtain an updated first value; and determining whether the first event is flicker noise based on the relationship between the updated first value and a second threshold.

[0015] In this invention, the first event and the event preceding the first event both originate from the first pixel.

[0016] In one type of embodiment, the non-zero constant in the subtraction of a non-zero constant and the non-zero constant in the addition of a non-zero constant are the same, and both are equal to 1.

[0017] In one embodiment, the first event and the event preceding the first event both originate from the first pixel in the event imaging device; the first storage space includes a plurality of storage units, and there is a one-to-one correspondence between the plurality of storage units and a plurality of pixels in the pixel array of the event imaging device.

[0018] In one embodiment, the first event originates from a first pixel, and a first storage unit corresponding to the first event is determined in the first storage space based on the coordinates of the first pixel in the pixel array of the event imaging device.

[0019] In one type of embodiment, if the non-zero constant is positive, and if the updated first value is less than a second threshold, then the first event is determined to be flicker noise; or,

[0020] If the non-zero constant is negative, and if the updated first value is greater than the second threshold, then the first event is determined to be flicker noise.

[0021] In one embodiment, the storage length of the first storage unit is 3, 4, or 5 bits.

[0022] In one type of embodiment, if the first event is not judged as flicker noise, then the first event is handed over to a subsequent processing stage.

[0023] In one embodiment, the timestamp of the first event is stored in the second storage unit corresponding to the first event in the second storage space.

[0024] In one embodiment, after determining whether the first event is flickering noise or obtaining the time difference between the first event and its predecessor, the timestamp of the first event is stored in the second storage unit corresponding to the first event in the second storage space.

[0025] In one embodiment, the second storage space includes a plurality of storage units, and there is a one-to-one correspondence between the plurality of storage units and a plurality of pixels in the pixel array of the event imaging device.

[0026] In one embodiment, the first event originates from a first pixel, and a second storage unit corresponding to the first event is determined in the second storage space based on the coordinates of the first pixel in the pixel array of the event imaging device.

[0027] In one embodiment, the timestamp of the previous event of the first event is taken from the second storage unit corresponding to the first event in the second storage space.

[0028] In one embodiment, if the time difference is less than a first threshold and greater than a third threshold, then a non-zero constant is subtracted from the first value in the first storage unit corresponding to the first event in the first storage space; otherwise, a non-zero constant is added to the first value.

[0029] A flicker noise filtering method includes receiving a first event, obtaining the time difference between the first event and the previous event based on the timestamp of the first event and the timestamp of the previous event; determining the magnitude relationship between the time difference and the first threshold, and writing the result representing the magnitude relationship into a first position in a first list corresponding to the first event in a first storage space; and determining whether the first event is flicker noise based on several results representing the magnitude relationship stored in several positions in the first list corresponding to the first event in the first storage space.

[0030] In one embodiment, both the first event and the event preceding the first event originate from the first pixel.

[0031] In one embodiment, if the first list contains only the first position: if the result representing the size relationship is that the time difference is less than a first threshold, then the first event is determined to be flicker noise; otherwise, the first event is determined not to be flicker noise. Alternatively, if the first list contains at least two positions: then the first event is determined to be flicker noise based on the number of results representing the same size relationship among the results representing the size relationship most recently stored in the other positions of the first list.

[0032] In one type of embodiment, when storing results representing size relationships in several locations of a first list corresponding to a first event in a first storage space, the first-in-first-out principle is followed.

[0033] In one embodiment, the timestamp of the first event is stored in the second storage unit corresponding to the first event in the second storage space.

[0034] In one embodiment, the second storage space includes a plurality of storage units, and there is a one-to-one correspondence between the plurality of storage units and a plurality of pixels in the pixel array of the event imaging device; based on the coordinates of the first pixel in the pixel array of the event imaging device, a second storage unit in the second storage space corresponding to the first event is determined, and the timestamp of the first event is stored in the second storage unit in the second storage space corresponding to the first event; the timestamp of the previous event of the first event is taken from the second storage unit in the second storage space corresponding to the first event.

[0035] A flicker noise filtering device includes at least a first storage space, and the flicker noise filtering device performs noise filtering on a received first event based on the first storage space and the flicker noise filtering method as described in any of the preceding claims.

[0036] A sensor is an event imaging device comprising a pixel array and a first storage space, the pixel array including a first pixel that generates a first event and a previous event that generates the first event, and noise filtering of the first event based on the first storage space and a flicker noise filtering method as described in any of the preceding claims.

[0037] A chip includes an event imaging device, a processor, and a first storage space, wherein noise filtering is performed on a first event generated by the event imaging device at least according to the first storage space and a flicker noise filtering method as described in any of the preceding claims; the processor processes the event generated by the event imaging device based on the first event after at least the aforementioned noise filtering.

[0038] An electronic device having a chip as described above deployed thereon and used to process environmental signals.

[0039] In one type of embodiment, the processor is a neuromorphic processor (neuromorphic chip).

[0040] Some or all of the embodiments of the present invention have the following beneficial technical effects:

[0041] 1) Completely event-driven, the solution directly uses the event timestamps and does not require clock intervention;

[0042] 2) It does not require caching information about consecutive events to perform comparisons, nor does it require counting these events;

[0043] 3) Threshold comparison is used instead of finite state machines, which often require a large amount of computation;

[0044] 4) Low latency: For each event, the filtering operation occurs only locally and only once.

[0045] Further beneficial effects will be described in the preferred embodiments.

[0046] The technical solutions / features disclosed above are intended to summarize the technical solutions and features described in the Detailed Embodiments section, and therefore the scope of the description may not be entirely the same. However, these new technical solutions disclosed in this section are also part of the numerous technical solutions disclosed in this invention document. The technical features disclosed in this section, together with the technical features disclosed in the subsequent Detailed Embodiments section and some contents in the drawings not explicitly described in the specification, disclose more technical solutions in a reasonable combination.

[0047] The technical solution formed by combining all the technical features disclosed at any position in this invention is used to support the summary of the technical solution, the modification of the patent document, and the disclosure of the technical solution. Attached Figure Description

[0048] Figure 1 This is a schematic diagram illustrating the imaging effect of an event camera capturing dynamic images;

[0049] Figure 2 These are schematic diagrams illustrating the effects of camera captures on images at different moments when there is a flickering background light source.

[0050] Figure 3 These are schematic diagrams illustrating the effects of camera captures on images against a strong background with flickering light sources at different moments.

[0051] Figure 4 This is a schematic diagram of an embodiment of the present invention;

[0052] Figure 5 This is a flowchart of the flicker noise filtering scheme of the present invention;

[0053] Figure 6 This is a flowchart of a flicker noise filtering scheme in an alternative embodiment of the present invention;

[0054] Figure 7 These are comparison images of the imaging effects under different schemes when there is a flickering light source in the background and no moving objects;

[0055] Figure 8 These are comparison images of the imaging effects under different schemes when there is a flickering background light source and moving objects. Detailed Implementation

[0056] Since it is impossible to exhaustively describe all alternative solutions, the key points of the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Other technical solutions and details not disclosed in detail below generally belong to technical objectives or features that can be achieved by conventional means in the art, and due to space limitations, they will not be described in detail here.

[0057] Unless it refers to division, the " / " in any position in this invention represents logical "OR". The serial numbers "first", "second", etc., in any position in this invention are merely descriptive distinguishing marks and do not imply an absolute temporal or spatial order, nor do they imply that terms prefixed with such serial numbers necessarily refer to different things than the same terms prefixed with other modifiers.

[0058] This invention describes various key points used to combine into various specific embodiments, which will be incorporated into various methods and products. In this invention, even if a key point is described only when introducing a method / product solution, it means that the corresponding product / method solution also explicitly includes that technical feature.

[0059] The description of the existence or inclusion of a step, module, or feature at any location in this invention does not imply that such existence is exclusive or unique. Those skilled in the art can obtain other embodiments by supplementing the technical solutions disclosed in this invention with other technical means. The embodiments disclosed in this invention are generally for the purpose of disclosing preferred embodiments, but this does not imply that opposite embodiments of the preferred embodiments are excluded by this invention. As long as such opposite embodiments at least solve one of the technical problems of this invention, they are intended to be covered by this invention. Based on the key points described in the specific embodiments of this invention, those skilled in the art can substitute, delete, add, combine, or change the order of certain technical features to obtain a technical solution that still follows the concept of this invention. These solutions that do not depart from the technical concept of this invention are also within the protection scope of this invention.

[0060] An event camera, essentially an event-driven image sensor, is also known as a dynamic vision sensor (DVS). Based on this principle, some technical solutions fuse it with traditional frame image pixels, resulting in a sensor that can output both events and pixel brightness. Examples include the DAVIS sensor and the ATIS sensor. These event-based sensors (EBS) are collectively referred to as event imaging devices in this invention, and they belong to the category of sensors. This invention uses an event camera as an example to disclose a scheme for filtering flicker noise.

[0061] refer to Figure 4 For the first event *e* generated by the event camera, without loss of generality, it can be any event generated by the event camera originating from the first pixel. The coordinates of the first pixel in the event camera pixel array are (x, y), and the time when event *e* is generated is called the timestamp and is denoted as ts(e). An event generated before event *e* and generated by the same pixel (the first pixel) is denoted as "e-1". It is the preceding event of event *e* (or simply the preceding event). The two events originate from the same pixel and have the same coordinates, but have different timestamps. The timestamp of the preceding event *e-1* is denoted as ts(e-1). After the first pixel generates the preceding event *e-1*, events *e* are subsequently generated. The information of the event may also include the direction of the brightness change of the pixel's light sensitivity, which is called polarity.

[0062] In addition, there are two storage spaces, referred to as the first storage space and the second storage space, which store data called a Boolean map (boolMap) and an activity map (activityMap), respectively. There is a one-to-one correspondence between any storage unit in the Boolean map and the activity map and any pixel in the event camera pixel array. In other words, each pixel has a corresponding storage unit in both the first and second storage spaces. Without loss of generality, for any event e, these are referred to as the first storage unit and the second storage unit, respectively. For simplicity, the coordinates of the generated pixel of any event e are (x, y), and logically, the coordinates of its corresponding first and second storage units in the Boolean map and the activity map are both (x, y). This logical mapping relationship can be any reasonable manner, and this invention does not limit it.

[0063] The Boolean graph described above stores a count in its first storage space, which reflects the confidence level of the corresponding pixel. The larger the count value corresponding to event e, the less likely that event e is caused by a flickering light source. Preferably, the storage unit can be 3 bits long, with 1 bit being a very special case; it can also be 4 bits or 5 bits. If it is 3 bits, in one embodiment, the stored value is between -4 and 3, with an initial value of 0.

[0064] The activity graph described above stores the timestamp of the latest event generated by the corresponding pixel in its corresponding second storage space. In other words, the data in the activity graph is the timestamp of the last event emitted by each pixel in the entire pixel array. Preferably, the storage unit can be 16 bits in length. It is worth mentioning that the shorter the storage length, the smaller the required storage space. Under the premise of meeting the accuracy requirements, choosing the minimum storage length will help reduce the storage space / chip area. Therefore, the timestamp here may be a timestamp after reducing the accuracy of the pixel timestamp generated in the event camera (original timestamp accuracy), but of course, choosing to store it with the original timestamp accuracy is also possible.

[0065] The first and second storage spaces are part of the chip's storage area. The pixel array of the event camera is the photosensitive part of the chip, and the pixel circuit may include photodiodes. The chip formed by the pixel array, the first storage space, and the second storage space can simply be an event imaging device, i.e., a sensor. This sensor can also be connected to a processor via an adapter board to form a "sensor-processor integrated" chip. In this case, the first and second storage spaces can be considered part of the interface circuitry used to build noise reduction circuits / modules.

[0066] refer to Figure 5It demonstrates the flicker noise filtering module 10 of the event camera, where the device generating the event can be any event imaging device. The timestamp ts(e) of the first event e is subtracted from the timestamp ts(e-1) of the previous event e-1, and the result is compared with the first threshold θ. t Comparison: If the difference between the two timestamps is within the first threshold θ t If the value is within the first threshold (i.e., less than the first threshold, the Boolean result is "true"), then a non-zero constant is subtracted from the first value boolMAP(x, y) (essentially a confidence count) stored in the first storage unit of the Boolean graph. Otherwise (the result is "false"), a non-zero constant is added to the first value boolMAP(x, y) stored in the first storage unit of the Boolean graph. This non-zero constant can be positive or negative. If it is negative, it means that after comparing the first value with the second threshold, the logic for judging event e will be reversed. The aforementioned non-zero constant is preferably 1.

[0067] As a further preferred embodiment of the foregoing embodiments, the result obtained by the aforementioned subtraction is further compared with a third threshold θ′. t (For example, the second threshold / 2) comparison, if: the difference between the two timestamps is at the first threshold θ t Within, and greater than the third threshold θ′ t (Only then will the Boolean result be "true"). Then, for the first value boolMAP(x, y) stored in the first storage unit of the Boolean graph, subtract a non-zero constant; otherwise, add a non-zero constant.

[0068] Then, based on the first value boolMAP(x, y) stored in the first storage unit of the Boolean graph and the second threshold θ c If the first value is less than the second threshold θ, then... c If the event e is not considered to be flickering noise caused by a flickering light source, then the event e is considered not to be flickering noise, and subsequent processing is performed.

[0069] Furthermore, the terms "greater than" and "less than" in this invention are essentially logical comparisons. To obtain the same logical comparison result, the boundary values ​​can be slightly modified. However, this is merely an equivalent and conventional substitution method in the art. For example, "≥2" and ">1" are equivalent in some cases. These basic logical transformations or boundary value modifications can usually be logically altered or replaced by those skilled in the art, which also does not depart from the basic concept of this invention and remains within the protection scope of this invention.

[0070] Regardless of whether it is identified as flickering noise, the timestamp ts(e) of event e is stored in the second storage unit at coordinates (x, y) in the second storage space, and its value is denoted as activityMap(x, y). This storage operation can be performed after obtaining the time difference between the first event and its predecessor, or after determining whether the first event is flickering noise. Subsequently, when the pixel at coordinates (x, y) in the pixel array emits a new event e+1, the aforementioned activityMap(x, y) value is read and used as the timestamp ts(e) of the previous event e before the new event e+1. In other words, during the update of the activity graph, the timestamp ts(e-1) of the previous event e-1 of event e is overwritten by the timestamp ts(e) of event e. Therefore, this is why the data in the activity graph is the timestamp of the last event emitted by each pixel in the entire pixel array.

[0071] As a special case, a Boolean graph can also have a storage length of only 1 bit. Such an embodiment does not require a second threshold comparison because a Boolean graph can only store two states: 0 or 1 (a form representing the result of a size relationship). Therefore, in this embodiment, it is only necessary to compare the timestamp difference between the current event e and the previous event e-1 from the same pixel; if it is less than the second threshold θ... c If event e is considered flicker noise, then it is considered flicker noise; otherwise, it is not. The noise reduction performance of this special case scheme may not be as good as other schemes.

[0072] As an alternative embodiment, the Boolean graph stored in the first storage space can also store the aforementioned Boolean results of multiple successively generated events corresponding to the same pixel using a set of data. In other words, the aforementioned embodiment only increments / decrements the first value boolMAP(x, y) based on the aforementioned Boolean result of the latest event e, while the alternative embodiment retains the historical record of multiple Boolean results (in principle, storing the timestamps of multiple events before and after each pixel is also feasible, but it requires more storage space). Obviously, from the richer historical record, it is easy to calculate the corresponding first value and its relationship with the second threshold θ. c The size relationship. This solution can solve the flicker noise problem, but the drawback is that it requires a larger initial storage space. These special cases or alternative embodiments are still within the technical concept disclosed in this invention.

[0073] In other words, the flicker noise filtering method disclosed herein includes the following steps: receiving a first event; obtaining the time difference between the first event and its predecessor based on the timestamp of the first event and the timestamp of the event preceding the first event; determining the magnitude relationship between the time difference and a first threshold, and writing the result representing the magnitude relationship into a first position in a first list corresponding to the first event in a first storage space; and determining whether the first event is flicker noise based on several results representing the magnitude relationship stored in several positions in the first list corresponding to the first event in the first storage space.

[0074] Preferably, the first event and the event preceding the first event both originate from the first pixel.

[0075] Preferably, if only the first position exists in several positions of the first list (in this case, the first position can be a temporary position storing the result of the aforementioned determination of the size relationship): if the result representing the size relationship is that the time difference is less than a first threshold, then the first event is determined to be flicker noise; otherwise, the first event is determined not to be flicker noise. This embodiment corresponds to the aforementioned special case.

[0076] Preferably, if at least two of the positions in the first list represent the same size relationship, then the first event is determined to be flicker noise based on the number of similar size relationship results stored at the first position and the number of such results stored at other positions in the first list. The reason for selecting the most recently stored size relationship results at other positions is that the most recent triggering of the size relationship by the same pixel is more likely to reflect whether the first event e is flicker noise. For example, if the first list stores 5 values ​​representing a less than relationship and 2 values ​​representing a greater than relationship, this is equivalent to the aforementioned case where the first value equals 3.

[0077] Preferably, when storing the results representing size relationships in several locations of the first list corresponding to the first event in the first storage space, the first-in-first-out (FIFO) principle is followed. Following this principle makes it easier to find the most recently stored results representing size relationships in other locations within the scheme. For example, selecting all the size relationships stored in all locations of the first list that follow the FIFO principle will ensure that they are all the most recent size relationship results.

[0078] Preferably, the timestamp of the first event is stored in the second storage unit corresponding to the first event in the second storage space.

[0079] It should be noted that the alternative embodiments or special examples have other technical features that are the same as those described above (without obviously violating logic), and are listed herein by reference.

[0080] refer toFigure 6 This illustrates another alternative embodiment. In this embodiment, the flicker noise filtering module 10 of the event camera and Figure 5 The approach is the same as described above. The difference lies in that a step of reducing the timestamp precision is performed before executing the flicker noise filtering method. As mentioned earlier, in order to reduce the size of the second storage space, the timestamp precision is reduced by shifting, but the performance requirements are still met. For example, after shifting the timestamp to the left, a low-precision timestamp ts(e) is obtained, and then the timestamp with reduced precision is sent to the aforementioned flicker noise filtering module 10.

[0081] The above flicker noise filtering module or method can be implemented by designing corresponding integrated circuits to achieve the corresponding information processing flow, forming the final event imaging device or chip. It can also be implemented using FPGA or software methods. The physical carrier for implementing the above method is the flicker noise filtering device. It should be noted that all threshold parameters in this invention are configurable, allowing for easy adjustment according to different scenarios. Furthermore, the storage units in the first and second storage spaces can be centralized or distributed.

[0082] Event imaging devices employing the aforementioned flicker noise filtering method, or including the aforementioned noise filtering device, can exhibit good flicker noise resistance. Furthermore, electronic devices employing the aforementioned flicker noise filtering method, or including the aforementioned noise filtering device and event imaging device, can relatively easily operate under AC-powered light sources.

[0083] In other words, this discloses a flicker noise filtering device that includes at least a first storage space, and that performs noise filtering on a received first event based on the first storage space and the flicker noise filtering method as described in any of the preceding claims.

[0084] A sensor is an event imaging device comprising a pixel array and a first storage space, the pixel array including a first pixel that generates a first event and a previous event that generates the first event, and noise filtering of the first event based on the first storage space and a flicker noise filtering method as described in any of the preceding claims.

[0085] A chip includes an event imaging device, a processor, and a first storage space, wherein noise filtering is performed on a first event generated by the event imaging device at least according to the first storage space and a flicker noise filtering method as described in any of the preceding claims; the processor processes the event generated by the event imaging device based on the first event after at least the aforementioned noise filtering.

[0086] An electronic device is provided that incorporates the chip described above. This electronic device can be an edge device such as a toy or electronic lock, where the chip facilitates always-on intelligent detection.

[0087] In contrast, one existing technology uses a band stop to filter noise from the AC operating frequency, and its performance is shown in... Figure 7 Part (b) and Figure 8 Part (b)

[0088] Figure 7 The event imaging effects of different schemes are shown when there are no moving objects and only a flashing light source. Figure 7 Part (a) corresponds to the imaging effect without any noise reduction measures. It can be seen that the flickering light source and the reflected light on the object (within the ellipse in the figure) are continuously forming an image. Figure 7 Part (b) shows the denoising effect of the aforementioned comparative denoising scheme. It can be seen that the flickering light source and the reflected light on the object (within the ellipse in the figure) are still faintly visible in the background. Figure 7 Part (c) corresponds to the denoising effect of the present invention (excluding other denoising schemes). In the figure, most of the noise events caused by the background flickering light source have been eliminated, leaving only some background random noise. This random noise can be eliminated by other specialized denoising schemes.

[0089] Figure 8 The event imaging effects of different schemes are shown when there are moving objects and flashing light sources. Figure 8 The middle part (a) corresponds to the imaging effect without any noise reduction measures. It can be seen that the flickering light source and the reflected light on the object are continuously forming an image. The outline of one side of the moving office chair (ellipse in the figure) is clearly visible. Figure 8 Part (b) corresponds to the noise reduction effect of the aforementioned comparison noise reduction scheme. Apart from the outline of one side of the office chair, a large amount of flicker noise (ellipse in the figure) still exists. Figure 8 The noise reduction effect of the noise reduction scheme of the present invention corresponding to part (c) is that, except for the outline of one side of the office chair (ellipse in the figure), the flicker noise is basically eliminated, leaving only background random noise.

[0090] Although the invention has been described with reference to specific features and embodiments, various modifications, combinations, and substitutions can be made therein without departing from the invention. The scope of protection of this invention is not limited to the specific embodiments of processes, machines, manufactures, material compositions, apparatuses, methods, and steps described in the specification, and these methods and modules may also be implemented in one or more related, interdependent, cooperative, or upstream / downstream products or methods.

[0091] Therefore, the specification and drawings should be simply regarded as a description of some embodiments of the technical solutions defined by the appended claims, and thus the appended claims should be interpreted in accordance with the principle of the greatest reasonable interpretation, and are intended to cover as much as possible all modifications, variations, combinations or equivalents within the scope of the invention, while avoiding unreasonable interpretations.

[0092] To achieve better technical effects or for the needs of certain applications, those skilled in the art may make further improvements to the technical solution based on this invention. However, even if such improvements / designs are inventive and / or progressive, as long as they rely on the technical concept of this invention and cover the technical features defined in the claims, the technical solution should also fall within the protection scope of this invention.

[0093] The technical features mentioned in the appended claims may have alternative technical features, or the order of certain technical processes or material organization may be rearranged. Those skilled in the art, upon learning of this invention, will readily conceive of these alternative means, or alter the order of the technical processes or material organization, and then employ substantially the same means to solve substantially the same technical problems and achieve substantially the same technical effects. Therefore, even if the claims explicitly define the aforementioned means and / or order, these modifications, alterations, and substitutions should all fall within the scope of protection of the claims based on the principle of equivalents.

[0094] The method steps or modules described in the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application or design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered outside the scope of protection claimed by this invention.

Claims

1. A flicker noise filtering method, characterized in that: Receive the first event, and obtain the time difference between the first event and the previous event based on the timestamp of the first event and the timestamp of the previous event. Determine the relationship between the time difference and the first threshold, and write the result representing the relationship into the first position of the first list corresponding to the first event in the first storage space; Based on the results representing size relationships stored at several positions in the first list corresponding to the first event in the first storage space, determine whether the first event is flickering noise; as well as, The first event and the event preceding the first event both originate from the first pixel.

2. The flicker noise filtering method according to claim 1, characterized in that: When storing the results representing the size relationship in several locations of the first list corresponding to the first event in the first storage space, the first-in-first-out principle is followed.

3. The flicker noise filtering method according to claim 1, characterized in that: The timestamp of the first event is stored in the second storage unit corresponding to the first event in the second storage space.

4. The flicker noise filtering method according to claim 3, characterized in that: The second storage space includes a plurality of storage units, and there is a one-to-one correspondence between the plurality of storage units and a plurality of pixels in the pixel array of the event imaging device; Based on the coordinates of the first pixel in the pixel array of the event imaging device, the second storage unit corresponding to the first event in the second storage space is determined, and the timestamp of the first event is stored in the second storage unit corresponding to the first event in the second storage space. The timestamp of the previous event of the first event is taken from the second storage unit corresponding to the first event in the second storage space.

5. The flicker noise filtering method according to any one of claims 1-4, characterized in that: If several positions in the first list contain only the first position: if the result representing the size relationship indicates that the time difference is less than a first threshold, then the first event is determined to be flicker noise; otherwise, the first event is determined not to be flicker noise; or... If at least two of the positions in the first list are present, then the first event is determined to be flickering noise based on the number of results representing the same type of size relationship stored in the first position and the number of results representing the same size relationship stored in the other positions of the first list.

6. A flicker noise filtering device, characterized in that: The flicker noise filtering device includes at least a first storage space, and the flicker noise filtering device performs noise filtering on the received first event according to the first storage space and the flicker noise filtering method as described in any one of claims 1-5.

7. A sensor, the sensor being an event imaging device, the event imaging device comprising a pixel array and a first storage space, the pixel array including a first pixel that generates a first event and a previous event that generates the first event, characterized in that: The first event is noise filtered according to the first storage space and the flicker noise filtering method according to any one of claims 1-5.

8. A chip comprising an event imaging device, a processor, and a first storage space, characterized in that: According to the first storage space, and the flicker noise filtering method according to any one of claims 1-5, at least the first event generated by the event imaging device is subjected to noise filtering; The processor processes the events generated by the event imaging device based on a first event that has at least undergone the aforementioned noise filtering.

9. An electronic device, characterized in that: The electronic device is equipped with the chip as described in claim 8.

Images