High-pass filtering method and device, chip and electronic equipment

Abstract

The application discloses a high-pass filtering method and device, a chip and electronic equipment. In order to solve the problem that the prior art lacks a high-pass filtering scheme based on a pulse neuron, the application realizes the technical effect that pulse events are output only when the frequency of input pulse events is higher than a cutoff frequency through special configuration of parameters of a first pulse neuron with a leakage function, such as synaptic weights of the first pulse neuron for projecting input pulse events, the cutoff frequency of high-pass filtering, a leakage amount of the first pulse neuron, a leakage frequency of the first pulse neuron and the like. The scheme is extremely efficient when a target point is efficiently captured based on a neuromorphic sensor, and the application has the technical effects of low power consumption, low delay and low calculation amount. The application is suitable for the field of neuromorphic computing.

Description

Technical Field

[0001] This invention relates to a high-pass filtering method, apparatus, chip, and electronic device, specifically to a method, apparatus, chip, and electronic device that utilizes neuromorphic chip convolution and LIF neurons to achieve high-pass filtering. Background Technology

[0002] High-pass filtering is a common information processing method in the field of signal processing, which allows only signals with frequencies higher than a certain limit to pass through. Band-pass filtering, on the other hand, only allows signals within a specific frequency band to pass through.

[0003] Spiking neural networks (SNNs) are third-generation neural networks that transmit information through pulse events, mimicking the spiking patterns of the brain's neural networks. While typically used for reasoning, the applicant, facing the technical challenge of efficiently extracting visual information (such as target points), independently discovered that SNN-based high-pass / band-pass filtering is a feasible technical approach. However, how to achieve high-pass / band-pass filtering using SNNs for high-frequency input pulses remains unknown. Summary of the Invention

[0004] To solve or alleviate some or all of the above-mentioned technical problems, the present invention is achieved through the following technical solution:

[0005] A high-pass filtering method is proposed to output pulse events only when the frequency of the input pulse event is higher than the cutoff frequency, project the input pulse event to a first spiking neuron, and at least by configuring the following parameters to satisfy W×f c +b×f leak =0, where: W is the synaptic weight projecting the input pulse event to the first spiking neuron, f c Let b be the cutoff frequency of the high-pass filter, b be the leakage of the first spiking neuron, and f be the leakage of the first spiking neuron. leak This represents the leakage frequency of the first spiking neuron.

[0006] In one embodiment, the input pulse event is a pulse event output by a neuromorphic sensor pixel.

[0007] In some embodiments, when implementing the high-pass filter, T is also satisfied. min ×(W×f e +b×f leak ) = θ, where T min For the minimum dwell time, f e θ represents the external stimulus frequency, and θ represents the ignition threshold of the first spiking neuron.

[0008] In one type of embodiment, the first spiking neuron leaks at least once within the minimum dwell time, i.e., T min >1 / fleak .

[0009] In one type of embodiment, with a cutoff frequency f c A flickering light source, within a minimum dwell time, causes the membrane voltage of the first spiking neuron to accumulate at least twice: T min ×f c >2.

[0010] In one type of embodiment, the synaptic weight W ≤ θ × f leak / f c .

[0011] A high-pass filter is used to output pulse events only when the frequency of the input pulse event is higher than the cutoff frequency, project the input pulse event to a first spiking neuron, and at least by configuring the following parameters to satisfy W×f c +b×f leak =0, where: W is the synaptic weight projecting the input pulse event to the first spiking neuron, f c Let b be the cutoff frequency of the high-pass filter, b be the leakage of the first spiking neuron, and f be the leakage of the first spiking neuron. leak This represents the leakage frequency of the first spiking neuron.

[0012] In one embodiment, the input pulse event is a pulse event output by a neuromorphic sensor pixel.

[0013] In some embodiments, when implementing the high-pass filter, T is also satisfied. min ×(W×f e +b×f leak ) = θ, where T min For the minimum dwell time, f e θ represents the external stimulus frequency, and θ represents the ignition threshold of the first spiking neuron.

[0014] In one type of embodiment, the first spiking neuron leaks at least once within the minimum dwell time, i.e., T min >1 / f leak .

[0015] In one type of embodiment, with a cutoff frequency f c A flickering light source, within a minimum dwell time, causes the membrane voltage of the first spiking neuron to accumulate at least twice: T min ×f c >2.

[0016] In one type of embodiment, the synaptic weight W ≤ θ × f leak / f c .

[0017] A chip comprising a high-pass filter as described in any of the preceding claims.

[0018] In one embodiment, the chip further includes a neuromorphic sensor.

[0019] In one embodiment, the neuromorphic sensor is an event camera.

[0020] An electronic device comprising a high-pass filter as described in any of the preceding claims; or, the electronic device comprising a chip as described in any of the preceding claims.

[0021] In one embodiment, the electronic device includes a first component, wherein the first component includes a light-emitting element and the light-emitting element is configured to flash light at a first frequency; the electronic device further includes a second component, wherein the second component includes a neuromorphic sensor for capturing the light-emitting element; and, pulse events output by the pixels of the neuromorphic sensor are filtered according to a high-pass filtering device as described in any of the preceding claims.

[0022] In one embodiment, the first frequency is above the cutoff frequency of the high-pass filter.

[0023] In one type of embodiment, the light emitter includes a stuttering light emission mode and / or a continuous flickering light emission mode.

[0024] In one embodiment, the electronic device is an air drum kit, and the first component is a drumstick.

[0025] Some or all of the embodiments of the present invention have the following beneficial technical effects:

[0026] 1) It can achieve intelligent visual information processing with low latency, low data volume, low computational load, low power consumption, and low cost for target points.

[0027] 2) Only neuromorphic hardware is used, without the need for other complex and redundant processing components and processes.

[0028] Further beneficial effects will be described in the preferred embodiments.

[0029] The technical solutions / features disclosed above are intended to summarize the technical solutions and features described in the Detailed Embodiments section, and therefore the scope of the description may not be entirely the same. However, these new technical solutions disclosed in this section are also part of the numerous technical solutions disclosed in this invention document. The technical features disclosed in this section, together with the technical features disclosed in the subsequent Detailed Embodiments section and some contents in the drawings not explicitly described in the specification, disclose more technical solutions in a reasonable combination.

[0030] The technical solution formed by combining all the technical features disclosed at any position in this invention is used to support the summary of the technical solution, the modification of the patent document, and the disclosure of the technical solution. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating the relationship between the event camera and the target neuron. Detailed Implementation

[0032] Since it is impossible to exhaustively describe all alternative solutions, the key points of the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Other technical solutions and details not disclosed in detail below generally belong to technical objectives or features that can be achieved by conventional means in the art, and due to space limitations, they will not be described in detail here.

[0033] Unless it refers to division, the " / " in any position in this invention represents logical "OR". The serial numbers "first", "second", etc., in any position in this invention are merely descriptive distinguishing marks and do not imply an absolute temporal or spatial order, nor do they imply that terms prefixed with such serial numbers necessarily refer to different things than the same terms prefixed with other modifiers.

[0034] This invention describes various key points used to combine into various specific embodiments, which will be incorporated into various methods and products. In this invention, even if a key point is described only when introducing a method / product solution, it means that the corresponding product / method solution also explicitly includes that technical feature.

[0035] The description of the existence or inclusion of a step, module, or feature at any location in this invention does not imply that such existence is exclusive or unique. Those skilled in the art can obtain other embodiments by supplementing the technical solutions disclosed in this invention with other technical means. The embodiments disclosed in this invention are generally for the purpose of disclosing preferred embodiments, but this does not imply that opposite embodiments of the preferred embodiments are excluded by this invention. As long as such opposite embodiments at least solve one of the technical problems of this invention, they are intended to be covered by this invention. Based on the key points described in the specific embodiments of this invention, those skilled in the art can substitute, delete, add, combine, or change the order of certain technical features to obtain a technical solution that still follows the concept of this invention. These solutions that do not depart from the technical concept of this invention are also within the protection scope of this invention.

[0036] An event camera, also known as a dynamic vision sensor (DVS), is a novel type of vision sensor that captures changes in light intensity and asynchronously sends pulse events (or events) to subsequent systems. An ON event is sent when the light intensity increases, and an OFF event is sent when it decreases. For example, for a 50Hz AC-driven light source, since light intensity changes are not phase-sensitive, it is effectively equivalent to a 100Hz light intensity change frequency. Within one cycle, the light intensity increases and decreases once. Based on the assumption that each increase and decrease generates one pulse, the event generation rate is 200 events / second. In a certain scenario of this invention, for a light intensity change frequency of f... e Due to the ON / OFF polarity of the external light source, the event generation frequency (including ON and OFF events) of the event camera is 2f. e .

[0037] like Figure 1 As shown, for a spiking neural network (SNN), it can receive pulse events output by an event camera (or other types of neuromorphic sensors; this invention uses an event camera as an example only). In one embodiment of this invention, the SNN is designed as a high-pass filter to only allow events above a set cutoff frequency f. c The high-pass filter of this invention can be used as part of an SNN or several layers, or in any other applicable location; this invention does not limit this.

[0038] In one embodiment of the invention, the neuron is implemented as a spiking neuron with leakage. Preferably, the leakage is a linear leakage of membrane potential, with a single leakage amount of b and a leakage (clock) frequency of f. leak .

[0039] For a neuromorphic sensor, the pixel receiving frequency at coordinates (x, y) is f. e The input stimulus will output a pulse event and project it to the target neuron (for the target neuron, the output pulse event of the neuromorphic sensor is the input pulse event of the target neuron). The target neuron is the first spiking neuron U corresponding to the position (x, y) on the feature map. x,y Let its membrane voltage be V. x,y .

[0040] Then the membrane voltage of the target neuron is between t0 and t... T The change (increment) over time is:

[0041] .

[0042] If at some time t, the modulus voltage V x,y If (t) is greater than the ignition threshold θ, then the target neuron U x,y Send a pulse event, denoted as Ux,y (t) = 1, and the reset membrane voltage is the resting potential, for example, V. x,y (t) = 0; where W is the synaptic weight.

[0043] The stimulus source, such as a light source flashing in a specific frequency pattern, is used only when its frequency is greater than the cutoff frequency f. c If an image is only captured by the event camera at a certain time, then the following condition must be met: W×f c +b×f leak =0. In other words, no matter how long the time period, if the frequency of the stimulus does not exceed the cutoff frequency f. c Therefore, there will be no accumulation of membrane voltage, and it will be impossible to stimulate the target neuron to fire pulses.

[0044] If the actual frequency of the received stimulus is f e >f c At that time, Wf e +bf leak >0, and after positive accumulation over time, the membrane voltage V of the target neuron x,y The sum of the initial membrane voltage and the membrane voltage increment will inevitably exceed the ignition threshold θ at some point, receiving this f e The target neurons stimulated by the frequency will fire pulses, which demonstrates the "high-pass" characteristic.

[0045] Furthermore, for pixels and frequencies of f e For the input stimulus, there exists a corresponding minimum dwell time T. min If the light source's image dwell time on a pixel of the event camera is less than the minimum dwell time, it means that the stimulus source cannot be imaged on that pixel of the event camera, and therefore will not trigger the pixel to emit a pulse event.

[0046] In other words, at the minimum dwell time T min Inside, the frequency is f e For the input stimulus, the increment of the membrane voltage is equal to the ignition threshold θ:

[0047] .

[0048] That is: T min ×(W×f e +b×f leak =θ. The implication for practical applications is that the frequency f of the luminescent body as a stimulus source... e The higher the frequency, the shorter the minimum dwell time allowed for the event camera. For example, the frequency of the light source used to mark the target point can be designed to be 250Hz, 270Hz, or 300Hz. This avoids common AC frequencies and reduces the minimum dwell time, allowing the target point or light source to move more quickly.

[0049] For some common scenarios, such as human activity, gestures, and background noise within the field of view, the signal is low-frequency for a single pixel. If a cutoff frequency f is designed... c This ensures that only stimuli above the cutoff frequency can trigger the target neuron to fire a pulse event, thus achieving high-pass filtering in the pulse domain.

[0050] As a practical limitation, preferably, it can be further specified that: within the minimum residence time, there is at least one leakage: T min >1 / f leak With the cutoff frequency f c A flickering light source accumulates the membrane voltage at least twice within the minimum dwell time: T min ×f c >2. And, the synaptic weight W ≤ θ × f leak / f c .

[0051] Preferably, f can be set leak =f c In other words, when the frequency of the external stimulus is the cutoff frequency, each accumulation of membrane voltage corresponds to one leakage, at which point the sum of the synaptic weight W and the leakage amount b is W+b=0.

[0052] Therefore, by setting high-pass filter parameters (such as synaptic weights, leakage rate, and leakage frequency) that meet the above conditions, a high-pass filter based on SNN can be implemented, with a cutoff frequency of f. c It is worth mentioning that these parameters are often not unique, but rather exist in multiple combinations.

[0053] The aforementioned neuromorphic sensors and SNNs can be designed into the same chip to form an edge AI chip that integrates sensing and computing, and applied to various electronic devices (such as interactive devices) to achieve intelligent visual information processing with low latency, low data volume, low computational load, low power consumption, and low cost at target points.

[0054] Through the aforementioned Qualcomm design, neuromorphic sensors can output (sense) only luminous stimuli of specific frequencies, which can efficiently capture dynamic information. For example, in various interactive devices, such as the top of the drumsticks of an air drum kit, the top of a smart pointer, or the top of the baton of a smart sand table, at least one luminous object emitting light in a specific frequency pattern can be designed. Through the aforementioned neuromorphic sensor and SNN design, static visual information and dynamic visual information other than the luminous object can be shielded, ultimately achieving the capture of target points in an extremely sparse data format. This information processing scheme can be widely applied in AR / VR motion-sensing interactive devices.

[0055] The light source can be designed in a stuttering light emission mode (detecting that the light source / target point is in a stuttering motion state, especially a stuttering motion in a specific direction) or / and a continuous flashing light emission mode. The former allows for a simpler information transmission and processing mode, while the combination of the two can achieve a richer interactive mode. The stuttering motion includes one or more of the following: sudden stop, sudden turn, and sudden reversal.

[0056] Although the invention has been described with reference to specific features and embodiments, various modifications, combinations, and substitutions can be made therein without departing from the invention. The scope of protection of this invention is not limited to the specific embodiments of processes, machines, manufactures, material compositions, apparatuses, methods, and steps described in the specification, and these methods and modules may also be implemented in one or more related, interdependent, cooperative, or upstream / downstream products or methods.

[0057] Therefore, the specification and drawings should be simply regarded as a description of some embodiments of the technical solutions defined by the appended claims, and thus the appended claims should be interpreted in accordance with the principle of the greatest reasonable interpretation, and are intended to cover as much as possible all modifications, variations, combinations or equivalents within the scope of the invention, while avoiding unreasonable interpretations.

[0058] To achieve better technical effects or for the needs of certain applications, those skilled in the art may make further improvements to the technical solution based on this invention. However, even if such improvements / designs are inventive and / or progressive, as long as they rely on the technical concept of this invention and cover the technical features defined in the claims, the technical solution should also fall within the protection scope of this invention.

[0059] The technical features mentioned in the appended claims may have alternative technical features, or the order of certain technical processes or material organization may be rearranged. Those skilled in the art, upon learning of this invention, will readily conceive of these alternative means, or alter the order of the technical processes or material organization, and then employ substantially the same means to solve substantially the same technical problems and achieve substantially the same technical effects. Therefore, even if the claims explicitly define the aforementioned means and / or order, these modifications, alterations, and substitutions should all fall within the scope of protection of the claims based on the principle of equivalents.

[0060] The method steps or modules described in the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the steps and components of each embodiment have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application or design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered outside the scope of protection claimed by this invention.

Claims

1. A high-pass filtering method for outputting pulse events only when the frequency of the input pulse event is higher than the cutoff frequency, characterized in that: The input pulse event is projected onto the first spiking neuron, and at least the following parameters are configured to satisfy W×f. c +b×f leak =0, where: W is the synaptic weight that projects the input pulse event to the first spiking neuron, f c Let b be the cutoff frequency of the high-pass filter, b be the leakage of the first spiking neuron, and f be the leakage of the first spiking neuron. leak This represents the leakage frequency of the first spiking neuron.

2. The high-pass filtering method according to claim 1, characterized in that: The input pulse event is a pulse event output by the pixels of the neuromorphic sensor.

3. The high-pass filtering method according to claim 1 or 2, characterized in that: When implementing the high-pass filter, T must also be satisfied. min ×(W×f e +b×f leak ) = θ, where T min For the minimum dwell time, f e θ represents the external stimulus frequency, and θ represents the ignition threshold of the first spiking neuron.

4. The high-pass filtering method according to claim 3, characterized in that: Within the minimum dwell time, the first spiking neuron leaks at least once, i.e., T min >1 / f leak .

5. The high-pass filtering method according to claim 4, characterized in that: With cutoff frequency f c A flickering light source, within a minimum dwell time, causes the membrane voltage of the first spiking neuron to accumulate at least twice: T min ×f c >2.

6. The high-pass filtering method according to claim 5, characterized in that: The synaptic weight W ≤ θ × f leak / f c .

7. A high-pass filter for outputting a pulse event only when the input pulse event frequency is higher than the cutoff frequency, characterized in that: The input pulse event is projected onto the first spiking neuron, and at least the following parameters are configured to satisfy W×f. c +b×f leak =0, where: W is the synaptic weight that projects the input pulse event to the first spiking neuron, f c Let b be the cutoff frequency of the high-pass filter, b be the leakage of the first spiking neuron, and f be the leakage of the first spiking neuron. leak This represents the leakage frequency of the first spiking neuron.

8. The high-pass filter device according to claim 7, characterized in that: The input pulse event is a pulse event output by the pixels of the neuromorphic sensor.

9. The high-pass filter device according to claim 7 or 8, characterized in that: When implementing the high-pass filter, T must also be satisfied. min ×(W×f e +b×f leak ) = θ, where T min For the minimum dwell time, f e θ represents the external stimulus frequency, and θ represents the ignition threshold of the first spiking neuron.

10. The high-pass filter device according to claim 9, characterized in that: Within the minimum dwell time, the first spiking neuron leaks at least once, i.e., T min >1 / f leak .

11. The high-pass filter device according to claim 10, characterized in that: With cutoff frequency f c A flickering light source, within a minimum dwell time, causes the membrane voltage of the first spiking neuron to accumulate at least twice: T min ×f c >2.

12. The high-pass filter device according to claim 11, characterized in that: The synaptic weight W ≤ θ × f leak / f c .

13. A chip, characterized in that: The chip includes the high-pass filter device according to any one of claims 7 to 12.

14. The chip according to claim 13, characterized in that... : The chip also includes a neuromorphic sensor.

15. The chip according to claim 14, characterized in that: The neuromorphic sensor is an event camera.

16. An electronic device, characterized in that: The electronic device includes the high-pass filter according to any one of claims 7 to 12; or, the electronic device includes the chip according to any one of claims 13 to 15.

17. The electronic device according to claim 16, characterized in that: The electronic device includes a first component, wherein the first component includes a light-emitting element, and the light-emitting element is configured to flash light at a first frequency; The electronic device further includes a second component, which comprises a neuromorphic sensor for capturing the light-emitting element; and... The high-pass filter is used to filter the pulse events output by the pixels of the neuromorphic sensor.

18. The electronic device according to claim 17, characterized in that: The first frequency is above the cutoff frequency of the high-pass filter.

19. The electronic device according to any one of claims 17-18, characterized in that: The light-emitting body includes a stuttering light emission mode and / or a continuous flashing light emission mode.

20. The electronic device according to any one of claims 17-18, characterized in that: The electronic device is an air drum kit, and the first component is a drumstick.

Images