Event signal processing method, electronic device, storage medium and program product
By processing event stream data through continuously coupled neural networks and wavelet transform, the problems of insufficient stability and generalization ability of existing methods are solved, and more accurate motion information extraction and event frame construction are achieved.
Patent Information
- Application Number
- CN202510734706.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-04
- Publication Date
- 2025-10-17
Smart Images
Figure CN120804679A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application belongs to the technical field of signal processing, and particularly relates to an event signal processing method, an electronic device, a storage medium and a program product. BACKGROUND
[0002] An event camera is a kind of neuromorphic sensor inspired by the peripheral three-layer structure of the primate retina, aiming to realize a visual system based on silicon material. Each pixel of the sensor can be independently and continuously operated to monitor the change of light intensity in the scene. When the change of light intensity exceeds a preset threshold, an event signal will be triggered. The event signal carries spatiotemporal information including timestamp, spatial coordinates and polarity. Based on this unique sampling mechanism, event stream data has the advantages of high temporal resolution, low data redundancy, wide dynamic range and extremely low delay.
[0003] However, since the traditional frame-based visual algorithm is mainly designed for image sequences, it is difficult to be directly applied to asynchronous and sparse event data. To solve such problems, researchers have proposed a processing method of compressing event stream into frame images, that is, generating two-dimensional event image frames by frequency accumulation. At the same time, some researches have proposed an event representation method based on timestamp, called event surface, which reflects the dynamic changes of objects in spatial position by recording the time information of the latest event.
[0004] To more effectively utilize the spatiotemporal feature information contained in the event stream, a voxel grid representation method based on spatiotemporal histogram is further proposed. This method discretizes the time axis and uses linear weighting to map events to corresponding voxel units, thereby constructing a three-dimensional voxel tensor to represent event density distribution.
[0005] With the development of deep learning technology and large-scale computing power, end-to-end data-driven neural network models have been widely applied in the field of event vision, which can effectively mine the asynchronous spatiotemporal characteristics of event stream. In addition, the bionic spiking neural network simulates the pulse signal processing mechanism of biological neurons, and fires a pulse when the signal accumulation exceeds the set threshold, thereby realizing dynamic sampling of events.
[0006] However, the existing event representation methods still have great limitations in generalization ability and stability, which are embodied in inconsistent performance across datasets, insufficient robustness, and high dependence on specific representation structures. Especially in the face of sparse data or high dynamic range scenarios, these methods often struggle to maintain stable processing capabilities, further exposing their shortcomings in adaptability and robustness. In addition, most current event representation methods are usually designed based on pre-set representation frameworks, such as spatio-temporal voxel grids or local feature extraction, which limits their applicability in diverse event data processing. Thus leading to insufficient accuracy of motion information extracted from event stream data. SUMMARY
[0007] In view of the problems in the prior art, the present application provides an event signal processing method, electronic equipment, storage medium and program product, which at least partially solves the problem of insufficient accuracy of motion information extraction in the prior art.
[0008] In a first aspect, the embodiments of the present disclosure provide an event signal processing method, comprising:
[0009] sampling event sequences in an event stream;
[0010] inputting the sampled data into a continuous coupled neural network, thereby converting event sequences with constant polarity in the same sampling period into signals with periodic characteristics, and converting event sequences with changing polarity into signals exhibiting chaotic dynamic behavior;
[0011] performing frequency domain analysis on the output signals of the continuous coupled neural network using continuous wavelet transform to extract multi-scale time-frequency features of the output signals, thereby obtaining motion patterns and semantic features in the event sequences;
[0012] performing low-pass filtering on the frequency domain information after frequency domain analysis to extract motion-related information, and constructing event frame data for visual task processing based on the motion-related information.
[0013] Optionally, the sampling of the event sequences in the event stream comprises:
[0014] The event points in the event stream data contain the position coordinates x, y, timestamp t and polarity p of the moving object; the coordinate information is set as the key, and the polarity sequence corresponding to the same coordinate is set as the value of the key for fixed sampling.
[0015] Optionally, the inputting of the sampled data into the continuous coupled neural network comprises:
[0016] The neurons of the continuous coupled neural network output periodic sequences when subjected to constant stimulation, and the frequency of the period is determined by the intensity of the input stimulation;
[0017] The neurons of the continuously coupled neural network output chaotic sequences when subjected to periodic stimulation.
[0018] Optionally, in the continuous wavelet transform, the Gaussian wavelet is used as a Gaussian wavelet basis function, and the Gaussian wavelet basis function is obtained by translation and scaling of a Gaussian function.
[0019] Optionally, the formula of the continuous wavelet transform is:
[0020] ,
[0021] where t is an input signal, is a standard deviation, is a Gaussian wavelet function, a is a scale parameter, and controls the scale of the wavelet function, and b is a translation parameter, and controls the position of the wavelet function.
[0022] Optionally, the inputting of the sampling data into the continuously coupled neural network, thereby converting the event sequence with constant polarity in the same sampling period into a signal with periodic characteristics, and converting the event sequence with changed polarity into a signal exhibiting chaotic dynamic behavior, comprises:
[0023] Separating the events with constant polarity from the events with changed polarity using the continuously coupled neural network.
[0024] Optionally, the output signal of the continuously coupled neural network is subjected to frequency domain analysis using the continuous wavelet transform to extract the multi-scale time-frequency features of the output signal, thereby obtaining the motion patterns and semantic features in the event sequence, comprising:
[0025] Processing the separated event sequence using the continuous wavelet transform to extract key spatiotemporal feature information and establish a high-order mapping relationship from the event stream to the event frame.
[0026] In a second aspect, the embodiments of the present disclosure further provide an electronic device, comprising:
[0027] at least one processor; and
[0028] a memory in communication connection with the at least one processor; wherein
[0029] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the event signal processing method of any one of the first aspect.
[0030] In a third aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute any event signal processing method described in the first aspect.
[0031] In a fourth aspect, an embodiment of the present disclosure further provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements any event signal processing method described in the first aspect.
[0032] The present invention provides an event signal processing method, electronic device, storage medium and program product, wherein the event signal processing method effectively separates constant polarity events from changing polarity events within the same sampling period, and then uses continuous wavelet transform (CWT) to process the separated event sequence to extract key spatiotemporal feature information and establish a high-order mapping relationship from event stream to event frame, thereby achieving the purpose of improving the accuracy of motion information extraction. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.
[0034] Figure 1 A block diagram of the event signal processing principle provided by an embodiment of the present disclosure;
[0035] Figure 2 Schematic diagram of the human brain's visual cortex recognizing moving objects through the dorsal and ventral pathways;
[0036] Figure 3 Input and output characteristic diagram of the CCNN provided in the embodiment of the present disclosure;
[0037] Figure 4 Waveform diagram and phase space diagram of CCNN neurons provided by embodiments of the present disclosure;
[0038] Figure 5 A heat map of the real part of the CWT matrix provided by an embodiment of the present disclosure. DETAILED DESCRIPTION
[0039] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.
[0040] It should be apparent that the following description illustrates by way of example only a number of possible embodiments of the present disclosure. Those skilled in the art will readily understand other advantages and benefits of the present disclosure from the description that follows, with the aid of the accompanying drawings. It should be apparent that the described embodiments are only a part of all possible embodiments of the present disclosure, and are not all-inclusive of all possible embodiments. The present disclosure can be implemented or carried out by other different specific embodiments, and the details of the description can be modified based on different views and applications, without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.
[0041] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein can be implemented in a wide variety of forms and that any particular structure and / or function described herein is merely illustrative. Based on the teachings herein one skilled in the art will appreciate that one or more aspects described herein can be implemented independently of any other aspects and that an aspect can be implemented independently of or in combination with any other aspect. For example, an apparatus can be implemented and / or a method can be practiced using any number of the aspects set forth herein. In addition, an apparatus can be implemented and / or a method can be practiced using other structure and / or functionality in addition to or other than one or more of the aspects set forth herein.
[0042] It should also be noted that the drawings included in the following description are only schematic and that actual implementations can differ from those described in the figures. The methods disclosed herein can be implemented by one or more computing devices and / or components thereof appropriately programmed using software and / or firmware constructed in accordance with the teachings contained herein.
[0043] In addition, in the following description, specific details are provided to thoroughly understand the examples. However, one skilled in the art will understand that the described aspects can be practiced without these specific details.
[0044] The embodiments disclosed herein provide a method for processing event signals, comprising:
[0045] Sampling event sequences in an event stream;
[0046] Inputting the sampled data into a continuous coupling neural network, thereby converting event sequences with constant polarity in the same sampling period into signals with periodic characteristics, and converting event sequences with changing polarity into signals exhibiting chaotic dynamic behavior;
[0047] The continuous wavelet transform is used for frequency domain analysis on the output signal of the continuous coupled neural network, so as to extract the time-frequency features of the output signal in multiple scales, and obtain the motion mode and semantic features in the event sequence.
[0048] The frequency domain information after the frequency domain analysis is subjected to low-pass filtering processing, the motion related information is extracted, and the event frame data for visual task processing is constructed based on the motion related information.
[0049] Optionally, the sampling of the event sequence in the event stream comprises:
[0050] The event point in the event stream data comprises position coordinates x, coordinate y, time stamp t and polarity p of the moving object, the coordinate information is set as a key, and the polarity sequence corresponding to the same coordinate is set as the value of the key for fixed sampling.
[0051] Optionally, the sampling data is input into the continuous coupled neural network, comprising:
[0052] The neuron of the continuous coupled neural network outputs a periodic sequence when subjected to constant stimulation, and the frequency of the period is determined by the intensity of the input stimulation.
[0053] The neuron of the continuous coupled neural network outputs a chaotic sequence when subjected to periodic stimulation.
[0054] Optionally, in the continuous wavelet transform, the Gaussian wavelet is used as a Gaussian wavelet basis function, and the Gaussian wavelet basis function is obtained by translating and scaling the Gaussian function.
[0055] Optionally, the formula of the continuous wavelet transform is:
[0056] ,
[0057] where t is the input signal, is the standard deviation, is a Gaussian wavelet function, a is a scale parameter, which controls the scale of the wavelet function, and b is a translation parameter, which controls the position of the wavelet function.
[0058] Optionally, the sampling data is input into the continuous coupled neural network, so that the event sequence with constant polarity in the same sampling period is converted into a signal with periodic characteristics, and the event sequence with changed polarity is converted into a signal exhibiting chaotic dynamic behavior, comprising:
[0059] The continuous coupled neural network is used to separate the events with constant polarity from the events with changed polarity.
[0060] Optionally, the continuous wavelet transform is used to perform frequency domain analysis on the output signal of the continuous coupled neural network, so as to extract the multi-scale time-frequency features of the output signal, and thus obtain the motion mode and semantic features in the event sequence, including:
[0061] The separated event sequence is processed using the continuous wavelet transform, and key spatiotemporal feature information is extracted therefrom, so as to establish a high-order mapping relationship from the event stream to the event frame.
[0062] As shown in Figure 1 and Figure 2 , the event stream data is input to the CCNN neuron in the form of coordinates. The dynamic characteristics of the neuron are analyzed by the continuous wavelet transform (CWT), and the event representation is extracted in combination with the low-pass filter (LPF).
[0063] This embodiment learns the cooperative mechanism of the dorsal pathway and the ventral pathway in the visual cortex of the human brain to realize motion target recognition. The event camera simulates the three-layer structure of the human retina, and encodes the motion target as event stream data. Based on the chaotic convolutional neural network (CCNN), the chaotic dynamics framework is constructed to map the event stream data to the event representation, and this process corresponds to the processing mechanism of the dorsal pathway. Subsequently, the event representation is delivered to the module analogous to the IT region from the module analogous to the MT region, and finally the motion target is recognized through the multi-layer neural network.
[0064] In a specific application scenario, the event stream generated by the event camera can be regarded as a point cloud set in a three-dimensional space, and each event point is four-dimensional data including spatial coordinates x and y, polarity and timestamp. The point cloud set is represented by the following function:
[0065] (1)
[0066] Wherein the spatiotemporal information and polarity of each event point in the event stream are represented by the Dirac function, and in the entire three-dimensional space , each event point is , when the spatiotemporal information and polarity are consistent with the information of the event point , the Dirac impulse is generated, indicating the occurrence of the event. Through the function , the event point cloud set is continuously represented.
[0067] The event representation inspired by the dorsal pathway,
[0068] 1. Sampling
[0069] In processing three-dimensional event stream data, sampling is the first step, which can be generally divided into two categories: fixed sampling and adaptive sampling. The event bin after sampling is shown in the following formula:
[0070] (2)
[0071] wherein, is the resolution of event frame, is the polarity of time, , wherein is the start time, is the number of event bins, is the time interval, is the adjustment factor, for fixed sampling mode, always remains constant; for adaptive sampling, adjusts the size according to specific requirements.
[0072] In this embodiment, the fixed sampling mode is selected. An event point in the event stream data contains the position coordinate information x and y of the moving object, the time stamp t and the polarity p. The coordinate information is set as the key, and the polarity sequence corresponding to the same coordinate is set as the value of the key. After aligning all the values, they are uniformly input into the CCNN, and different polarity change sequences output different types of signals.
[0073] (3)
[0074] wherein is the mapping function, is the spatial coordinate, as the key, represents the polarity time stamp sequence corresponding to the coordinate, as the value.
[0075] 2. Continuous Coupled Neural Network
[0076] When all the mapped polarity sequences in the same sampling period are input into the CCNN, in order to simplify, the non-connected CCNN is selected, and the mathematical model is as shown in the formula:
[0077]
[0078] (4)
[0079]
[0080] wherein, is an independent variable, which is only affected by the external input , which is the polarity sequence of a certain coordinate in this paper. When the polarity changes constantly, the general formula of U(k) is:
[0081] (5)
[0082] After derivation, the expression of period k is obtained:
[0083] (6)
[0084] Thus, the CCNN neuron outputs a periodic sequence when it is stimulated by a constant stimulus The frequency of the periodic sequence is determined by the intensity of the input stimulus.
[0085] When the polarity of the periodic sequence changes, The general formula of U(k) is:
[0086] (7)
[0087] According to formula (4), each update of E(k) is affected by Y(k), which cannot be expressed by a general mathematical formula. Therefore, the CCNN model exhibits special dynamic characteristics when stimulated by a periodic signal. Figure 3 The waveforms and phase space diagrams of the CCNN neuron stimulated by a square wave signal are shown in Figure 2, which demonstrates its complex dynamic characteristics. Figure 3 In Figure 2, (a) shows that the CCNN outputs a chaotic sequence when stimulated by an event signal with changing polarity, and (b) shows that the CCNN outputs a periodic sequence when stimulated by an event signal with constant polarity. Figure 4 In Figure 3, (a) is the waveform diagram of U, (b) is the waveform diagram of E, (c) is the waveform diagram of Y, (d) is the phase space diagram of U-E, (e) is the phase space diagram of U-Y, and (f) is the phase space diagram of E-Y. Next, we use nonlinear analysis methods to analyze the dynamic behavior of the CCNN. The equilibrium points of the model are defined as follows:
[0088] (8)
[0089] Using the Taylor expansion of to simplify the above formula, we obtain the simplified formula (9):
[0090] (9)
[0091] When , , and , the discriminant , and at this time can be expressed as:
[0092] (10)
[0093] Since U(k) is an independent variable, formula (4) is a two-dimensional discrete dynamic system. According to the above analysis, it has two equilibrium points, which can be expressed as:
[0094]
[0095] (11)
[0096]
[0097] Therefore, CCNN neuron outputs chaotic sequence when it is subjected to periodic stimulation .
[0098] 3. Continuous wavelet transform
[0099] As shown in Figure 5 , (a) is the real part of CWT matrix corresponding to chaotic sequence; (b) is the real part of CWT matrix corresponding to periodic sequence.
[0100] In the continuous wavelet transform, after many experimental attempts, the Gaussian wavelet is selected as the wavelet basis function, which is obtained by translating and scaling the Gaussian function. Its mathematical form is shown in (16):
[0101] (12)
[0102] where is the Gaussian function, t is the input signal, is the standard deviation, which controls the width of the Gaussian function. is the Gaussian wavelet function, a is the scale parameter, which controls the scale of the wavelet function, and b is the translation parameter, which controls the position of the wavelet function.
[0103] It is emphasized that the scale parameter in wavelet transform is similar to the scale of the map. Larger scale corresponds to the global description of the signal, while smaller scale provides detailed description of the signal. In the experiment, it aims to distinguish between periodic signal and chaotic signal, which have significant differences in global distribution. Therefore, larger scale is considered to be the most appropriate choice.
[0104] In the selection of scale parameter of wavelet transform, the scale parameter of wavelet transform is adjusted monotonically from 1, and finally the scale range of 10 is selected.
[0105]
[0106] (13)
[0107] where, is the output of the event polarity sequence after CCNN, is the Gaussian wavelet function, denotes convolution, is each element of the coefficient matrix after wavelet transform of the signal.
[0108] The empirical analysis of experimental data reveals a significant pattern: for periodic sequences, the real parts of wavelet coefficients are mostly distributed on the left side of zero, while for chaotic sequences, the opposite trend is shown. By summing the real parts of all wavelet coefficients, we can effectively distinguish between sequences with constant polarity and those with chaotic polarity.
[0109] (14)
[0110] where, represents the sum of the real parts of all elements of the coefficient matrix.
[0111] 4. Low-pass filter
[0112] After passing the constant polarity sequence and the chaotic polarity sequence through the CCNN and CWT, we get values on both sides of zero. In order to truly reflect the position of the moving object, we need to use a linear low-pass filter to extract the effective event points as the pixel points of the event frame
[0113] The expression is shown in the following formula:
[0114] (15)
[0115] Setting the coordinate points less than zero to 255 and the coordinate points greater than zero to 0. This processing method not only filters out irrelevant information, but also retains the most critical dynamic changes in the event stream, thereby generating clearer and more accurate event frames .
[0116] (16)
[0117] This event frame can be used for subsequent high-level tasks such as object detection, tracking, or optical flow estimation. Therefore, the input-output characteristics of CCNN provide an effective solution for event camera data processing, which helps better understand and utilize important information in the event stream.
[0118] Object classification is an important research direction in computer vision, widely used in video surveillance, autonomous driving, pose recognition and other important tasks. Object classification based on asynchronous events solves the problem of capture failure in low or high exposure scenes and motion blur of fast-moving objects in traditional frame image detection, and has great application potential.
[0119] As shown in Table 1, the stability and generalization ability of the event representation method proposed in this embodiment are verified on the following four event data sets: N-MNIST, N-Caltech101, N-CARS and ASL-DVS. Among them, the N-MNIST data set is a pulse neural version of frame-based MNIST, containing 60,000 training samples and 10,000 test samples (28x28 pixels). The data set is generated by installing an ATIS sensor on an electric pan-tilt device to collect event streams. The N-Caltech101 data set is derived from Caltech101, containing a total of 8,677 samples distributed in 101 categories, with about 40-800 samples in each category (about 300x200 pixels). The N-CARS data set is a real-world event-based car classification data set, containing 12,336 car samples and 11,693 non-car samples, which are collected by using an ATIS camera to shoot 100-millisecond event fragments. The ASL-DVS data set contains 100,800 samples, covering 24 American sign language letters (excluding J), each sample is 100 milliseconds long, and is collected by a DAVIS240c event camera in a controlled office environment.
[0120] Table 1, Experimental verification table of event representation on four event data sets
[0121]
[0122] As shown in Table 2, the event representation model proposed in this embodiment shows excellent performance on four mainstream event classification data sets. On the N-CARS data set, the model achieves an accuracy of 99.9%, which is 4.5 and 5.4 percentage points higher than the TOKEN and TORE methods respectively; on the N-Caltech101 data set, the accuracy is improved by 17.5 and 9.05 percentage points compared with the HATS and EST methods, respectively, which reflects its good adaptability to complex and high-variability categories; on the N-MNIST data set, the model achieves an accuracy of 97.37%, showing good robustness; on the ASL-DVS data set, the model achieves an accuracy of 99.2%, close to the current highest level, verifying its effectiveness in dynamic gesture recognition tasks.
[0123] Table 2, Event representation model shows excellent performance on four mainstream event classification data sets
[0124]
[0125] To comprehensively evaluate the model complexity of the proposed framework, three indicators are adopted in this embodiment: the number of trainable parameters, multiply-accumulate operations (MACs), and average inference time. On the N-Caltech101 dataset, the framework achieves a high classification accuracy while maintaining a moderate computational load of about 3.67G MACs, verifying its performance and efficiency balance in event representation learning. Further, on the N-CARS dataset, a workstation configured with an Intel Core i9 CPU, an NVIDIA RTX 4060 GPU, and 16GB RAM is used to test the inference speed. The results show that the single-sample average inference time is 2.12 milliseconds, corresponding to a throughput of about 472 samples / second, indicating that the framework has practical feasibility for application in high-speed scenarios.
[0126] The proposed framework of this embodiment and existing methods are evaluated on the N-Caltech101 dataset in terms of Intersection over Union (IoU) performance. Each method is tested under different numbers of events, including 30,000, 50,000, 70,000, and 100,000. The results are shown in Table 3. The proposed framework achieves higher IoU values under all event quantities, with an IoU of 0.4879 at 30,000 events, and continuously increasing IoU as the number of events increases, reaching 0.6450 at 100,000 events. The experimental results show that the framework can more fully extract and utilize the spatiotemporal information in the event stream as the number of events increases, helping to reconstruct more complete object shapes and features and improving the representation effect.
[0127] Table 3: IoU performance comparison of the proposed method on the N-Caltech101 dataset
[0128]
[0129] In the present application, a continuous coupled neuron network (CCNN) inspired by the primary visual cortex is proposed. The network possesses electrophysiological properties similar to mammalian neuron clusters: periodic sequence outputs are generated when constant input signals are received, while chaotic sequence outputs are generated when input signals change. Such input-output response mechanisms enable the network to effectively distinguish between stable events (i.e. constant polarity patterns) and dynamic events (i.e. changing polarity patterns). Based on the unique behavior characteristics of the CCNN, constant polarity events and changing polarity events are effectively separated within the same sampling period. Subsequently, the separated event sequences are processed using a continuous wavelet transform (CWT) to extract key spatiotemporal feature information, and a high-order mapping relationship from event streams to event frames is established. To further improve the accuracy of motion information extraction, a deep neural network module is integrated to achieve accurate positioning and recognition of moving targets.
[0130] An electronic device according to an embodiment of the present disclosure includes a memory and a processor. The memory is configured to store non-transitory computer-readable instructions. Specifically, the memory can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may, for example, include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may, for example, include read-only memory (ROM), hard disk, flash memory, etc.
[0131] The processor can be a central processing unit (CPU) or other form of processing unit having data processing and / or instruction execution capabilities, and can control other components in the electronic device to perform desired functions. In an embodiment of the present disclosure, the processor is configured to execute the computer-readable instructions stored in the memory, so that the electronic device performs all or part of the steps of the event signal processing method of the embodiments of the present disclosure described above.
[0132] Those skilled in the art will understand that, in order to solve the technical problem of how to obtain a good user experience effect, the present embodiment can also include well-known structures such as a communication bus, an interface, etc., which should also be included in the protection scope of the present disclosure.
[0133] Detailed descriptions of the present embodiment can be found in the corresponding descriptions of the foregoing embodiments, which will not be repeated here.
[0134] A computer readable storage medium according to embodiments of the present disclosure has non-transitory computer readable instructions stored thereon. When the non-transitory computer readable instructions are run by a processor, all or part of the steps of the event signal processing method of the embodiments of the present disclosure described above are performed.
[0135] The computer readable storage medium described above includes, but is not limited to, an optical storage medium (for example, a CD-ROM and a DVD), a magneto-optical storage medium (for example, an MO), a magnetic storage medium (for example, a magnetic tape or a mobile hard disk), a medium with a built-in rewritable nonvolatile memory (for example, a memory card), and a medium with a built-in ROM (for example, a ROM cartridge).
[0136] Detailed descriptions of the embodiments are described above with reference to the corresponding descriptions of the previous embodiments, and will not be repeated here.
[0137] The above describes the basic principles of the present disclosure in combination with specific embodiments, but it should be noted that the advantages, advantages, effects and the like mentioned in the present disclosure are only examples and are not limiting, and these advantages, advantages, effects and the like cannot be considered as the must-have of each embodiment of the present disclosure. In addition, the specific details of the above disclosure are only for the purpose of example and for the purpose of understanding, and are not limited to the above specific details, and the above specific details do not limit the present disclosure to be implemented with the above specific details.
[0138] In the present disclosure, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. The block diagrams of devices, apparatuses, equipment, systems involved in the present disclosure are only illustrative examples and are not intended to require or imply the connection, arrangement, configuration shown in the block diagram. As those skilled in the art will recognize, these devices, apparatuses, equipment, systems can be connected, arranged, configured in any manner. Words such as "include", "contain", "have" and the like are open-ended words, which mean "including but not limited to", and can be used interchangeably. The words "or" and "and" used herein mean the word "and / or", and can be used interchangeably unless the context clearly indicates otherwise. The word "such as" used herein means the phrase "such as but not limited to", and can be used interchangeably.
[0139] In addition, as used herein, "or" used in the list of items preceded by "at least one of" means a disjunctive list, such that, for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "example" does not mean that the described example is preferred or better than other examples.
[0140] It is also important to note that the systems and methods of the present disclosure can be implemented in a variety of ways. The systems and methods can be implemented using computer hardware, software, firmware, or any combination thereof. For example, the systems and methods can be implemented using a variety of different types of computing devices, such as a server, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a personal digital assistant, a mobile telephone, an Internet appliance or any other device suitable for carrying out the functions described herein. The systems and methods can be implemented using a variety of different types of computing devices, such as a server, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a personal digital assistant, a mobile telephone, an Internet appliance or any other device suitable for carrying out the functions described herein. The systems and methods can be implemented using a variety of different types of computing devices, such as a server, a desktop computer, a laptop computer, a tablet computer, a handheld computer, a personal digital assistant, a mobile telephone, an Internet appliance or any other device suitable for carrying out the functions described herein.
[0141] Various changes, modifications and alterations to the techniques described herein can be made without departing from the teachings of the technology defined by the appended claims. In addition, the scope of the claims of the present disclosure is not limited to the specific aspects described above. The specific aspects are described above for illustrative purposes, but one of ordinary skill in the art will appreciate that other aspects can be employed without departing from the scope of the present disclosure. Consequently, to the extent that the claims are broader in scope than any of the above-described aspects, then the claims should not be limited to the specific aspects described above.
[0142] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
[0143] The above description has been given for the purpose of illustration and description. Furthermore, this description does not purport to be exhaustive or to limit the embodiments of the disclosure to the precise forms disclosed. Although various example aspects and implementations have been discussed above, those of skill in the art will recognize that certain modifications, changes, substitutions, additions and rearrangements are possible without departing from the scope of the disclosure.
Claims
1. A method for processing an event signal, characterized in that: include: Sampling event sequences in an event stream; The sampling data is input into a continuously coupled neural network, so that the event sequence with constant polarity in the event sequence within the same sampling period is converted into a signal with periodic characteristics, and the event sequence with changing polarity is converted into a signal showing chaotic dynamic behavior; Continuous wavelet transform is used to perform frequency domain analysis on the output signal of the continuous coupled neural network to extract the multi-scale time-frequency features of the output signal, thereby obtaining the motion pattern and semantic features in the event sequence; After frequency domain analysis, the frequency domain information is low-pass filtered to extract motion-related information, and event frame data for visual task processing is constructed based on the motion-related information.
2. The event signal processing method according to claim 1, characterized in that: The sampling of the event sequence in the event stream includes: The event points in the event stream data include the position coordinates x, y, timestamp t and polarity p of the moving object; the coordinate information is set as the key, and the polarity sequence corresponding to the same coordinate is fixedly sampled as the value of the key.
3. The event signal processing method according to claim 2, characterized in that: The step of inputting the sampled data into the continuously coupled neural network comprises: The neurons of a continuously coupled neural network output a periodic sequence when subjected to a constant stimulus, and the frequency of the cycle is determined by the intensity of the input stimulus; Neurons of continuously coupled neural networks output chaotic sequences when they are stimulated periodically.
4. The event signal processing method according to claim 3, characterized in that: In continuous wavelet transform, Gaussian wavelet is used as Gaussian wavelet basis function, which is obtained by Gaussian function translation and scaling.
5. The event signal processing method according to claim 4, characterized in that: The formula for continuous wavelet transform is; ， Where t is the input signal, is the standard deviation, is the Gaussian wavelet function, a is the scale parameter that controls the scale of the wavelet function, and b is the translation parameter that controls the position of the wavelet function.
6. The event signal processing method according to claim 5, characterized in that: The sampling data is input into a continuously coupled neural network, thereby converting an event sequence with constant polarity in an event sequence within the same sampling period into a signal with periodic characteristics, and converting an event sequence with changed polarity into a signal exhibiting chaotic dynamic behavior, including: Use a continuously coupled neural network to separate events where polarity remains constant from events where polarity changes.
7. The event signal processing method according to claim 6, characterized in that: The continuous wavelet transform is used to perform frequency domain analysis on the output signal of the continuous coupled neural network to extract the multi-scale time-frequency features of the output signal, thereby obtaining the motion pattern and semantic features in the event sequence, including: The separated event sequence is processed using continuous wavelet transform to extract key spatiotemporal feature information and establish a high-order mapping relationship from event stream to event frame.
8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the event signal processing method according to any one of claims 1 to 7.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the event signal processing method according to any one of claims 1 to 7.
10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the event signal processing method according to any one of claims 1 to 7 is implemented.