Event processing method and apparatus, electronic device, computer readable medium

Abstract

The present disclosure provides an event processing method and device, electronic equipment and computer readable medium, the method comprising: obtaining an event stream to be processed; performing event detection on the event stream, determining the current event detection result of the event stream; adjusting the time step for processing the event stream according to the event detection result; processing the event stream using the time step to obtain the processing result of the event stream. According to the embodiment of the present disclosure, the computational complexity in the processing process can be reduced on the basis of ensuring the accuracy of the processing.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, and particularly relates to an event processing method and device, electronic equipment and computer readable medium. BACKGROUND

[0002] Neuromorphic technology refers to a technology of simulating the structure of a neural system in a living being by using electronic technology to process data. In the related art, a neuromorphic model (also referred to as a spiking neural network) can be implemented by a hardware circuit (for example, a neuromorphic chip) or a software system.

[0003] When a system processes by using a neuromorphic model (for example, brain simulation), a time step is usually set in advance before running, and processing is performed at a period of the time step. The time step is usually a fixed value, for example, 0.1 ms, that is, for example, 10000 biological time steps per second are processed in the brain simulation process.

[0004] In the related art, the time step needs to be set small enough to process the required detail changes, which may result in too large an amount of computation. If a larger step is set, details may be lost, and even the accuracy and authenticity of processing cannot be guaranteed. SUMMARY

[0005] The present disclosure provides an event processing method and device based on a many-core system, electronic equipment and computer readable medium.

[0006] In a first aspect, the present disclosure provides an event processing method, which comprises:

[0007] obtaining an event stream to be processed; performing event detection on the event stream to determine an event detection result of the event stream at present; adjusting a time step for processing the event stream according to the event detection result; processing the event stream by using the time step to obtain a processing result of the event stream.

[0008] In some possible implementation manners, the event detection result comprises whether event information is detected in the event stream, and the adjusting of the time step for processing the event stream according to the event detection result comprises: in the case that event information is detected in the event stream, adjusting the time step to a second time step, the second time step being smaller than an initial first time step.

[0009] In some possible implementation manners, the event detection result comprises whether event information is detected in the event stream, and the adjusting the time step length for processing the event stream according to the event detection result comprises: in a case where event information is not detected in the event stream and the current time step length is a second time step length, adjusting the time step length to an initial first time step length, the second time step length being smaller than the first time step length.

[0010] In some possible implementation manners, the event stream comprises a plurality of regions, and the event detection result comprises whether event information is detected in each region of the event stream, and the adjusting the time step length for processing the event stream according to the event detection result comprises:

[0011] in a case where there is a target region in the plurality of regions, adjusting the time step length of the target region to a second time step length, the target region being a region in which event information is detected, and the second time step length being smaller than an initial first time step length.

[0012] In some possible implementation manners, the event detection result comprises whether event information is detected in the event stream and an event density of the event stream, and the adjusting the time step length for processing the event stream according to the event detection result comprises: in a case where event information is detected in the event stream, adjusting the time step length according to the event density,

[0013] wherein the adjusting the time step length according to the event density comprises: in a case where the event density is greater than or equal to a first density threshold and smaller than or equal to a second density threshold, adjusting the time step length to a second time step length, the second density threshold being greater than the first density threshold, and the second time step length being smaller than an initial first time step length; or in a case where the event density is greater than the second density threshold, adjusting the time step length to a third time step length, the third time step length being smaller than the second time step length.

[0014] In some possible implementation manners, the event detection on the event stream to determine the current event detection result of the event stream comprises: event detection on the event stream to determine an event density of the event stream; and in a case where the event density is greater than or equal to a third density threshold, determining that the event detection result is that event information is detected in the event stream.

[0015] In some possible implementation manners, the event detection on the event stream, and determination of the current event detection result of the event stream, include: dividing the event stream into a plurality of event frames according to a preset time length; and performing event detection on a current event frame of the event stream by using an event detection network to obtain the event detection result.

[0016] In some possible implementation manners, the processing of the event stream by using the time step to obtain a processing result of the event stream includes: performing brain simulation processing on the event stream by using the time step through a spiking neural network to obtain response information of the spiking neural network to the event stream, and the processing result includes the response information, where a differential expression of a neuron parameter in brain simulation is adjusted according to the time step.

[0017] In some possible implementation manners, the event stream is an event stream of a target scene collected by an event collection device, and the event stream is used to represent brightness change information of the target scene.

[0018] In a second aspect, the present disclosure provides an event processing apparatus, which comprises:

[0019] an event stream acquisition module configured to acquire an event stream to be processed; an event detection module configured to perform event detection on the event stream to determine a current event detection result of the event stream; a step adjustment module configured to adjust a time step for processing the event stream according to the event detection result; and an event stream processing module configured to process the event stream by using the time step to obtain a processing result of the event stream.

[0020] In some possible implementation manners, the event detection result includes event information whether detected in the event stream, and the step adjustment module is configured to: in a case where the event information is detected in the event stream, adjust the time step to a second time step, the second time step being smaller than an initial first time step.

[0021] In some possible implementation manners, the event detection result includes event information whether detected in the event stream, and the step adjustment module is configured to: in a case where the event information is not detected in the event stream and a current time step is a second time step, adjust the time step to an initial first time step, the second time step being smaller than the first time step.

[0022] In some possible implementation manners, the event stream includes a plurality of regions, and the event detection result includes whether event information is detected in each region of the event stream, and the step adjustment module is configured to: in a case where there is a target region in the plurality of regions, adjust a time step of the target region to a second time step, the target region is a region in which event information is detected, and the second time step is smaller than an initial first time step.

[0023] In some possible implementation manners, the event detection result includes whether event information is detected in the event stream and an event density of the event stream, and the step adjustment module includes an adjustment submodule configured to: in a case where event information is detected in the event stream, adjust the time step according to the event density,

[0024] wherein the adjustment submodule is configured to: in a case where the event density is greater than or equal to a first density threshold and less than or equal to a second density threshold, adjust the time step to a second time step, the second density threshold is greater than the first density threshold, and the second time step is smaller than an initial first time step; or in a case where the event density is greater than the second density threshold, adjust the time step to a third time step, the third time step is smaller than the second time step.

[0025] In some possible implementation manners, the event detection module is configured to: perform event detection on the event stream to determine a current event density of the event stream; and in a case where the event density is greater than or equal to a third density threshold, determine that the event detection result is that event information is detected in the event stream.

[0026] In some possible implementation manners, the event detection module is configured to: divide the event stream into a plurality of event frames according to a preset time length; and perform event detection on a current event frame of the event stream by using an event detection network to obtain the event detection result.

[0027] In some possible implementation manners, the event stream processing module is configured to: perform brain simulation processing on the event stream by using the pulse neural network to obtain response information of the pulse neural network for the event stream, the processing result includes the response information, and a differential expression of a neuron parameter in brain simulation is adjusted according to the time step.

[0028] In some possible implementation manners, the event stream is an event stream of a target scene collected by an event collection device, and the event stream is used to represent brightness change information of the target scene.

[0029] In a third aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the event processing method described above.

[0030] In a fourth aspect, the present disclosure provides a computer readable medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the event processing method described above.

[0031] The embodiments provided by the present disclosure can perform event detection on an event stream to determine an event detection result, adjust a time step for processing the event stream according to the event detection result, and process the event stream using the time step to obtain a processing result, thereby achieving dynamic adjustment of the time step and reducing the amount of computation in the entire processing process on the basis of ensuring the accuracy and authenticity of the processing.

[0032] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0033] The accompanying drawings are included to provide a further understanding of the present disclosure and constitute a part of the specification, which together with the embodiments of the present disclosure are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of the specific example embodiments, with reference to the accompanying drawings, in which:

[0034] Figure 1 A flowchart of an event processing method provided by an embodiment of the present disclosure;

[0035] Figure 2 A schematic diagram of a processing process of an event processing method provided by an embodiment of the present disclosure;

[0036] Figure 3 A block diagram of an event processing device provided by an embodiment of the present disclosure;

[0037] Figure 4 A block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0038] In order to enable a person skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the drawings, including various details of the embodiments of the present disclosure to help understanding, which should be considered only as exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of clarity and conciseness, the description below omits the description of well-known functions and structures.

[0039] The embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict, if possible.

[0040] As used herein, the term “and / or” includes any and all combinations of one or more of the associated listed items.

[0041] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the terms “comprise” and / or “consist of,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The terms “connected” or “coupled” and / or similar terms are not limited to a physical or mechanical connection or linkage to either another part or device and can also include an electrical connection, whether direct or indirect.

[0042] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an overly formal or overly strict sense unless expressly so defined herein.

[0043] As mentioned previously, in the related art, when processing (e.g., brain simulation) is performed using a neuromorphic model, a fixed time step is set in advance before running, which can result in an excessively large amount of computation or loss of details.

[0044] However, neurons in a neuromorphic model are driven by events, i.e., neurons start computing only when there is an input event, and the system is generally more concerned about the response of neurons after receiving input (stimulus).

[0045] According to the event processing method provided in the embodiments of the present disclosure, the time step can be dynamically adjusted during processing of input information (for example, an event stream), a more detailed time step is used when a stimulus signal is received, so as to improve the accuracy of processing; a coarser time step is used when there is no input or no stimulus signal (also referred to as a signal of interest), so as to reduce the computational load of the system, thereby reducing the computational load in the entire processing process on the basis of ensuring the accuracy and authenticity of processing.

[0046] The event processing method provided in the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer readable instructions stored in a memory. Alternatively, the method can be executed by a server.

[0047] Figure 1 A flowchart of an event processing method is provided in the embodiments of the present disclosure. Referring to Figure 1 , the method comprises:

[0048] In step S11, an event stream to be processed is acquired;

[0049] In step S12, event detection is performed on the event stream, and a current event detection result of the event stream is determined;

[0050] In step S13, the time step for processing the event stream is adjusted according to the event detection result;

[0051] In step S14, the event stream is processed by using the time step, and a processing result of the event stream is obtained.

[0052] For example, the event stream to be processed can be acquired in step S11, and the event stream can be a pulse signal (or pulse train) with spatial information and time information. For example, the event stream corresponds to M*N spatial positions (M and N are integers greater than 1), and each spatial position has a pulse signal generated with time t. The pulse signal can be a binary 0, 1 sequence, or a ternary -1, 0, 1 sequence, and the specific representation form of the pulse signal is not limited in the present disclosure.

[0053] In some possible implementation manners, the event stream can be an event stream of a target scene collected by an event collection device, and the event stream is used to represent the brightness change information of the target scene.

[0054] The target scene can be a scene including a corresponding geographical area of a building, a landscape, a person, a vehicle, etc. The event collection device can be an event camera, for example, which can collect high-time-frequency asynchronous brightness changes (events) and output event data in a stream form (event stream).

[0055] In some possible implementation manners, the initial event stream collected by the event collection device can be binarized or ternarized to obtain the event stream to be processed, and the disclosure does not limit the specific processing manner.

[0056] Those skilled in the art should understand that the event stream to be processed can also be any event data in a stream form having other physical meanings, such as voice, text, etc., and the disclosure does not limit the specific content of the event stream.

[0057] In some possible implementation manners, in step S12, event detection can be performed on the event stream to determine an event detection result of the event stream at present. The event detection result includes whether event information is detected in the event stream, i.e., event information is detected or event information is not detected. The disclosure does not limit the specific manner of event detection.

[0058] In some possible implementation manners, the density of event information in the event stream can be calculated in real time, for example, the proportion of positions having event information (e.g., taking a value of 1) in all spatial positions in M*N spatial positions of the current event stream is calculated as the event density; when the event density is greater than or equal to a preset density threshold, it is determined that event information is detected; otherwise, when the event density is less than the density threshold, it is determined that event information is not detected.

[0059] In some possible implementation manners, a corresponding event detection network can also be trained in advance, and the current event stream is input into the event detection network for classification to determine whether event information is detected. The event detection network can be a convolutional neural network or a spiking neural network, and the disclosure does not limit this.

[0060] In some possible implementation manners, in step S13, the time step for processing the event stream by the system can be adjusted according to the event detection result. For example, a larger time step can be used when starting to run; a smaller time step can be adjusted after event information is detected; the larger time step can be adjusted back after the event information ends; a plurality of gear time steps can also be set according to the number / density of event information, and the disclosure does not limit the specific adjustment manner.

[0061] In some possible implementation manners, in step S14, the system can process the event stream by using the time step length mentioned above, for example, brain simulation, to obtain a processing result of the event stream, so as to implement a corresponding processing task.

[0062] Figure 2 A schematic diagram of a processing procedure of an event processing method provided by an embodiment of the present disclosure is shown in FIG. 2. Referring to FIG. 2, Figure 2 The event detection component 21 and the event stream processing component 22 can be configured to perform event detection and event stream processing respectively. The event detection component 21 and the event stream processing component 22 can be hardware modules or software modules, and the present disclosure does not limit the same.

[0063] In an example, the input event stream 23 can be input into the event detection component 21 and the event stream processing component 22 respectively. The event detection component 21 performs event detection on the event stream 23 to determine an event detection result, adjusts a time step length according to the event detection result, and sends the time step length to the event stream processing component 22. The event stream processing component 22 processes the event stream 23 by using the time step length to output a corresponding processing result 24, thereby implementing the entire adjustment and processing procedure.

[0064] According to an embodiment of the present disclosure, event detection can be performed on an event stream to determine an event detection result, a time step length for processing the event stream is adjusted according to the event detection result, and the event stream is processed by using the time step length to obtain a processing result, thereby dynamically adjusting the time step length, reducing the amount of computation in the entire processing procedure on the basis of ensuring the accuracy and authenticity of processing.

[0065] The event processing method according to an embodiment of the present disclosure is described below.

[0066] As described above, the event stream to be processed can be obtained in step S11, and event detection is performed on the event stream in step S12.

[0067] In some possible implementation manners, step S12 can include:

[0068] performing event detection on the event stream to determine an event density of the event stream at present;

[0069] in a case where the event density is greater than or equal to a third density threshold, determining that the event detection result is that event information is detected in the event stream.

[0070] For example, event detection can be performed in a statistical manner. For example, the density of event information in the event stream is statistically determined in real time, and the proportion of positions with event information (for example, taking values of 1 and / or -1) in all spatial positions in M*N spatial positions of the current event stream is calculated as the event density.

[0071] In some possible implementation manners, a length of the event stream can also be selected, for example, event data within a time length before and after a current time point, and a proportion of event information in the event data can be counted as the event density.

[0072] In some possible implementation manners, different weights can also be set for different spatial positions, for example, a weight of a region of interest in M*N spatial positions is larger, and a weight of other regions is smaller, and the event density of the event stream can be determined by weighted summation of event information of the spatial positions.

[0073] In some possible implementation manners, the M*N spatial positions can also be divided into multiple regions, and event densities of the regions can be counted respectively. The disclosure does not limit the specific counting manner.

[0074] In some possible implementation manners, event information of interest can also be set, for example, a pulse signal with a value of 1 in the event stream is taken as the event information of interest, and a pulse signal with a value of -1 is ignored, so that the counted event density is the density of the event information of interest, and the event detection is more targeted, thereby further improving the effect of event detection.

[0075] In some possible implementation manners, if the event density is greater than or equal to a preset density threshold (referred to as a third density threshold), it can be determined that the event detection result is that event information is detected in the event stream. Otherwise, it can be determined that the event detection result is that event information is not detected in the event stream. The disclosure does not limit the specific value of the third density threshold.

[0076] In some possible implementation manners, in the case of counting the event density in regions, whether event information is detected in each region can be determined respectively, that is, the event detection result includes whether event information is detected in each region.

[0077] In this way, whether event information exists in the event stream can be detected in a simple manner, the flexibility of detection is improved, and the amount of calculation of detection is reduced.

[0078] In some possible implementation manners, the event detection on the event stream in step S12 can further include:

[0079] The event stream is divided into multiple event frames according to a preset time length;

[0080] The current event frame of the event stream is detected by the event detection network to obtain the event detection result.

[0081] For example, event detection can be performed by means of a neural network. The event stream is divided into a plurality of event frames according to a preset time length. The time length can be set as a sampling time length of the event stream, for example, to improve the real-time performance of processing; or can be set as a plurality of sampling time lengths to improve the robustness of processing. The specific value of the time length is not limited in the present disclosure.

[0082] In some possible implementation manners, the current event frame of the event stream can be input into a pre-trained event detection network for processing, to classify whether the event stream is in an input period, and output an event detection result, i.e., whether event information is detected in the event stream.

[0083] The event detection network can be a convolutional neural network or a spiking neural network, for example, and the specific network structure and training method of the event detection network are not limited in the present disclosure.

[0084] In this way, the accuracy of event detection can be improved.

[0085] In some possible implementation manners, after the event detection result is determined in step S12, the time step for processing the event stream can be adjusted according to the event detection result in step S13.

[0086] In some possible implementation manners, step S13 can include:

[0087] In the case where event information is detected in the event stream, the time step is adjusted to a second time step, which is smaller than the initial first time step.

[0088] That is, if event information is detected in the event stream, the time step for processing the event stream can be adjusted to a smaller time step (referred to as a second time step). The second time step is smaller than the time step when the system is initially running (referred to as a first time step). For example, the first time step is 10 ms, and the second time step is 0.1 ms.

[0089] It should be understood that the first time step and the second time step can be set by a person skilled in the art according to actual conditions, and the specific values of the first time step and the second time step, and the specific adjustment method of the time step are not limited in the present disclosure.

[0090] In this way, a more detailed time step can be used when there is event information, so as to improve the accuracy of processing.

[0091] In some possible implementation manners, step S13 can include:

[0092] If no event information is detected in the event stream and the current time step is the second time step, the time step is adjusted to the initial first time step, where the second time step is smaller than the first time step.

[0093] In other words, if no event information is detected in the event stream, adjustments can be made based on the current time step. If the current time step is already a larger first time step, no adjustment is made; if the current time step is a smaller second time step, the time step is adjusted back to the larger first time step.

[0094] In this way, the amount of computation can be reduced during periods without event information.

[0095] In some possible implementations, step S13 may include:

[0096] If a target region exists in the plurality of regions, the time step of the target region is adjusted to a second time step, wherein the target region is the region where event information is detected, and the second time step is less than the initial first time step.

[0097] For example, an event stream may include multiple regions, such as dividing the M*N spatial locations of the event stream into 9 regions of 3*3. In step S12, it is determined whether event information is detected in each region, that is, the event detection result of the event stream includes whether event information is detected in each region of the event stream. This disclosure does not limit the specific detection method.

[0098] In some possible implementations, the area where the event information is detected can be designated as the target area. This target area may contain moving objects, such as a football rolling on the field or a vehicle driving on the street, while objects in other areas remain stationary.

[0099] In some possible implementations, if a target region exists in multiple regions of the event stream, the time step for processing the target region is adjusted to a smaller second time step, while the time steps for other regions remain unchanged, still being a larger first time step. Conversely, if no target region exists in multiple regions, the time step for processing the entire event stream is adjusted to a larger first time step.

[0100] In this way, the time step of event stream processing can be adjusted more precisely, improving the accuracy of step adjustment and further reducing the computational load of the system.

[0101] In some possible implementations, the event detection result includes whether event information is detected in the event stream and the event density of the event stream, wherein step S13 may include:

[0102] in the event stream, adjusting the time step according to the event density,

[0103] wherein the step of adjusting the time step according to the event density comprises:

[0104] in a case where the event density is greater than or equal to a first density threshold and less than or equal to a second density threshold, adjusting the time step to a second time step, the second density threshold being greater than the first density threshold, and the second time step being less than an initial first time step; or

[0105] in a case where the event density is greater than the second density threshold, adjusting the time step to a third time step, the third time step being less than the second time step.

[0106] For example, the event detection result can further comprise an event density of the event stream, so as to set multiple gears of time step adjustment, thereby performing more refined adjustment.

[0107] In some possible implementation manners, if event information is detected in the event stream, the time step can be adjusted according to the event density. Multiple density thresholds of the event density can be preset, for example, a first density threshold and a second density threshold, the second density threshold (for example, taking a value of 0.3) being greater than the first density threshold (for example, taking a value of 0.1).

[0108] In some possible implementation manners, if the event density is greater than or equal to the first density threshold and less than or equal to the second density threshold, it can be considered that there is low-density event information (there is event information but the quantity is small), and the time step can be adjusted to a second time step. The second time step (for example, 1 ms) is less than an initial first time step (for example, 10 ms), thereby not excessively increasing the calculation amount while ensuring processing precision.

[0109] In some possible implementation manners, if the event density is greater than the second density threshold, it can be considered that there is high-density event information (there is event information and the quantity is large), and the time step can be adjusted to a third time step. The third time step (for example, 0.1 ms) is less than the second time step (for example, 1 ms), thereby improving the accuracy and authenticity of processing.

[0110] In some possible implementation manners, the event stream can also be divided into multiple regions, and the same or different density thresholds and time steps of each gear are set for the multiple regions. For example, a region of interest in the event stream is set with a lower density threshold and more detailed time step.

[0111] It should be understood that the number and values of the density threshold and the specific values of the time steps of each gear can be set by the person skilled in the art according to actual conditions, and the present disclosure does not limit this.

[0112] In this way, the accuracy of the step adjustment can be further improved, and on the basis of ensuring the accuracy and authenticity of the processing, the computational amount of the system is further reduced.

[0113] In some possible implementation ways, the event stream is processed by using the corresponding time step in step S14. Step S14 can include the following steps.

[0114] The event stream is processed by using the time step and by using the spiking neural network to obtain response information of the spiking neural network for the event stream, and the processing result includes the response information.

[0115] For example, the processing of the event stream can be brain simulation processing, which simulates the processing of the event stream by the brain through the spiking neural network. A spiking neural network can be preset, and the spiking neural network includes a plurality of neurons, each neuron is connected to other neurons and can receive and / or send pulses. The present disclosure does not limit the network structure and implementation of the spiking neural network.

[0116] In some possible implementation ways, the processing task performed by the spiking neural network can include any one of an image processing task, a speech processing task, a text processing task, a video processing task, and the like. The present disclosure does not limit this.

[0117] In some possible implementation ways, the event stream can be processed by using the time step adjusted in step S13 and by using the spiking neural network to obtain response information of the spiking neural network for the event stream as a processing result of the event stream, for example, identifying a vehicle driving in a scene.

[0118] In some possible implementation ways, the differential expression of the neuron parameter in the brain simulation is adjusted according to the time step. The neuron parameter can include a membrane potential, a delay of a synapse, and the like, and the present disclosure does not limit the specific categories of the neuron parameter.

[0119] For example, in the case of a variable time step, various calculation formulas in the brain simulation can change accordingly, wherein the differential equation (such as dv / dt) in the brain simulation is unchanged, and after being converted into a difference expression, the variable related to the time step Δt in the difference equation of the system needs to be adjusted according to the time step Δt, that is, the time step Δt is no longer a constant.

[0120] For example, the calculation formula of the membrane potential of the neuron can be expressed as:

[0121] Vupd(n) = Vupd(n-1) * φ(Δt) + f(Δt) (1)

[0122] In formula (1), Vupd(n) represents the membrane potential of the neuron in the current nth processing period; Vupd(n-1) represents the membrane potential of the neuron in the previous (n-1)th processing period; φ(Δt) is a product term of the change function of the membrane potential, used to represent the state decay of the membrane potential; f(Δt) is a summation term of the change function of the membrane potential, f(Δt) includes the membrane potential change caused by the input current, and a related term of the resting potential.

[0123] It can be seen that the change amount of the membrane potential is a function of the time step Δt.

[0124] In addition, the delay of the synapse in the neuron also needs to be adjusted according to the time step Δt when converted into a differential expression (expressed as the number of time steps). For example, if the time step Δt is 0.1 ms, the value of the delay of 1 ms is represented as 10; if the time step Δt is 0.01 ms, the value of the delay of 1 ms is represented as 100.

[0125] In this way, event stream processing under dynamic time steps can be implemented, the adaptability of the processing process is improved, and the accuracy of the processing is improved.

[0126] According to the event processing method provided in the embodiments of the present disclosure, the time step can be dynamically adjusted during processing of the input event stream, a more detailed time step is used when event information is detected, so as to improve the accuracy of the processing; a coarser time step is used when there is no input or no event information of interest, so as to reduce the computational amount of the system, thereby reducing the computational amount in the entire processing process on the basis of ensuring the accuracy and authenticity of the processing.

[0127] According to the event processing method provided in the embodiments of the present disclosure, the method can be applied to various application scenarios in various application fields such as brain simulation, computer vision, target detection, and target tracking, to implement corresponding processing of the event stream and improve the computing efficiency.

[0128] Figure 3 A block diagram of an event processing device provided in the embodiments of the present disclosure is shown in FIG. 3.

[0129] Referring to Figure 3 The embodiments of the present disclosure provide an event processing device, which comprises:

[0130] An event stream acquisition module 31 is configured to acquire an event stream to be processed.

[0131] an event detection module 32, configured to perform event detection on the event stream to determine an event detection result of the event stream at present;

[0132] a step adjustment module 33, configured to adjust a time step for processing the event stream according to the event detection result;

[0133] an event stream processing module 34, configured to process the event stream by using the time step to obtain a processing result of the event stream.

[0134] In some possible implementation manners, the event detection result includes whether event information is detected in the event stream, and the step adjustment module is configured to: in a case where the event information is detected in the event stream, adjust the time step to a second time step, the second time step being smaller than an initial first time step.

[0135] In some possible implementation manners, the event detection result includes whether event information is detected in the event stream, and the step adjustment module is configured to: in a case where the event information is not detected in the event stream and the current time step is a second time step, adjust the time step to the initial first time step, the second time step being smaller than the first time step.

[0136] In some possible implementation manners, the event stream includes a plurality of regions, and the event detection result includes whether event information is detected in each region of the event stream, and the step adjustment module is configured to: in a case where there is a target region in the plurality of regions, adjust a time step of the target region to a second time step, the target region being a region in which event information is detected, and the second time step being smaller than an initial first time step.

[0137] In some possible implementation manners, the event detection result includes whether event information is detected in the event stream and an event density of the event stream, and the step adjustment module includes an adjustment sub-module configured to: in a case where the event information is detected in the event stream, adjust the time step according to the event density,

[0138] The adjustment sub-module is configured to: in a case where the event density is greater than or equal to a first density threshold and smaller than or equal to a second density threshold, adjust the time step to a second time step, the second density threshold being greater than the first density threshold, and the second time step being smaller than an initial first time step; or in a case where the event density is greater than the second density threshold, adjust the time step to a third time step, the third time step being smaller than the second time step.

[0139] In some possible implementation manners, the event detection module is configured to: perform event detection on the event stream, determine a current event density of the event stream; and determine that the event detection result is that event information is detected in the event stream, in a case where the event density is greater than or equal to a third density threshold.

[0140] In some possible implementation manners, the event detection module is configured to: divide the event stream into a plurality of event frames according to a preset time length; and perform event detection on a current event frame of the event stream by using an event detection network, to obtain the event detection result.

[0141] In some possible implementation manners, the event stream processing module is configured to: perform brain simulation processing on the event stream by using the pulse neural network according to the time step, to obtain response information of the pulse neural network to the event stream, and the processing result includes the response information, wherein a differential expression of a neuron parameter in the brain simulation is adjusted according to the time step.

[0142] In some possible implementation manners, the event stream is an event stream of a target scene collected by an event collection device, and the event stream is used to represent brightness change information of the target scene.

[0143] Figure 4 A block diagram of an electronic device is provided for the embodiments of the present disclosure.

[0144] With reference to Figure 4 The embodiments of the present disclosure provide an electronic device, which includes: at least one processor 501; and a memory 502 connected with the at least one processor 501 in communication; wherein the memory 502 stores one or more computer programs executable by the at least one processor 501, and the one or more computer programs are executed by the at least one processor 501 to enable the at least one processor 501 to perform the event processing method described above.

[0145] In addition, the embodiments of the present disclosure further provide a computer readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the event processing method described above.

[0146] Those of ordinary skill in the art will realize and understand that all or some of the steps in the methods disclosed above and the functional modules / units in the systems and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on computer-readable media, which can include computer storage media (or non-transitory media) and communication media (or transitory media). As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, it is common technical knowledge that communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0147] Example embodiments have been disclosed herein and, although the use of specific terms is expressly used herein, they are intended in a generic sense only and are not intended to limit the scope of the present disclosure. In some instances, it will be apparent to those skilled in the art that features, characteristics or / and elements described in connection with a particular embodiment can be used in conjunction with other embodiments unless otherwise explicitly stated. As such, those skilled in the art will appreciate that various changes can be made in form and detail without departing from the scope of the disclosure as set forth in the appended claims.

Claims

1. An event processing method, characterized by, The method comprises: acquiring an event stream to be processed; performing event detection on the event stream to determine a current event detection result of the event stream; adjusting a time step for processing the event stream according to the event detection result; processing the event stream by using the time step to obtain a processing result of the event stream; the processing of the event stream by using the time step to obtain the processing result of the event stream comprises: processing the event stream by using the time step through a spiking neural network to obtain response information of the spiking neural network to the event stream, wherein the processing result comprises the response information, and a differential expression of a neuron parameter in brain simulation is adjusted according to the time step; the spiking neural network is used to perform any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks.

2. The method of claim 1, wherein, The event detection result comprises information about whether an event is detected in the event stream, wherein the adjusting of the time step for processing the event stream according to the event detection result comprises: in a case where the event information is detected in the event stream, the time step is adjusted to a second time step, and the second time step is smaller than an initial first time step.

3. The method of claim 1, wherein, The event detection result comprises information about whether an event is detected in the event stream, wherein the adjusting of the time step for processing the event stream according to the event detection result comprises: in a case where the event information is not detected in the event stream and the current time step is a second time step, the time step is adjusted to an initial first time step, and the second time step is smaller than the first time step.

4. The method of claim 1, wherein, The event stream comprises a plurality of regions, and the event detection result comprises information about whether an event is detected in each region of the event stream, wherein the adjusting of the time step for processing the event stream according to the event detection result comprises: in a case where there is a target region in the plurality of regions, the time step of the target region is adjusted to a second time step, the target region is a region where the event information is detected, and the second time step is smaller than an initial first time step.

5. The method of claim 1, wherein, The event detection result comprises information about whether an event is detected in the event stream and an event density of the event stream, wherein the adjusting of the time step for processing the event stream according to the event detection result comprises adjusting the time step according to the event density in a case where the event information is detected in the event stream, wherein the adjusting of the time step according to the event density comprises: in a case where the event density is greater than or equal to a first density threshold and less than or equal to a second density threshold, the time step is adjusted to a second time step, the second density threshold is greater than the first density threshold, and the second time step is smaller than an initial first time step; or in a case where the event density is greater than the second density threshold, the time step is adjusted to a third time step, and the third time step is smaller than the second time step.

6. The method according to any one of claims 1 to 5, characterized in that, The event detection on the event stream to determine the event detection result of the event stream at present includes: The event detection on the event stream to determine the event density of the event stream at present; In a case where the event density is greater than or equal to a third density threshold, the event detection result is determined as: event information is detected in the event stream.

7. The method according to any one of claims 1 to 5, characterized in that, The event detection on the event stream to determine the event detection result of the event stream at present includes: According to a preset time length, the event stream is divided into a plurality of event frames; The current event frame of the event stream is detected by an event detection network to obtain the event detection result.

8. The method according to any one of claims 1-5, characterized in that, The event stream is an event stream of a target scene collected by an event collection device, and the event stream is used to represent brightness change information of the target scene.

9. An event processing apparatus, characterized by comprising: Comprise: An event stream acquisition module configured to acquire an event stream to be processed; An event detection module configured to perform event detection on the event stream to determine an event detection result of the event stream at present; A step adjustment module configured to adjust a time step for processing the event stream according to the event detection result; An event stream processing module configured to process the event stream using the time step to obtain a processing result of the event stream; The processing of the event stream using the time step to obtain the processing result of the event stream includes: The event stream is processed by a spiking neural network using the time step to obtain response information of the spiking neural network to the event stream, and the processing result includes the response information, wherein a differential expression of a neuron parameter in brain simulation is adjusted according to the time step. The spiking neural network is used to perform any one of an image processing task, a speech processing task, a text processing task, and a video processing task.

10. An electronic device, comprising: Comprise: At least one processor; And A memory in communication connection with the at least one processor; wherein The memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the event processing method of any one of claims 1-8.

11. A computer readable medium having stored thereon a computer program, characterized in that The computer program, when executed by the processor, implements the event processing method of any one of claims 1-8.

Images