Event position identification method and device

Abstract

The invention discloses an event position identification method and device, and the method comprises the steps: carrying out the cutting processing of an event stream, and obtaining event voxels; converting the event stream into an image frame; taking the event voxel as time information of the event stream, taking the image frame as spatial information of the event stream, and performing feature extraction on the time information and the spatial information to obtain time features and spatial features of the event stream; determining sharing features according to the time features and the space features; performing feature enhancement processing and feature fusion on the time features, the spatial features and the shared features, and endowing different weights; and transferring the corresponding weight to the spiking neural network to optimize the spiking neural network, performing feature extraction on the event flow based on the optimized spiking neural network, matching the extracted features with the features in the existing event position database, and determining the position information of the event according to the matching result. According to the method, the event stream can be described more finely, and the visual position identification precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an event location recognition method and device. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section.

[0003] Visual location recognition is a key sub-problem of autonomous navigation for mobile robots. It aims to help robots relocate by confirming whether the current image query sequence exists in a known environmental image database. Visual location recognition can help mobile robots or autonomous unmanned systems locate themselves in environments where GPS is not available. However, the current visual location recognition method based on event cameras cannot describe the event flow more finely based on multi-dimensional information, resulting in low accuracy of visual location recognition. Therefore, an event location recognition method is urgently needed to solve the above problems. Summary of the invention

[0004] The embodiment of the present invention provides an event position recognition method for more finely describing an event flow and improving the accuracy of visual position recognition. The method includes:

[0005] In the process of visual position recognition, the event stream is processed as follows based on the timestamp of the event stream: the event stream is cut according to the timestamp of the event stream, and the event stream segments obtained by cutting are represented in the form of voxel grids to obtain event voxels; the timestamp of the event stream is converted into the grayscale of the pixel, and the event stream is converted into an image frame based on the grayscale of the pixel;

[0006] Taking the event voxel as the time information of the event stream and the image frame as the space information of the event stream, extracting the time information and the space information to obtain the time feature and the space feature of the event stream; determining the shared feature according to the time feature and the space feature; the shared feature is the same feature in the time feature and the space feature;

[0007] Using a feature enhancement algorithm, the temporal feature, the spatial feature, and the shared feature are enhanced to obtain the processed temporal enhancement feature, the spatial enhancement feature, and the shared enhancement feature;

[0008] Perform feature fusion on the temporal enhancement features, spatial enhancement features and shared enhancement features, and assign different weights to the temporal enhancement features, spatial enhancement features and shared features based on the attention mechanism after feature fusion;

[0009] The weights corresponding to the time enhancement features, space enhancement features and shared features are transferred to the spiking neural network to optimize the spiking neural network. The event stream is extracted based on the optimized spiking neural network. The extracted features are matched with the features in the existing event location database, and the location information of the event is determined according to the matching results.

[0010] The embodiment of the present invention further provides an event position recognition device for more finely describing the event flow and improving the accuracy of visual position recognition. The device includes:

[0011] The event stream processing module performs the following processing on the event stream based on the timestamp of the event stream during the visual position recognition process: cutting the event stream according to the timestamp of the event stream, representing the event stream segments obtained by cutting in the form of voxel grids to obtain event voxels; converting the timestamp of the event stream into the grayscale of pixels, and converting the event stream into image frames based on the grayscale of pixels;

[0012] A feature extraction module is used to extract features of the time information and the space information by taking the event voxel as the time information of the event flow and the image frame as the space information of the event flow, so as to obtain the time features and space features of the event flow; and to determine the shared features according to the time features and the space features; the shared features are the same features in the time features and the space features;

[0013] A feature enhancement processing module is used to use a feature enhancement algorithm to perform feature enhancement processing on the time feature, the space feature, and the shared feature to obtain the processed time enhancement feature, the space enhancement feature, and the shared enhancement feature;

[0014] A feature fusion module is used to fuse the temporal enhancement features, spatial enhancement features and shared enhancement features. After the feature fusion, different weights are assigned to the temporal enhancement features, spatial enhancement features and shared features based on the attention mechanism.

[0015] The event location determination module is used to migrate the weights corresponding to the time enhancement features, space enhancement features and shared features to the pulse neural network to optimize the pulse neural network, extract features of the event stream based on the optimized pulse neural network, match the extracted features with the features in the existing event location database, and determine the location information of the event based on the matching results.

[0016] An embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned event location identification method when executing the computer program.

[0017] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned event location identification method is implemented.

[0018] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned event location identification method is implemented.

[0019] In the embodiment of the present invention, during the visual position recognition process, the event stream is processed as follows based on the timestamp of the event stream: the event stream is cut according to the timestamp of the event stream, and the event stream segments obtained by cutting are represented in the form of voxel grids to obtain event voxels; the timestamp of the event stream is converted into the grayscale of pixels, and the event stream is converted into image frames based on the grayscale of pixels; the event voxels are used as the time information of the event stream, and the image frames are used as the spatial information of the event stream, and the time information and spatial information are extracted to obtain the time features and spatial features of the event stream; based on the time features and spatial features, shared features are determined; the shared features are the same features in the time features and the spatial features; and Using a feature enhancement algorithm, the temporal features, spatial features, and shared features are enhanced to obtain the processed temporal enhancement features, spatial enhancement features, and shared enhancement features; the temporal enhancement features, spatial enhancement features, and shared enhancement features are fused, and different weights are assigned to the temporal enhancement features, spatial enhancement features, and shared features based on the attention mechanism after feature fusion; the weights corresponding to the temporal enhancement features, spatial enhancement features, and shared features are transferred to the pulse neural network to optimize the pulse neural network, and feature extraction is performed on the event stream based on the optimized pulse neural network, and the extracted features are matched with the features in the existing event location database, and the location information of the event is determined according to the matching results. In the above process, the embodiment of the present invention converts the event stream into event voxels and image frames, determines the features of the event stream from the temporal dimension and the spatial dimension, thereby more finely describing the event stream and improving the accuracy of visual location recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0021] Figure 1 is a flow chart of an event location identification method according to an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of obtaining shared features in an embodiment of the present invention;

[0023] Figure 3 This is a flow chart of feature fusion in an embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of an event location identification device in an embodiment of the present invention;

[0025] Figure 5 Schematic diagram of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0027] Figure 1 Flow chart of the event location identification method in an embodiment of the present invention, the method includes:

[0028] Step 101, during the visual position recognition process, the event stream is processed as follows based on the timestamp of the event stream: the event stream is segmented according to the timestamp of the event stream, and the segmented event stream is represented in the form of a voxel grid to obtain an event voxel; the timestamp of the event stream is converted into a grayscale of a pixel, and the event stream is converted into an image frame based on the grayscale of the pixel;

[0029] Step 102, taking the event voxel as the time information of the event stream and the image frame as the space information of the event stream, extracting the time information and the space information to obtain the time feature and the space feature of the event stream; determining the shared feature according to the time feature and the space feature; the shared feature is the same feature in the time feature and the space feature;

[0030] Step 103, using a feature enhancement algorithm to perform feature enhancement processing on the time feature, the space feature, and the shared feature to obtain processed time enhancement features, space enhancement features, and shared enhancement features;

[0031] Step 104, performing feature fusion on the temporal enhancement feature, the spatial enhancement feature and the shared enhancement feature, and assigning different weights to the temporal enhancement feature, the spatial enhancement feature and the shared feature based on the attention mechanism after the feature fusion;

[0032] Step 105, the weights corresponding to the time enhancement features, space enhancement features and shared features are transferred to the pulse neural network to optimize the pulse neural network, and the event stream is extracted based on the optimized pulse neural network. The extracted features are matched with the features in the existing event location database, and the location information of the event is determined according to the matching results.

[0033] Each step is described in detail below.

[0034] In step 101, during the visual position recognition process, the event stream is processed as follows based on the timestamp of the event stream: the event stream is cut according to the timestamp of the event stream, and the event stream segments obtained by cutting are represented in the form of voxel grids to obtain event voxels; the timestamp of the event stream is converted into the grayscale of the pixel, and the event stream is converted into an image frame based on the grayscale of the pixel.

[0035] In a specific embodiment, the event stream contains rich temporal information and spatial information, and the event stream is converted into two representation forms: one is the form of voxel grid, and the other is the form of image frame, so as to obtain the temporal information and spatial information of the event stream from the temporal dimension and the spatial dimension. Therefore, according to the timestamp, the event stream is processed in different dimensions, thereby avoiding the problem of directly transmitting the event stream to the pulse neural network, which leads to high network training costs. In addition, the present invention processes the event stream in different dimensions, which is different from the traditional method in which the event stream is highly sparse in space at a single time step, limiting the ability of the pulse neural network to effectively encode spatial neighborhood information. In order to improve the accuracy of visual position recognition, the event stream cannot be described by relying solely on a single-dimensional representation, and a more refined description of the event stream based on multi-dimensional information is required.

[0036] In step 102, event voxels are used as time information of the event stream, and image frames are used as spatial information of the event stream. Feature extraction is performed on the time information and spatial information to obtain the time features and spatial features of the event stream. Based on the time features and spatial features, shared features are determined; the shared features are the same features in the time features and spatial features.

[0037] Figure 2 This is a schematic diagram of obtaining shared features in an embodiment of the present invention. In a specific embodiment, in view of the fact that the time features and spatial features in the two representation forms of the event stream contain the same information and different information, in order to ensure that the extracted features meet actual requirements, the present invention designs a shared and specific feature extractor based on the characteristics of the pulse neural network. Figure 2 The features circled in the middle are the same features in the temporal features and the spatial features. The shared features are obtained by sharing the same features in the temporal features and the spatial features extracted by a specific feature extractor. Figure 2The multiplication logic rule is: 0×0=0, 0×1=0, 1×0=0, 1×1=1.

[0038] In one embodiment, after determining the shared features according to the time features and the space features, the method further includes:

[0039] According to the time characteristics, space characteristics and shared characteristics, the distinguishing characteristics between the time characteristics and the space characteristics are determined.

[0040] In one embodiment, the distinguishing feature between the time feature and the space feature is determined according to the following formula:

[0041] F 1_specific =F1-F share

[0042] F 2_specific =F2-F share

[0043] Among them, F1 is the time feature, F2 is the spatial feature, and F 1_specific is the distinguishing feature of temporal features relative to spatial features, F 2_specific is the distinguishing feature of spatial features relative to temporal features, F share is the shared feature, F share =F1×F2.

[0044] In a specific embodiment, in order to extract specific features of different representation forms of event streams, it can be achieved by subtracting shared features from their respective basic features (basic features include: time features or space features), thereby obtaining specific features of different representation forms. Since the values ​​in the matrix are only 0 or 1, the result after subtraction still meets the expression requirements of specific features, so the tensor after subtraction can be used as a specific feature. After determining the specific features, the shared features, the specific features of the time dimension, and the specific features of the space dimension are globally averaged pooled based on the SNN pulse neural network, and the processing results are L2 normalized to generate three corresponding basic descriptors. Among them, one descriptor contains shared features, and the other two descriptors contain specific features of the time dimension and specific features of the space dimension, respectively. Since different descriptors contain independent feature information, this feature diversity provides richer feature support for cross-descriptor fusion, thereby improving the overall performance of the pulse neural network.

[0045] In step 103, a feature enhancement algorithm is used to perform feature enhancement processing on the temporal features, spatial features, and shared features to obtain processed temporal enhancement features, spatial enhancement features, and shared enhancement features; in step 104, feature fusion is performed on the temporal enhancement features, spatial enhancement features, and shared enhancement features. After feature fusion, different weights are assigned to the temporal enhancement features, spatial enhancement features, and shared features based on the attention mechanism.

[0046] Figure 3 This is a flow chart of feature fusion in an embodiment of the present invention. In one embodiment, feature fusion is performed on the time enhancement feature, the space enhancement feature and the shared enhancement feature. After the feature fusion, different weights are assigned to the time enhancement feature, the space enhancement feature and the shared feature based on the attention mechanism, including:

[0047] Step 301, concatenating and fusing the tensor of the temporal enhancement feature, the tensor of the spatial enhancement feature, and the tensor of the shared enhancement feature in a preset channel dimension to obtain a fused feature tensor;

[0048] Step 302, after feature fusion, different weights are assigned to the temporal enhancement features, spatial enhancement features and shared features based on the attention mechanism according to the fused feature tensor.

[0049] In a specific embodiment, the three basic descriptors obtained are added element by element to generate a fused feature tensor. Then, this tensor is input into an SNN pulse neural network consisting of two fully connected layers (the number of neurons in the input layer is 256, the number of neurons in the hidden layer is 64, and the number of neurons in the output layer is 3). The input size of the SNN pulse neural network is the number of channels of the fused feature tensor, and the output size is the number of basic descriptors. Subsequently, the output is averaged in the time dimension (T), the pulse excitation frequency is calculated, and it is normalized by the softmax function to generate the weight of each basic descriptor, that is, the weight of the time enhancement feature, the weight of the space enhancement feature, and the weight of the shared feature.

[0050] In step 105, the weights corresponding to the time enhancement features, space enhancement features and shared features are transferred to the pulse neural network to optimize the pulse neural network, and the event stream is feature extracted based on the optimized pulse neural network. The extracted features are matched with the features in the existing event location database, and the location information of the event is determined according to the matching results.

[0051] In a specific embodiment, the weights and biases corresponding to the time enhancement features, space enhancement features and shared features are migrated to the spiking neural network to optimize the spiking neural network. The specific implementation process is that the weights and biases are normalized using the maximum activation value in the channel dimension of each layer, and the normalized weights and biases are migrated to the spiking neural network to optimize the spiking neural network.

[0052] The present invention also provides an event location identification device, as described in the following embodiments. Since the principle of solving the problem by the device is similar to that of the event location identification method, the implementation of the device can refer to the implementation of the event location identification method, and the repeated parts will not be repeated.

[0053] Figure 4 This is a schematic diagram of an event location identification device in an embodiment of the present invention, the device comprising:

[0054] The event stream processing module 401 performs the following processing on the event stream based on the timestamp of the event stream during the visual position recognition process: cutting the event stream according to the timestamp of the event stream, representing the event stream segments obtained by cutting in the form of voxel grids to obtain event voxels; converting the timestamp of the event stream into the grayscale of pixels, and converting the event stream into image frames based on the grayscale of pixels;

[0055] The feature extraction module 402 is used to extract features of the time information and the space information by taking the event voxel as the time information of the event flow and the image frame as the space information of the event flow, so as to obtain the time feature and the space feature of the event flow; and determine the shared feature according to the time feature and the space feature; the shared feature is the same feature in the time feature and the space feature;

[0056] The feature enhancement processing module 403 is used to use a feature enhancement algorithm to perform feature enhancement processing on the time feature, the space feature, and the shared feature to obtain the processed time enhancement feature, the space enhancement feature, and the shared enhancement feature;

[0057] A feature fusion module 404 is used to fuse the temporal enhancement feature, the spatial enhancement feature and the shared enhancement feature, and assign different weights to the temporal enhancement feature, the spatial enhancement feature and the shared feature based on an attention mechanism after the feature fusion;

[0058] The event location determination module 405 is used to transfer the weights corresponding to the time enhancement features, space enhancement features and shared features to the pulse neural network to optimize the pulse neural network, extract features of the event stream based on the optimized pulse neural network, match the extracted features with the features in the existing event location database, and determine the location information of the event based on the matching results.

[0059] In one embodiment, the feature extraction module 402 is further used to:

[0060] According to the time characteristics, space characteristics and shared characteristics, the distinguishing characteristics between the time characteristics and the space characteristics are determined.

[0061] In one embodiment, the feature extraction module 402 is specifically used to:

[0062] The distinguishing features between temporal features and spatial features are determined according to the following formula:

[0063] F 1_specific =F1-F share

[0064] F 2_specific =F2-F share

[0065] Among them, F1 is the time feature, F2 is the spatial feature, and F 1_specific is the distinguishing feature of temporal features relative to spatial features, F 2_specific is the distinguishing feature of spatial features relative to temporal features, F share is the shared feature, F share =F1×F2.

[0066] In one embodiment, the feature fusion module 404 is specifically used to:

[0067] The tensor of the temporal enhancement feature, the tensor of the spatial enhancement feature, and the tensor of the shared enhancement feature are concatenated and fused in the preset channel dimension to obtain a fused feature tensor;

[0068] After feature fusion, different weights are assigned to temporal enhancement features, spatial enhancement features, and shared features based on the attention mechanism according to the fused feature tensor.

[0069] An embodiment of the present invention further provides a computer device, Figure 5 Schematic diagram of a computer device in an embodiment of the present invention. The computer device 500 includes a memory 510, a processor 520, and a computer program 530 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 530, the above-mentioned event location identification method is implemented.

[0070] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned event location identification method is implemented.

[0071] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned event location identification method is implemented.

[0072] In the embodiment of the present invention, during the visual position recognition process, the event stream is processed as follows based on the timestamp of the event stream: the event stream is cut according to the timestamp of the event stream, and the event stream segments obtained by cutting are represented in the form of voxel grids to obtain event voxels; the timestamp of the event stream is converted into the grayscale of pixels, and the event stream is converted into image frames based on the grayscale of pixels; the event voxels are used as the time information of the event stream, and the image frames are used as the spatial information of the event stream, and the time information and spatial information are extracted to obtain the time features and spatial features of the event stream; based on the time features and spatial features, shared features are determined; the shared features are the same features in the time features and the spatial features; and Using a feature enhancement algorithm, the temporal features, spatial features, and shared features are enhanced to obtain the processed temporal enhancement features, spatial enhancement features, and shared enhancement features; the temporal enhancement features, spatial enhancement features, and shared enhancement features are fused, and different weights are assigned to the temporal enhancement features, spatial enhancement features, and shared features based on the attention mechanism after feature fusion; the weights corresponding to the temporal enhancement features, spatial enhancement features, and shared features are transferred to the pulse neural network to optimize the pulse neural network, and feature extraction is performed on the event stream based on the optimized pulse neural network, and the extracted features are matched with the features in the existing event location database, and the location information of the event is determined according to the matching results. In the above process, the embodiment of the present invention converts the event stream into event voxels and image frames, determines the features of the event stream from the temporal dimension and the spatial dimension, thereby more finely describing the event stream and improving the accuracy of visual location recognition.

[0073] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0075] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0077] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying an event location, characterized in that: include: In the process of visual position recognition, the event stream is processed as follows based on the timestamp of the event stream: the event stream is cut according to the timestamp of the event stream, and the event stream segments obtained by cutting are represented in the form of voxel grids to obtain event voxels; Convert the timestamp of the event stream into the grayscale of the pixel, and convert the event stream into an image frame based on the grayscale of the pixel; Taking the event voxel as the time information of the event stream and the image frame as the space information of the event stream, extracting the time information and the space information to obtain the time feature and the space feature of the event stream; determining the shared feature according to the time feature and the space feature; the shared feature is the same feature in the time feature and the space feature; Using a feature enhancement algorithm, the temporal feature, the spatial feature, and the shared feature are enhanced to obtain the processed temporal enhancement feature, the spatial enhancement feature, and the shared enhancement feature; Perform feature fusion on the temporal enhancement features, spatial enhancement features and shared enhancement features, and assign different weights to the temporal enhancement features, spatial enhancement features and shared features based on the attention mechanism after feature fusion; The weights corresponding to the time enhancement features, space enhancement features and shared features are transferred to the spiking neural network to optimize the spiking neural network. The event stream is extracted based on the optimized spiking neural network. The extracted features are matched with the features in the existing event location database, and the location information of the event is determined according to the matching results.

2. The method according to claim 1, characterized in that After determining the shared features based on temporal and spatial features, it also includes: According to the time characteristics, space characteristics and shared characteristics, the distinguishing characteristics between the time characteristics and the space characteristics are determined.

3. The method according to claim 2, characterized in that The distinguishing features between temporal features and spatial features are determined according to the following formula: F 1_specific =F1-F share F 2_specific =F2-F share Among them, F1 is the time feature, F2 is the spatial feature, and F 1_specific is the distinguishing feature of temporal features relative to spatial features, F 2_specific is the distinguishing feature of spatial features relative to temporal features, F share is the shared feature, F share =F1×F2.

4. The method according to claim 1, characterized in that The temporal enhancement features, spatial enhancement features and shared enhancement features are fused. After the features are fused, different weights are assigned to the temporal enhancement features, spatial enhancement features and shared features based on the attention mechanism, including: The tensor of the temporal enhancement feature, the tensor of the spatial enhancement feature, and the tensor of the shared enhancement feature are concatenated and fused in the preset channel dimension to obtain a fused feature tensor; After feature fusion, different weights are assigned to temporal enhancement features, spatial enhancement features, and shared features based on the attention mechanism according to the fused feature tensor.

5. An event location recognition device, characterized in that: include: The event stream processing module performs the following processing on the event stream based on the timestamp of the event stream during the visual position recognition process: cutting the event stream according to the timestamp of the event stream, representing the event stream segments obtained by cutting in the form of voxel grids, and obtaining event voxels; Convert the timestamp of the event stream into the grayscale of the pixel, and convert the event stream into an image frame based on the grayscale of the pixel; A feature extraction module is used to extract features of the time information and the space information by taking the event voxel as the time information of the event flow and the image frame as the space information of the event flow, so as to obtain the time features and space features of the event flow; and to determine the shared features according to the time features and the space features; the shared features are the same features in the time features and the space features; A feature enhancement processing module is used to use a feature enhancement algorithm to perform feature enhancement processing on the time feature, the space feature, and the shared feature to obtain the processed time enhancement feature, the space enhancement feature, and the shared enhancement feature; A feature fusion module is used to fuse the temporal enhancement features, spatial enhancement features and shared enhancement features. After the feature fusion, different weights are assigned to the temporal enhancement features, spatial enhancement features and shared features based on the attention mechanism. The event location determination module is used to migrate the weights corresponding to the time enhancement features, space enhancement features and shared features to the pulse neural network to optimize the pulse neural network, extract features of the event stream based on the optimized pulse neural network, match the extracted features with the features in the existing event location database, and determine the location information of the event based on the matching results.

6. The device according to claim 5, characterized in that The feature extraction module is also used to: According to the time characteristics, space characteristics and shared characteristics, the distinguishing characteristics between the time characteristics and the space characteristics are determined.

7. The device according to claim 6, characterized in that Feature extraction module, specifically used for: The distinguishing features between temporal features and spatial features are determined according to the following formula: F 1_specific =F1-F share F 2_specific =F2-F share Among them, F1 is the time feature, F2 is the spatial feature, and F 1_specific is the distinguishing feature of temporal features relative to spatial features, F 2_specific is the distinguishing feature of spatial features relative to temporal features, F share is the shared feature, F share =F1×F2.

8. The device according to claim 5, characterized in that Feature fusion module, specifically used for: The tensor of the temporal enhancement feature, the tensor of the spatial enhancement feature, and the tensor of the shared enhancement feature are concatenated and fused in the preset channel dimension to obtain a fused feature tensor; After feature fusion, different weights are assigned to temporal enhancement features, spatial enhancement features, and shared features based on the attention mechanism according to the fused feature tensor.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images