Camera repositioning method and device based on event camera
Through the camera relocation method based on event cameras, the position correction is performed using event streams and ESPR-Net neural networks, and the problems of inaccurate and poor robustness of camera positioning in the prior art are solved, and higher positioning accuracy and robustness are achieved.
Patent Information
- Application Number
- CN202411564380.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2025-04-08
AI Technical Summary
Existing camera repositioning methods have degraded performance in scenarios such as fast motion and drastic lighting changes, resulting in inaccurate camera positioning and poor robustness.
The camera repositioning method based on the event camera is adopted, by obtaining the event stream collected by the event camera, searching the nearest event stream, coarse positioning and refined pose correction, the relative pose is output using the ESPR-Net neural network, and the pose transformation matrix is calculated to correct the camera pose.
Improves the accuracy and robustness of camera positioning, especially in scenarios with rapid movement and drastic light changes.
Smart Images
Figure CN119784830A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of visual design technology, and in particular to a camera repositioning method and device based on an event camera. Background Art
[0002] This section is intended to provide a background or context to the embodiments of the invention recited in the claims. No admission is made that the description herein is prior art by inclusion in this section.
[0003] Camera relocalization is the process of inferring the current position and orientation of a camera based on the observed scene, and has a wide range of applications in robotics, virtual reality (VR), and augmented reality (AR). Camera relocalization plays a vital role in robotic navigation, providing an initial pose based on the 3D environment when pose estimation is lost, thereby facilitating subsequent pose tracking. Current methods mainly rely on RGB images captured by standard cameras to achieve camera relocalization. While these methods perform well, the performance of standard cameras degrades significantly in challenging scenes such as fast motion and drastic lighting changes. However, current learning-based camera relocalization methods all use neural networks to encode the structure of the entire scene into network parameters, thereby achieving a direct mapping from RGB images to camera poses. However, these existing methods suffer from inaccurate camera localization, resulting in poor robustness of localization results. Summary of the invention
[0004] The embodiment of the present invention provides a camera repositioning method based on an event camera, which is used to improve the accuracy of camera positioning and the robustness of positioning results. The method includes:
[0005] Get the event stream collected by the event camera as the query event stream;
[0006] According to the query event stream, the neighboring event streams of the query event stream are retrieved from the database;
[0007] The query event stream is used to roughly locate the neighboring event stream, and the camera pose of the neighboring event stream is determined according to the rough positioning result;
[0008] Inputting the query event stream and the neighbor event stream into the trained ESPR-Net neural network, and outputting the relative position of the query event stream and the neighbor event stream; the ESPR-Net neural network is trained based on the historical query event stream, the historical neighbor event stream, and the relative position of the historical query event stream and the historical neighbor event stream;
[0009] Calculate the pose transformation matrix of the query event stream according to the relative poses of the query event stream and the neighboring event stream and the camera pose of the neighboring event stream;
[0010] According to the pose transformation matrix of the query event stream, the camera pose of the query event stream is corrected to obtain a corrected camera pose of the query event stream.
[0011] The embodiment of the present invention further provides a camera repositioning device based on an event camera, which is used to improve the accuracy of camera positioning and the robustness of positioning results. The device includes:
[0012] The query event stream acquisition module is used to obtain the event stream collected by the event camera as the query event stream;
[0013] A neighbor event stream retrieval module is used to retrieve neighbor event streams of the query event stream from the database according to the query event stream;
[0014] A neighbor event stream camera pose determination module is used to use the query event stream to roughly locate the neighbor event stream, and determine the camera pose of the neighbor event stream according to the rough positioning result;
[0015] A relative posture acquisition module is used to input the query event stream and the neighbor event stream into the trained ESPR-Net neural network, and output the relative posture of the query event stream and the neighbor event stream; the ESPR-Net neural network is trained based on the historical query event stream, the historical neighbor event stream, and the relative posture of the historical query event stream and the historical neighbor event stream;
[0016] A pose transformation matrix calculation module, used to calculate the pose transformation matrix of the query event stream according to the relative poses between the query event stream and the neighboring event stream and the camera pose of the neighboring event stream;
[0017] The query event stream camera pose acquisition module is used to correct the camera pose of the query event stream according to the pose transformation matrix of the query event stream to obtain the corrected camera pose of the query event stream.
[0018] An embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned camera repositioning method based on event camera when executing the computer program.
[0019] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the camera repositioning method based on an event camera is implemented.
[0020] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the camera repositioning method based on an event camera is implemented.
[0021] In an embodiment of the present invention, an event stream collected by an event camera is obtained as a query event stream; based on the query event stream, a neighboring event stream of the query event stream is retrieved from a database; the query event stream is used to roughly locate the neighboring event stream, and the camera pose of the neighboring event stream is determined based on the rough positioning result; the query event stream and the neighboring event stream are input into a trained ESPR-Net neural network, and the relative pose of the query event stream and the neighboring event stream is output; the ESPR-Net neural network is trained based on historical query event streams, historical neighboring event streams, and the relative pose of historical query event streams and historical neighboring event streams; the pose transformation matrix of the query event stream is calculated based on the relative pose of the query event stream and the neighboring event stream and the camera pose of the neighboring event stream; the camera pose of the query event stream is corrected based on the pose transformation matrix of the query event stream to obtain the corrected camera pose of the query event stream. In the above process, the embodiment of the present invention retrieves the neighboring event stream in the rough positioning stage and obtains the corresponding camera pose. In the pose refinement stage, the relative poses of the query event stream and the neighboring event streams are obtained through the ESPR-Net neural network, and the pose transformation is performed based on the relative pose to determine the camera pose of the query event stream, thereby improving the accuracy of camera positioning and the robustness of the positioning results. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0023] Figure 1 is a flow chart of a camera repositioning method based on an event camera in an embodiment of the present invention;
[0024] Figure 2 A flowchart of retrieving a neighbor event stream in an embodiment of the present invention;
[0025] Figure 3 This is a flow chart of preprocessing an event stream in an embodiment of the present invention;
[0026] Figure 4 It is an overall framework flow chart of an embodiment of the present invention;
[0027] Figure 5 Schematic diagram of a camera repositioning device based on an event camera in an embodiment of the present invention. DETAILED DESCRIPTION
[0028] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.
[0029] Figure 1 : is a flowchart of a camera repositioning method based on an event camera in an embodiment of the present invention, the method comprising:
[0030] Step 101, obtaining an event stream collected by an event camera as a query event stream;
[0031] Step 102, according to the query event flow, retrieve the neighboring event flow of the query event flow from the database;
[0032] Step 103, using the query event stream to roughly locate the neighboring event stream, and determining the camera pose of the neighboring event stream according to the rough positioning result;
[0033] Step 104, inputting the query event stream and the neighbor event stream into the trained ESPR-Net neural network, and outputting the relative position and posture of the query event stream and the neighbor event stream; the ESPR-Net neural network is trained based on the historical query event stream, the historical neighbor event stream, and the relative position and posture of the historical query event stream and the historical neighbor event stream;
[0034] Step 105, calculating the pose transformation matrix of the query event stream according to the relative poses of the query event stream and the neighboring event stream and the camera pose of the neighboring event stream;
[0035] Step 106, correcting the camera pose of the query event stream according to the pose transformation matrix of the query event stream to obtain a corrected camera pose of the query event stream.
[0036] Each step is described in detail below.
[0037] In step 101, an event stream collected by an event camera is obtained as a query event stream.
[0038] In a specific embodiment, the query event stream is a new event stream re-photographed in a known environment, which is independent of the database. The database D is constructed to store the prior environment information and the related camera poses:
[0039] D={E,F,T}
[0040] E,F,T={E 1 ,...,E r},{F 1 ,...,F r}, {T 1 ,...,Tr}
[0041] Where E consists of r prior event streams, F represents the similarity feature vector of the event stream, and T represents the camera pose corresponding to each event stream in the world coordinate system.
[0042] In step 102, based on the query event stream, neighboring event streams of the query event stream are retrieved from a database.
[0043] Figure 2 This is a flow chart of retrieving neighbor event streams in an embodiment of the present invention. In one embodiment, according to a query event stream, retrieving neighbor event streams of the query event stream from a database includes:
[0044] Step 201, constructing a similarity feature vector of the query event stream according to the time information and spatial information of the query event stream;
[0045] Step 202: retrieve neighboring event flows of the query event flow from the database according to the similarity feature vector.
[0046] In a specific embodiment, in the coarse positioning stage, in order to retain the high-resolution temporal properties inherent in the event, the event stream is regarded as a pseudo 3D point cloud and encoded as a similarity feature, so as to accurately retrieve the neighboring event stream and camera pose from the known prior environmental information.
[0047] In step 103, the query event stream is used to roughly locate the neighboring event stream, and the camera pose of the neighboring event stream is determined according to the rough positioning result.
[0048] In a specific embodiment, the event stream E q The 2D spatial information and 1D temporal information are aggregated into a pseudo point cloud as the event cloud P q This method preserves the fine-grained temporal information of events and helps to extract deeper temporal features; the event cloud P is classified according to the camera resolution and timestamp. q Normalize; apply radius filtering to the normalized event cloud to remove noise; encode the normalized event cloud after noise removal as a similarity feature F q ; By querying the event stream E q The similarity feature F q , through the KNN algorithm, the query event stream E is retrieved from the database q The neighbor event stream E c and the corresponding camera pose T c , where the camera pose T of the neighboring event stream c as a rough pose estimate.
[0049] In step 104, the query event stream and the neighbor event stream are input into the trained ESPR-Net neural network, and the relative position of the query event stream and the neighbor event stream is output; the ESPR-Net neural network is trained based on the historical query event stream, the historical neighbor event stream, and the relative position of the historical query event stream and the historical neighbor event stream.
[0050] In a specific embodiment, the network architecture and training process of ESPR-Net include: ESPR-Net is composed of three modules: feature extractor, time aggregation module and pose regressor.
[0051] (1) Feature Extractor
[0052] The feature extractor adopts the RCNN architecture. CNN consists of 9 convolutional layers, each of which is connected to a ReLU activation function. The CNN structure can compress the input event voxels into a higher-dimensional feature representation while extracting fine spatial features. The sequential modeling capability of RNN is very suitable for pose estimation problems. Therefore, the present invention stacks two LSTM layers after CNN, and the hidden state of the first LSTM layer serves as the input of the second LSTM layer. The RNN structure effectively learns the temporal features of event voxels while reducing the feature dimension. In addition, an average pooling layer is added between CNN and RNN to reduce the number of network parameters in the network.
[0053] (2) Time aggregation
[0054] The temporal aggregation module consists of a multi-layer perceptron (MLP) with an input dimension of S and an output dimension of 1, and a Softmax layer, which aggregates features along the time axis and enhances spatial features related to subtle changes over time.
[0055] (3) Pose Regressor
[0056] The pose regressor consists of two serially connected linear layers with an input dimension of L and an output dimension of 7, which is used to regress the extracted features to a 6-DOF pose. The first three dimensions correspond to the translation components The last four dimensions represent the rotation components Finally, and Convert to relative pose As the final output of ESPR-Net.
[0057] (5) Training process
[0058] The present invention applies an L2 loss function between the final output and the ground truth:
[0059]
[0060] Where α=0.3 and β=0.7 are weight ratio coefficients used to balance the translation and rotation ratios.
[0061] Figure 3 This is a flow chart of preprocessing an event stream in an embodiment of the present invention. In one embodiment, before the query event stream and the neighbor event stream are input into the trained ESPR-Net neural network, the process includes:
[0062] Step 301, converting the query event stream and the neighbor event stream into query event voxels and neighbor event voxels respectively;
[0063] Step 302, in the time dimension, stack the query event voxel and the neighboring event voxel in pairs to obtain a synthetic event voxel;
[0064] Input the query event stream and neighbor event stream into the trained ESPR-Net neural network, including:
[0065] The synthetic event voxels are input into the trained ESPR-Net neural network.
[0066] In a specific embodiment, the goal of the posture refinement stage is to regress the relative postures of the query event stream and the neighboring event streams. Specifically, the following steps are included:
[0067] 1) Represent the query event stream and neighbor event stream as query event voxel V q , neighboring event voxel V c ;
[0068] 2) The event voxels are used to regress the relative positions of the two event streams through the ESPR-Net network of the RCNN architecture.
[0069] In order to express more subtle geometric information about the arrangement of the two event streams in the spatiotemporal dimension, the query event voxel and the neighboring event voxels are preprocessed before inputting the network. They are stacked in pairs according to the time dimension to obtain the final synthesized event voxel V qc In addition, this stacking method is beneficial for RNN sequence modeling, enabling the network to learn more subtle spatiotemporal features.
[0070] In step 105, the pose transformation matrix of the query event stream is calculated according to the relative pose between the query event stream and the neighboring event stream and the camera pose of the neighboring event stream; in step 106, the camera pose of the query event stream is corrected according to the pose transformation matrix of the query event stream to obtain the corrected camera pose of the query event stream.
[0071] In one embodiment, the pose transformation matrix of the query event stream is calculated according to the relative poses of the query event stream and the neighboring event stream and the camera pose of the neighboring event stream, including:
[0072] Using matrix multiplication, the pose transformation matrix of the query event stream is calculated according to the relative poses between the query event stream and the neighboring event streams and the camera poses of the neighboring event streams.
[0073] In one embodiment, the camera pose of the event stream is calculated according to the following pose transformation matrix:
[0074]
[0075] in, represents the relative position of the query event stream and the neighboring event stream, T c represents the camera pose of the neighboring event stream, Represents the camera pose for querying event streams.
[0076] In the embodiment of the present invention, two stages are included: in the coarse positioning stage, in order to retain the high-resolution temporal attributes inherent in the event, the query event stream is regarded as a pseudo 3D point cloud and encoded as a similarity feature, so as to accurately retrieve the camera poses of the neighboring event streams and the adjacent event streams from the known prior environmental information. In the pose refinement stage, a spatiotemporal pose refinement network (ESPR-Net) based on the RCNN architecture is used. The network solves the challenge of refining the coarse pose by learning more refined spatiotemporal features to accurately predict the relative pose. Figure 4 It is a flowchart of the overall framework in an embodiment of the present invention.
[0077] The present invention also provides a camera repositioning device based on an event camera, as described in the following embodiments. Since the principle of solving the problem by the device is similar to that of the camera repositioning method based on an event camera, the implementation of the device can refer to the implementation of the camera repositioning method based on an event camera, and the repeated parts will not be repeated.
[0078] Figure 5 Schematic diagram of a camera repositioning device based on an event camera in an embodiment of the present invention, the device includes:
[0079] The query event stream acquisition module 501 is used to acquire the event stream collected by the event camera as the query event stream;
[0080] A neighbor event stream retrieval module 502 is used to retrieve neighbor event streams of the query event stream from a database according to the query event stream;
[0081] A neighbor event stream camera pose determination module 503 is used to roughly locate the neighbor event stream using the query event stream, and determine the camera pose of the neighbor event stream according to the rough positioning result;
[0082] The relative posture acquisition module 504 is used to input the query event stream and the neighbor event stream into the trained ESPR-Net neural network, and output the relative posture of the query event stream and the neighbor event stream; the ESPR-Net neural network is trained based on the historical query event stream, the historical neighbor event stream, and the relative posture of the historical query event stream and the historical neighbor event stream;
[0083] A pose transformation matrix calculation module 505, used to calculate the pose transformation matrix of the query event stream according to the relative poses of the query event stream and the neighboring event stream and the camera pose of the neighboring event stream;
[0084] The query event stream camera pose acquisition module 506 is used to correct the camera pose of the query event stream according to the pose transformation matrix of the query event stream to obtain the corrected camera pose of the query event stream.
[0085] In one embodiment, the neighbor event stream retrieval module 502 is specifically used to:
[0086] According to the time information and spatial information of the query event stream, a similarity feature vector of the query event stream is constructed;
[0087] According to the similarity feature vector, the neighboring event streams of the query event stream are retrieved from the database.
[0088] In one embodiment, an event stream preprocessing module is further included, which is specifically used to:
[0089] Convert the query event stream and the neighbor event stream into query event voxels and neighbor event voxels respectively;
[0090] In the time dimension, query event voxels and neighboring event voxels are stacked in pairs to obtain synthetic event voxels;
[0091] Input the query event stream and neighbor event stream into the trained ESPR-Net neural network, including:
[0092] The synthetic event voxels are input into the trained ESPR-Net neural network.
[0093] In one embodiment, the posture transformation matrix calculation module 505 is specifically used to:
[0094] Using matrix multiplication, the pose transformation matrix of the query event stream is calculated according to the relative poses between the query event stream and the neighboring event streams and the camera poses of the neighboring event streams.
[0095] In one embodiment, the posture transformation matrix calculation module 505 is specifically used to:
[0096] The camera pose of the event stream is calculated according to the following pose transformation matrix:
[0097]
[0098] in, represents the relative position of the query event stream and the neighboring event stream, T c represents the camera pose of the neighboring event stream, Represents the camera pose for querying event streams.
[0099] An embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned camera repositioning method based on event camera when executing the computer program.
[0100] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the camera repositioning method based on an event camera is implemented.
[0101] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the camera repositioning method based on an event camera is implemented.
[0102] In an embodiment of the present invention, an event stream collected by an event camera is obtained as a query event stream; based on the query event stream, a neighboring event stream of the query event stream is retrieved from a database; the query event stream is used to roughly locate the neighboring event stream, and the camera pose of the neighboring event stream is determined based on the rough positioning result; the query event stream and the neighboring event stream are input into a trained ESPR-Net neural network, and the relative pose of the query event stream and the neighboring event stream is output; the ESPR-Net neural network is trained based on historical query event streams, historical neighboring event streams, and the relative pose of historical query event streams and historical neighboring event streams; the pose transformation matrix of the query event stream is calculated based on the relative pose of the query event stream and the neighboring event stream and the camera pose of the neighboring event stream; the camera pose of the query event stream is corrected based on the pose transformation matrix of the query event stream to obtain the corrected camera pose of the query event stream. In the above process, the embodiment of the present invention retrieves the neighboring event stream in the rough positioning stage and obtains the corresponding camera pose. In the pose refinement stage, the relative poses of the query event stream and the neighboring event streams are obtained through the ESPR-Net neural network, and the pose transformation is performed based on the relative pose to determine the camera pose of the query event stream, thereby improving the accuracy of camera positioning and the robustness of the positioning results.
[0103] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0104] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0105] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0107] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A camera relocalization method based on an event camera, characterized in that: include: Get the event stream collected by the event camera as the query event stream; According to the query event stream, the neighboring event streams of the query event stream are retrieved from the database; The query event stream is used to roughly locate the neighboring event stream, and the camera pose of the neighboring event stream is determined according to the rough positioning result; Input the query event stream and the neighbor event stream into the trained ESPR-Net neural network, and output the relative position of the query event stream and the neighbor event stream; The ESPR-Net neural network is trained based on the historical query event stream, the historical neighbor event stream, and the relative position of the historical query event stream and the historical neighbor event stream; Calculate the pose transformation matrix of the query event stream according to the relative poses of the query event stream and the neighboring event stream and the camera pose of the neighboring event stream; According to the pose transformation matrix of the query event stream, the camera pose of the query event stream is corrected to obtain a corrected camera pose of the query event stream.
2. The method according to claim 1, characterized in that According to the query event stream, the neighboring event streams of the query event stream are retrieved from the database, including: According to the time information and spatial information of the query event stream, a similarity feature vector of the query event stream is constructed; According to the similarity feature vector, the neighboring event streams of the query event stream are retrieved from the database.
3. The method according to claim 1, characterized in that Before inputting the query event stream and the neighbor event stream into the trained ESPR-Net neural network, it includes: Convert the query event stream and the neighbor event stream into query event voxels and neighbor event voxels respectively; In the time dimension, query event voxels and neighboring event voxels are stacked in pairs to obtain synthetic event voxels; Input the query event stream and neighbor event stream into the trained ESPR-Net neural network, including: The synthetic event voxels are input into the trained ESPR-Net neural network.
4. The method according to claim 1, characterized in that According to the relative poses of the query event stream and the neighboring event stream and the camera pose of the neighboring event stream, the pose transformation matrix of the query event stream is calculated, including: Using matrix multiplication, the pose transformation matrix of the query event stream is calculated according to the relative poses between the query event stream and the neighboring event streams and the camera poses of the neighboring event streams.
5. The method according to claim 4, characterized in that The camera pose of the event stream is calculated according to the following pose transformation matrix: in, represents the relative position of the query event stream and the neighboring event stream, T c represents the camera pose of the neighboring event stream, Represents the camera pose for querying event streams.
6. A camera repositioning device based on an event camera, characterized in that: include: The query event stream acquisition module is used to obtain the event stream collected by the event camera as the query event stream; A neighbor event stream retrieval module is used to retrieve neighbor event streams of the query event stream from the database according to the query event stream; A neighbor event stream camera pose determination module is used to use the query event stream to roughly locate the neighbor event stream, and determine the camera pose of the neighbor event stream according to the rough positioning result; A relative posture acquisition module is used to input the query event stream and the neighbor event stream into the trained ESPR-Net neural network, and output the relative posture of the query event stream and the neighbor event stream; the ESPR-Net neural network is trained based on the historical query event stream, the historical neighbor event stream, and the relative posture of the historical query event stream and the historical neighbor event stream; A pose transformation matrix calculation module, used to calculate the pose transformation matrix of the query event stream according to the relative poses between the query event stream and the neighboring event stream and the camera pose of the neighboring event stream; The query event stream camera pose acquisition module is used to correct the camera pose of the query event stream according to the pose transformation matrix of the query event stream to obtain the corrected camera pose of the query event stream.
7. The device according to claim 6, characterized in that The neighbor event stream retrieval module is specifically used for: According to the time information and spatial information of the query event stream, a similarity feature vector of the query event stream is constructed; According to the similarity feature vector, the neighboring event streams of the query event stream are retrieved from the database.
8. The device according to claim 6, characterized in that It also includes event stream preprocessing modules, specifically for: Convert the query event stream and the neighbor event stream into query event voxels and neighbor event voxels respectively; In the time dimension, query event voxels and neighboring event voxels are stacked in pairs to obtain synthetic event voxels; Input the query event stream and neighbor event stream into the trained ESPR-Net neural network, including: The synthetic event voxels are input into the trained ESPR-Net neural network.
9. The device according to claim 6, characterized in that The pose transformation matrix calculation module is specifically used for: Using matrix multiplication, the pose transformation matrix of the query event stream is calculated according to the relative poses between the query event stream and the neighboring event streams and the camera poses of the neighboring event streams.
10. The device according to claim 9, characterized in that The pose transformation matrix calculation module is specifically used for: The camera pose of the event stream is calculated according to the following pose transformation matrix: in, represents the relative position of the query event stream and the neighboring event stream, T c represents the camera pose of the neighboring event stream, Represents the camera pose for querying event streams.
11. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.
12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.