Simultaneous localization and mapping system and method based on event characteristic data stream and depth map

Abstract

The invention discloses a simultaneous positioning and mapping system and method based on an event characteristic data stream and a depth map, and relates to the technical field of multi-sensor fusion, the system comprises a data acquisition module, a data preprocessing module and a data fusion module; the data acquisition module is composed of an event stream generation module and a depth map generation module, and is used for acquiring an event stream triggered by movement or illumination change and acquiring an environment depth map; the data preprocessing module reconstructs an event stream into an event frame and carries out space and time registration on the event frame and a depth map; the data fusion module carries out pose estimation through a front-end feature point method, global pose alignment is ensured by adopting a layered optimization strategy, a mapping part is represented by a voxel grid method and the like, and high-precision map construction is realized. According to the method, accurate registration of event stream data and depth map data, robust pose estimation and high adaptability in weak light and high dynamic contrast environments are realized, and the method is suitable for complex scenes such as unmanned driving and robot navigation.

Description

Technical Field

[0001] The present invention relates to the field of multi-sensor fusion technology, and in particular to a simultaneous positioning and mapping system and method based on event characteristic data stream and depth map, which is suitable for high-precision positioning and mapping requirements in complex environments such as unmanned driving and robot navigation. Background Art

[0002] Simultaneous Localization and Mapping (SLAM) technology is a key component of autonomous navigation systems and is used in areas such as robot navigation and intelligent driving. The SLAM system integrates sensing devices such as lidar, cameras, and depth sensors to estimate its own position and build an environmental map in real time, providing basic support for path planning, obstacle avoidance, and other functions.

[0003] Laser SLAM is one of the SLAM technology paths, which generates high-precision point clouds with laser radar. Visual SLAM is another technical path, which uses cameras to capture visual information and uses image features for positioning and map construction. However, current visual SLAM technology has difficulty extracting stable feature points in textureless, low-light, and high-dynamic contrast environments, and is easily disturbed by dynamic objects. Multi-source sensor fusion SLAM can improve the robustness and accuracy of the system in complex environments through the complementary characteristics of multiple sensors.

[0004] Multi-source sensor fusion technology integrates the advantages of different sensors and shows higher adaptability and robustness in complex environments. Traditional monocular CMOS cameras lack scale information, so the SLAM system needs to jointly optimize the camera pose and map. Event cameras and time-of-flight depth sensors (TOF) provide new possibilities for SLAM. TOF depth sensors obtain depth information by calculating the flight time of light signals, providing the system with scale prior information, solving the problem of scale uncertainty in monocular vision SLAM, and thus improving computational efficiency. The fusion of texture information and depth information captured by CMOS cameras strengthens two-dimensional features into three-dimensional features, providing richer information for feature matching and pose estimation.

[0005] However, traditional CMOS cameras are easily limited in high-dynamic scenes and low-light environments, and their rolling shutter exposure is prone to image blur when moving at high speed, affecting the accuracy and real-time performance of pose estimation. In order to meet the needs of high dynamic range, low latency and low energy consumption, event cameras came into being. Each pixel of an event camera works independently and outputs an "event" signal only when the pixel brightness change exceeds a preset threshold. Due to its asynchrony and high temporal resolution, the event camera has extremely low latency, up to 1 microsecond, and can respond to lighting changes in real time, and has lower energy consumption in static or less changing scenes. The high temporal resolution and pixel-level change detection capabilities make it possible to avoid motion blur even in fast motion. However, the application of event cameras still faces some challenges, including complex data processing, lack of mature algorithms and tools, and how to effectively integrate with traditional cameras or other sensors to provide more comprehensive perception information is still a difficult point in current research.

[0006] To this end, the present invention proposes a simultaneous positioning and mapping system and method based on event characteristic data stream and depth map to meet the SLAM requirements in low light and high dynamic contrast environments. Summary of the invention

[0007] The purpose of the present invention is to overcome the limitations of the existing visual SLAM technology in low light and high dynamic contrast environments, achieve high-precision positioning and mapping, and provide a simultaneous positioning and mapping system based on event characteristic data streams and depth maps.

[0008] To achieve the above object, the technical solution of the present invention is as follows:

[0009] A simultaneous positioning and mapping system based on event characteristic data stream and depth map, comprising a data acquisition module, a data preprocessing module and a data fusion module; the data acquisition module comprises an event stream generation module and a depth map generation module; the data preprocessing module comprises an event frame reconstruction module and a registration module; the data fusion module comprises a front-end module, a back-end module and a mapping module;

[0010] The event stream generation module is used to collect event streams in the environment; the depth map generation module is used to collect depth maps in the environment;

[0011] The event frame reconstruction module is used to reconstruct the event stream into an event frame and align the event frame with the depth map in time; the registration module is used to align the event frame with the depth map in space to maintain a consistent spatial correspondence between event data and depth data;

[0012] The front-end module is used to estimate the posture by a feature point matching method; the back-end module adopts a hierarchical optimization strategy to align the local posture with the global posture to complete global consistency posture optimization; the mapping module adopts at least one map construction method for representing a three-dimensional environment to represent the map to complete high-precision environment construction.

[0013] As a preferred solution, the method for generating the event characteristic data stream includes at least any one of the following: directly acquiring the event characteristic data stream through an event camera, generating the event characteristic data stream based on a grayscale frame difference method, generating the event characteristic data stream based on an optical flow field, or generating the event characteristic data stream based on deep learning; the event camera includes at least any one of the following: an event camera based on a dynamic vision sensor, an event camera based on an integrated time domain event sensor, an asynchronous time encoding event camera, and an event camera that can reconstruct event frames.

[0014] As a preferred solution, the depth map generation module generates the depth map in at least one of the following ways: generating depth map data through time-of-flight ToF technology, generating depth map data through structured light technology, generating depth map data through binocular stereo vision sensor, and generating depth map data through lidar sensor.

[0015] As a preferred solution, the reconstructed event frame module reconstructs the sparse event characteristic data stream collected by the event stream generation module into an event frame to improve adaptability to low-light and high dynamic contrast environments, and utilizes the high temporal resolution and timestamp synchronization technology of the event characteristic data stream to achieve precise temporal alignment of the event frame and the depth map.

[0016] As a preferred solution, the registration module uses Zhang Zhengyou's chessboard calibration method to perform preliminary calibration on the spatial alignment of the event frame and the depth map, and further accurately aligns the event frame and the depth map in space through the feature point matching method.

[0017] As a preferred solution, the feature point matching method includes a registration method based on geometric features and a registration method based on grayscale information.

[0018] As a preferred solution, the front-end module uses the depth map to match and filter the feature points to screen out stable feature point matching relationships. After completing the feature point matching and filtering, the front-end pose estimation is performed on the event frame and the depth map.

[0019] As a preferred solution, the back-end module adopts a hierarchical optimization strategy of local first and global later. The local optimization part divides multiple consecutive frames of data into an optimization block, and achieves precise alignment of inter-frame poses through intra-block optimization. The global optimization takes all blocks as a whole for further global consistency adjustment.

[0020] As a preferred solution, the map construction method for representing the three-dimensional environment includes a three-dimensional representation method based on voxel grid, point cloud, surface grid, neural radiation field NeRF and 3D Gaussian sputtering 3DGS.

[0021] To achieve the purpose of the invention, the present invention also provides a method for simultaneous positioning and mapping based on event characteristic data stream and depth map, which is applied to a simultaneous positioning and mapping system based on event characteristic data stream and depth map as described in the above technical solution, and the method comprises the following steps:

[0022] S1: Event stream and depth map in the acquisition environment;

[0023] S2: Reconstruct the event stream into event frames and align them with the depth map in time and space;

[0024] S3: The front-end pose is estimated by feature point matching method, and the global pose alignment is performed by hierarchical optimization strategy;

[0025] S4: construct an environment map using at least one map construction method for representing a three-dimensional environment, wherein the map construction method includes a three-dimensional representation method based on voxel grids, point clouds, surface grids, neural radiation fields NeRF and 3D Gaussian sputtering 3DGS.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] The present invention realizes the alignment of event frames and depth maps in time and space by converting event stream data into event frames with high dynamic range and adopting a coarse-to-fine registration method. The front-end pose estimation is performed by feature point matching method and a hierarchical optimization strategy is adopted to complete the global consistent pose optimization, so that the system shows significant robustness and adaptability in low-light environment and high dynamic contrast environment. The mapping part uses a three-dimensional environment composition method and combines time-space consistent data with precise pose to achieve stable mapping in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A system block diagram of a simultaneous positioning and mapping system based on event characteristic data stream and depth map provided in Example 1 of the present application;

[0029] Figure 2 A flowchart of time synchronization of event data and depth data provided in Example 1 of the present application;

[0030] Figure 3 A flowchart of spatial synchronization of event data and depth data provided in Example 3 of the present application;

[0031] Figure 4A SLAM flowchart of a hierarchical optimization strategy provided in Example 4 of the present application. DETAILED DESCRIPTION

[0032] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0033] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0034] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the embodiments of the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0035] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims. In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not have to be used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0036] In addition, in the description of this application, unless otherwise specified, "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0037] Example 1

[0038] It should be noted that in the present invention, event stream (i.e., event characteristic data stream or event data) refers to the data information stream generated by each pixel of the light detection pixel array independently and asynchronously responding to the change of ambient light intensity, which has the characteristics of high temporal resolution, sparsity, and low latency, and usually contains information such as pixel position, timestamp, and polarity. Event streams can be obtained in a variety of ways, including but not limited to: event cameras (such as dynamic vision sensors) directly collect event data; grayscale frame difference method, by calculating the brightness change between consecutive grayscale frames to simulate and generate event streams; event stream simulation based on optical flow field, by estimating brightness changes through optical flow field to generate event streams; based on deep learning model generation, using neural networks to learn event characteristics from grayscale images and generate event stream data.

[0039] Existing hardware that can generate event streams includes: DVS (Dynamic Vision Sensor) series event cameras, DAVIS series event cameras, ATIS (Asynchronous Time-based Image Sensor) series event cameras, Sony's Event-based Vision Sensor, Samsung event cameras, Celex event sensors, etc. The present invention covers all commercial technologies and emerging implementation methods that can generate data streams with event characteristics, and is not limited to the acquisition methods listed above. All data that can generate data that conforms to the characteristics of event streams, including other commercial sensors or software processing methods to be launched in the future, are within the scope of application of the present invention.

[0040] See also Figure 1 , Embodiment 1 of the present application provides a simultaneous positioning and mapping system based on event characteristic data stream and depth map, including: a data acquisition module, a data preprocessing module and a data fusion module; the data acquisition module includes an event stream generation module and a depth map generation module; the data preprocessing module includes a reconstruction event frame module and a registration module; the data fusion module includes a front-end module, a back-end module and a mapping module;

[0041] The data acquisition module is used to collect event streams and depth maps in the environment. The event stream generation module is used to capture illumination changes and generate event stream data at asynchronous high temporal resolution when the ambient illumination changes or when the camera motion triggers an event signal. The depth map generation module is used to generate depth map information of the environment. The event camera in the event stream generation module captures brightness change signals in low-light and high-dynamic contrast environments to ensure that clear event data can be collected in low-light and high-dynamic contrast environments; the depth camera in the depth map generation module provides accurate depth information based on TOF technology, which serves as the scale benchmark of the SLAM system.

[0042] It should be noted that, in a specific embodiment, the event camera includes at least any one of the following: an event camera based on a dynamic vision sensor DVS, an event camera based on an integrated time domain event sensor ATIS, an asynchronous time coded event camera and an event camera that can reconstruct event frames, or other sensors that implement the same or similar functions.

[0043] It should be noted that, in a specific embodiment, the depth camera includes at least any one of the following: a time-of-flight TOF depth sensor, a structured light depth camera, a binocular stereo vision camera and a lidar, or other sensors that achieve the same or similar functions.

[0044] The data preprocessing module is used to reconstruct the event stream into event frames and align the time and space. The specific steps are as follows:

[0045] S11: Please refer to Figure 2 , the event stream is reconstructed into event frames. First, the depth map is used as the reference frame, and the time stamps between adjacent reference frames (t d1 ~t d2 ) event stream, with the timestamp t of the next reference frame d2 The timestamp of the event stream output is used to achieve time alignment, and the sparse event stream data of the event camera is reconstructed into continuous event frame images with a high dynamic range through learning or model or an algorithm that combines the two, so that the event data can adapt to subsequent feature extraction and pose estimation to improve adaptability to low-light environments; the same processing is performed on the depth map and event stream collected subsequently.

[0046] In step S11 of the present embodiment 1, the event frame reconstruction algorithm is preferably a deep learning algorithm, such as the e2vid algorithm, and may also be replaced by other event frame reconstruction algorithms, such as an event stream to frame reconstruction model based on a physical model.

[0047] S12: Spatial alignment The registration module first implements preliminary registration through Zhang Zhengyou's checkerboard calibration method to align the spatial coordinate systems of the two sensors, the event camera and the depth camera, to the same reference system. Then, the scale-invariant feature transform (SIFT) feature point matching method and matching filtering algorithm are used to further refine the registration to ensure that the event frame and the depth map are highly consistent in space, support subsequent data fusion, and keep a consistent spatial correspondence between the event data and the depth data.

[0048] It should be noted that, in a specific embodiment, a feature point matching method based on geometric features or based on grayscale information may also be used but is not limited to it.

[0049] The data fusion module adopts a hierarchical optimization strategy to process and optimize data in the front-end module and the back-end module, and build an environment map in the mapping module. The specific steps are as follows:

[0050] S101: The front-end module matches and filters the feature points based on the event frame and the depth map, extracts the feature points, and selects stable matching relationships to enhance data accuracy. After the matching relationship is stable, the three-dimensional feature points are used to perform pose estimation to generate accurate pose data, which improves the estimation accuracy and reduces the calculation error caused by incorrect matching.

[0051] S102: The optimization in the backend module adopts a hierarchical optimization strategy of local first and global later to achieve joint optimization of long-range data. In the local optimization part, multiple frames of data are taken as an optimization block with a fixed number of frames, and the precise alignment of the poses between adjacent frames is ensured through in-block optimization; in the global optimization stage, all optimization blocks are taken as a whole and global consistency adjustments are made to improve the long-term stability and accuracy of the system.

[0052] S103: In the present embodiment 1, the three-dimensional environment representation method adopted by the mapping module in the process of constructing the environment map is the voxel hashing method. In the specific three-dimensional map construction process, the event frame provides clear texture information in the environment, and the registered depth map provides time-space consistent and reliable three-dimensional depth data for map construction. Continuous and precise poses ensure the stability and accuracy of mapping. Through these registered, estimated and fused inputs, the mapping module can reconstruct the three-dimensional environment with high precision, ensuring that the map remains stable and accurate in low light and high dynamic contrast environments.

[0053] It should be noted that, in a specific embodiment, the environment map can also be constructed using a three-dimensional representation method based on voxel grids, point clouds, surface grids, neural radiation fields (NeRF) and 3D Gauss Splatting (3DGS), or other methods for achieving the same or similar functions.

[0054] Example 2

[0055] Based on Example 1, this Example 2 provides a method for simultaneous positioning and mapping based on an event characteristic data stream and a depth map, which is applied to a simultaneous positioning and mapping system based on an event characteristic data stream and a depth map described in Example 1. The steps of the method include:

[0056] S21: Collect event streams and depth maps in the target environment;

[0057] S22: reconstruct the event stream into event frames and align them with the depth map in time and space;

[0058] S23: front-end pose estimation is performed by feature point matching method, and global pose alignment is performed by adopting hierarchical optimization strategy;

[0059] S24: constructing an environment map by using at least one map construction method for representing a three-dimensional environment, wherein the map construction method includes a three-dimensional representation method based on voxel grid, point cloud, surface grid, neural radiation field (NeRF) and 3D Gaussian sputtering (3DGS).

[0060] The other technical details and implementation steps of this embodiment 2 are the same as those of embodiment 1 and will not be repeated here.

[0061] Example 3

[0062] This embodiment 3 is based on embodiment 1 and further illustrates the spatial alignment and registration of the event camera and the depth camera in embodiment 1.

[0063] Please refer to Figure 3 , this embodiment 3 provides a coarse-first-fine-later registration scheme to achieve spatial alignment of the depth map and the event frame, as follows:

[0064] S31: Calibrate the intrinsic parameter K of the depth camera using the checkerboard calibration method d and the intrinsic parameter K of the event camera e ; Simultaneously record the external parameters T of the depth camera and event camera on the same chessboard plane d and T e ;

[0065] S32: By T d and T e Calculate the pose transformation relationship between the depth camera and the event camera Then from T ed The matrix extracts the relative rotation matrix R ed and the relative translation vector t ed ;

[0066] S33: Record multiple sets of scenes with distinct features at the same time, and use the depth camera to collect infrared images. ir and depth map u d , using the event camera to collect the grayscale image u e The depth camera collects infrared images and depth images using the same sensor element; the event camera collects grayscale images and event streams using the same sensor element;

[0067] S34: Matching the feature points p of the infrared image using feature point extraction and matching methods d And the event camera grayscale feature point p e , and then use the feature matching filtering algorithm to filter out the wrong matches;

[0068] S35: u in S33 d Obtain the depth information Zd, and then use the depth map and the rotation matrix R preliminarily calibrated in step S32 ed and the translation vector t ed , and the feature matching relationship (p d , p e )Establish geometric constraints between matching points and construct the objective function:

[0069]

[0070] Optimize using the objective function minimization method and The result R of the preliminary calibration in step S32 ed and t ed As the initial value, it makes the algorithm easier to converge.

[0071] In a specific example, in step S34, the feature point extraction method can be an Oriented FAST and Rotated BRIEF (ORB) algorithm or a SIFT algorithm; the feature matching filtering algorithm can be a Random Sample Consensus (RANSAC) filtering algorithm, or other matching filtering algorithms based on geometric consistency or photometric consistency.

[0072] The other technical details and implementation steps of this embodiment 3 are the same as those of embodiment 1 and will not be repeated here.

[0073] Example 4

[0074] This embodiment 4 is based on embodiment 1 and further explains the hierarchical optimization strategy of first local and then global in embodiment 1.

[0075] Please refer to Figure 4 , this embodiment 4 provides a hierarchical optimization strategy of first local and then global, which is as follows:

[0076] S41: input the first set of registered depth maps and event stream reconstruction frames as the first reference frame, and use the unit matrix as the initial pose;

[0077] S42: Afterwards, each set of registered depth maps and event stream reconstruction frames are input as a new frame, feature matching and filtering are performed on the current frame and the previous frame, and the posture conversion relationship closest to the matching relationship is calculated as the estimated value of the current frame posture;

[0078] S43: Packing multiple consecutive frames into a block with a fixed frame number, using the pose estimated in step S42 as an initial value, and performing local pose alignment on all frames in the block;

[0079] S44: Fusion of common features within the block as common feature points within the block, including but not limited to using the first frame within each block as a key frame to perform global pose alignment to improve the long-term stability of the system;

[0080] S45: Taking the global pose, the image of each frame and the depth information as input, including but not limited to establishing a 3D map in a voxel grid manner.

[0081] The other technical details and implementation steps of this embodiment 4 are the same as those of embodiment 1 and will not be repeated here.

[0082] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A simultaneous positioning and mapping system based on event characteristic data stream and depth map, characterized in that: The system includes a data acquisition module, a data preprocessing module and a data fusion module; the data acquisition module includes an event stream generation module and a depth map generation module; the data preprocessing module includes an event frame reconstruction module and a registration module; the data fusion module includes a front-end module, a back-end module and a mapping module; The event stream generation module is used to collect event streams in the environment; the depth map generation module is used to collect depth maps in the environment; The event frame reconstruction module is used to reconstruct the event stream into an event frame and align the event frame with the depth map in time; The registration module is used to spatially align the event frame with the depth map so that the event data and the depth data maintain a consistent spatial correspondence; The front-end module is used to estimate the posture by a feature point matching method; the back-end module adopts a hierarchical optimization strategy to align the local posture with the global posture to complete global consistency posture optimization; the mapping module adopts at least one map construction method for representing a three-dimensional environment to represent the map to complete high-precision environment construction.

2. A simultaneous positioning and mapping system based on event characteristic data stream and depth map according to claim 1, characterized in that: The method for generating the event characteristic data stream includes at least any one of the following: directly acquiring the event characteristic data stream through an event camera, generating the event characteristic data stream based on a grayscale frame difference method, generating the event characteristic data stream based on an optical flow field, or generating the event characteristic data stream based on deep learning; the event camera includes at least any one of the following: an event camera based on a dynamic vision sensor, an event camera based on an integrated time domain event sensor, an asynchronous time encoding event camera, and an event camera that can reconstruct event frames.

3. The simultaneous positioning and mapping system based on event characteristic data stream and depth map according to claim 1, characterized in that: The depth map generation module generates the depth map in at least one of the following ways: generating depth map data through time-of-flight ToF technology, generating depth map data through structured light technology, generating depth map data through a binocular stereo vision sensor, and generating depth map data through a lidar sensor.

4. The simultaneous positioning and mapping system based on event characteristic data stream and depth map according to claim 1, characterized in that: The event frame reconstruction module reconstructs the sparse event characteristic data stream collected by the event stream generation module into an event frame to improve adaptability to low-light and high dynamic contrast environments, and utilizes the high temporal resolution and timestamp synchronization technology of the event characteristic data stream to achieve precise temporal alignment of the event frame and the depth map.

5. The simultaneous positioning and mapping system based on event characteristic data stream and depth map according to claim 1, characterized in that: The registration module uses Zhang Zhengyou's chessboard calibration method to perform preliminary calibration on the spatial alignment of the event frame and the depth map, and further accurately aligns the event frame and the depth map in space through the feature point matching method.

6. The simultaneous positioning and mapping system based on event characteristic data stream and depth map according to claim 5, characterized in that: The feature point matching method includes a registration method based on geometric features and a registration method based on grayscale information.

7. The simultaneous positioning and mapping system based on event characteristic data stream and depth map according to claim 1, characterized in that: The front-end module uses the depth map to match and filter the feature points to screen out stable feature point matching relationships. After completing the feature point matching and filtering, the front-end pose estimation is performed on the event frame and the depth map.

8. The simultaneous positioning and mapping system based on event characteristic data stream and depth map according to claim 1, characterized in that: The back-end module adopts a hierarchical optimization strategy of local first and global later. The local optimization part divides multiple frames of data into an optimization block, and realizes the alignment of inter-frame poses through intra-block optimization. The global optimization further adjusts all blocks as a whole for global consistency.

9. The simultaneous positioning and mapping system based on event characteristic data stream and depth map according to claim 1, characterized in that: The map construction method for representing a three-dimensional environment includes a three-dimensional representation method based on voxel grids, point clouds, surface grids, neural radiation fields NeRF and 3D Gaussian sputtering 3DGS.

10. A method for simultaneous positioning and mapping based on event characteristic data stream and depth map, applied to a system for simultaneous positioning and mapping based on event characteristic data stream and depth map as described in claims 1-9, the method comprising the following steps: S1: Collect event streams and depth maps in the target environment; S2: Reconstruct the event stream into event frames and align them with the depth map in time and space; S3: The front-end pose is estimated by feature point matching method, and the global pose alignment is performed by hierarchical optimization strategy; S4: construct an environment map using at least one map construction method for representing a three-dimensional environment, wherein the map construction method includes a three-dimensional representation method based on voxel grids, point clouds, surface grids, neural radiation fields NeRF and 3D Gaussian sputtering 3DGS.

Images