Binocular SLAM three-dimensional reconstruction and positioning based on self-supervised nerve radiation field
Through the combination of neural radiation field and SLAM technology, a binocular camera is used to perform multi-view stereo matching and NeRF rendering to generate a high-precision three-dimensional point cloud map, solving the accuracy problem of three-dimensional reconstruction and positioning in an unmanned workshop, and achieving more High-quality 3D scene reconstruction and robot navigation.
Patent Information
- Application Number
- CN202410843804.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-27
- Publication Date
- 2024-11-08
AI Technical Summary
Traditional unmanned workshop material handling vehicles are difficult to achieve high-precision three-dimensional reconstruction and positioning in complex indoor environments, especially due to limited computing resources and insufficient sensor information, which leads to difficulty in map reconstruction accuracy and navigation positioning.
Using a combination of neural radiation field and SLAM technology, a point cloud is generated through multi-view stereo matching training through binocular cameras, three-dimensional reconstruction and pose estimation, and a high-precision 3D map and pose estimation are generated through back-end optimization. The specific steps include binocular camera calibration, feature matching, ICP pose estimation, NeRF rendering and disparity map generation, combining the disparity map transformation depth map, building a three-dimensional point cloud map, and correcting errors through loopback detection and back-end optimization.
It realizes high-precision three-dimensional scene reconstruction and positioning in the unmanned workshop, improves the accuracy and robustness of robot navigation, reduces the computational complexity of data processing, and solves the loss of mismatch points and high-frequency details in traditional methods. question.
Smart Images

Figure BDA0004915136150000021 
Figure BDA0004915136150000041 
Figure BDA0004915136150000042
Abstract
Description
Technical Field
[0001] The present invention is a binocular vision three-dimensional reconstruction and positioning method based on neural radiation field mapping. Background Art
[0002] The intelligence of unmanned workshops is to realize the intelligence of equipment, material handling and production processes. Among them, material handling is not only an important auxiliary production process, but also an important node in workshop logistics. The material handling system of the intelligent unmanned workshop optimizes the link between the warehouse and the production line, realizes the automated transportation of materials in each production link, and makes the production process more efficient, automated and intelligent. The robot's perception of the environment is the basis for practical application, so synchronous positioning and three-dimensional environment reconstruction are becoming more and more important.
[0003] Traditional AGV material handling faces complex indoor environments, and the acquisition of scene information is affected by a variety of interference factors. Due to limited computing resources, it is difficult to build high-precision map reconstruction with less sensor information. In turn, relying on a large amount of sensor information will inevitably generate costs in terms of power consumption in terms of acquisition and processing. In terms of vision, the current mainstream is to build a two-dimensional map to achieve navigation. The use of three-dimensional mapping in the field of path planning requires consideration of the complexity of real environment elements. Therefore, how to achieve high-quality three-dimensional reconstruction and robot navigation and positioning in three-dimensional scenes, how to determine the robot's posture and establish its geometric and collision relationship with the reconstructed scene are urgent issues to be solved.
[0004] Based on the combination of neural radiation field and SLAM, the present invention designs a method that can generate point cloud through multi-view stereo matching training to achieve 3D reconstruction in an unmanned workshop working environment, completes self-positioning and map reconstruction through pose estimation and back-end optimization, so as to realize high-precision 3D scene reconstruction and positioning for unmanned workshops indoors. Summary of the invention
[0005] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings. A method is designed to perform three-dimensional reconstruction and positioning through multi-view stereo geometry in an unmanned workshop working environment, which specifically includes the following steps:
[0006] Step 1: According to the imaging principle of the binocular camera, calibration and correction are performed to obtain the camera's intrinsic parameters, extrinsic parameters, and distortion parameters;
[0007] Step 2: Use ORB-SLAM2 to perform feature matching and extraction on the extracted image to obtain key frames;
[0008] Step 3: Use a binocular camera to collect image information of the unmanned workshop data, pre-process the image, remove mismatched points, and estimate and optimize the pose through ICP;
[0009] Step 4: Input the obtained keyframes into the pre-trained diffusion model, output a set of multi-view images of the target, introduce the self-supervised neural radiance field, and then fit a NeRF on each scene to render the stereo image pair and depth map. Finally, use the rendered data to train any existing stereo matching network and output the disparity map of the target;
[0010] Step 5: Generate a 3D point cloud map by combining the depth map converted from the disparity map to build a map;
[0011] Step 6: Eliminate errors and interference through loop detection correction;
[0012] Step 7: Backend optimization: optimize the transmitted map and posture again to generate the posture and global map.
[0013] Furthermore, in the process of step 1, a binocular camera is used to capture the scene from different perspectives to obtain full-view images. These multi-view images can help in research fields such as three-dimensional reconstruction, behavior analysis, and action recognition.
[0014] Furthermore, in the process of step 2, the ORB feature point calculation method is used for implementation. Among them, the ORB feature point operator is mainly composed of the accelerated segmentation test feature corner detection algorithm (Features from Accelerated SegmentTest, FAST) and the binary robust independent basic feature descriptor algorithm. When rotation, translation and lighting phenomena occur, the ORB feature points are still not affected.
[0015] Furthermore, according to step three, the mismatched points are eliminated, and the KNN algorithm is used for classification to obtain the global nearest neighbor points. The average Euclidean distance between the data point and the nearest neighbor value to the adaptive neighbor value of other data points is calculated to obtain the improved nearest neighbor distance of the data point.
[0016] Furthermore, the LSH algorithm is used for rough matching to improve the matching degree of the algorithm. Since the retrieval performance of the kd tree weakens with the increase of the dimension, RANSAC is used for secondary elimination;
[0017] Furthermore, the two-dimensional ICP algorithm is used to solve the pose and standard value, and the error curve is drawn to solve the mean and variance.
[0018] Furthermore, in step four, after the left camera image output by the binocular camera is prepared, it is necessary to use a binocular stereo matching algorithm to calculate the disparity value and convert it into a depth map. Method process: First, collect multi-view images from the left RGB image in the static scene. Then, fit a NeRF on each scene to render stereo image pairs and depth maps. Finally, use the rendered data to train any existing stereo matching network. The specific contents include: NeRF learns the color and density values of each pixel in 3D space, and presents a novel view using ray tracing and volume rendering. Specifically, the ray r(t)=o+td emitted by the camera is used to predict the pixel color value C(r) rendered along the ray direction, where o is the optical center of the camera, d is the viewing angle direction, and t is a set of samples from far to near on the ray, that is,
[0019] Among them, the volume density σ(r(t)) can represent the probability that the light ends at a point; c(r(t), d) is the color value of the sampling point on the ray; T(t) is the cumulative transmittance from t1 to t, that is, whether the light can penetrate a certain point in the 3D object, which is the attenuation variable of the integral of the volume density σ. The higher the volume density of the sampling point, the greater the probability that the ray stays at the point, and the greater the impact on the final rendered pixel color value.
[0020] Furthermore, the data set is input into the NeRF network structure for training, and the mapping relationship from light and sampling position to color and density is learned through the multi-layer perceptron network MLP, and finally the three-dimensional point cloud data of the target is output.
[0021] Among them, a recursive network is added to the NeRF network structure to extract the three-dimensional features of the target; a multiple sampling strategy is designed to cyclically extract multiple light sampling points of each viewing angle of the target; the parameters of each layer of the NeRF network are repeatedly fine-tuned through the back propagation algorithm, and the iterative training is performed until the generated point cloud and the real data meet the preset requirements in terms of key error indicators, thereby generating three-dimensional point cloud data.
[0022] Furthermore, the generated depth map is converted into a point cloud map to construct point cloud data.
[0023] Further, according to step six, loop detection helps correct the accumulated error, specifically by constructing a bag-of-words model, which is implemented using a K-ary tree method. An offline dictionary is obtained based on images in a database, and detection is performed based on the calculated similarity of feature vectors of the current frame and the historical frame.
[0024] Furthermore, the back-end optimization part uses the graph optimization principle to deal with the back-end nonlinear problems, and uses the Levenberg-Marquardt (LM) convergence algorithm to differentiate the nonlinear function; then g2o (General Graph Optimization) is used to optimize the visual SLAM related content.
[0025] Compared with the prior art, the present invention is suitable for indoor scene mapping. In feature point extraction and matching, it provides high-quality matching points in the tracking thread by screening out mismatched points. Considering the two-dimensional plane motion relationship of the mobile car, the matching point pairs are processed with dimension reduction, and the two-dimensional ICP algorithm is combined to realize the estimation of the robot camera motion trajectory, thereby reducing the data processing operation. A model method of training a neural network supervised stereo network on a synthetic data set is introduced. By collecting a single-view image of the target; inputting the single-view image into a pre-trained diffusion model, a set of multi-view images of the target are output; inputting the multi-view image into a pre-trained NeRF network, and outputting the three-dimensional point cloud data of the target; combining point cloud data and local optimization to generate a high-quality three-dimensional map, optimizing the three-dimensional scene features with high-frequency details lost, and solving the network forgetting problem caused by the scene being too large, thereby improving the overall accuracy and robustness of the three-dimensional scene reconstruction. Next, the optimization and positioning of the three-dimensional reconstruction are realized through loop detection and back-end optimization, so as to achieve a higher rendering effect quality and accuracy in the construction and positioning of the actual three-dimensional map. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is a flow chart of the present invention
[0027] Figure 2 This is the overall structure diagram of SLAM
[0028] Figure 3 This is the flowchart of the front-end processing part
[0029] Figure 4 This is the algorithm flow chart of ICP 3D to 2D reduction
[0030] Figure 5 This is the structure diagram of the self-supervised NeRF neural radiation field
[0031] Figure 6 This is the loop detection flow chart DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the technical solution of the present invention is specifically described below with reference to the accompanying drawings and examples.
[0032] Step 1: First, arrange the binocular cameras, split the composite image into two parts: the left image and the right image, and then calibrate the binocular cameras. Using the coordinate system of the left camera as the world coordinate system, the relationship between the two cameras can be obtained: R = R r (R l )t,t=t r -Rt l
[0033] Where R and t represent the external parameters to be calculated, R r , R l and t l ,t r Represents the rotation matrix and translation vector of the left and right cameras respectively.
[0034] During calibration, a checkerboard calibration plate is used for calibration, and then the Bouguet stereo correction method is used to divide the rotation matrix into r1 and r2, which are used as the left and right synthetic rotation matrices respectively to achieve image plane coplanarity and row alignment.
[0035] Then calculate the matrix to align the left camera pole horizontally and transform it to infinity R rect =[e1, e2, e3] T e3=e1×e2
[0036] Among them, t x ,t y ,t z They are translation vectors in three directions, which can be achieved by aligning the left and right cameras: R l =R rect r l R r =R rect r r
[0037] The above is to complete the binocular camera outputting parallel aligned images under ideal conditions.
[0038] refer to Figure 3 As shown, the specific front-end processing steps of the present invention are as follows:
[0039] Step 2: Collect images of the indoor unmanned workshop and use the ORB (Oriented FAST and Rotated BRIEF) algorithm to implement the following steps:
[0040] S1: Using the Features from Accelerated Segment Test (FAST) corner detection algorithm, the absolute value of the difference between each point around a pixel point and the gray value of that point is compared with a threshold, and the number of absolute values exceeding the threshold is calculated.
[0041] Select 16 points on the circle edge with radius r. If the threshold is 12, then each pixel point is a corner point. Specifically, it can be expressed as:
[0042] p represents the pixel point; ε d represents the threshold; I(l) and I(p) respectively represent the gray values of the i-th point and the adjacent point.
[0043] S2: Using the Binary Robust Independent Elementary Features (BRIEF) descriptor algorithm, a binary descriptor is generated by comparing the differences between 256 pairs of pixels within an arbitrary neighborhood of the central pixel point p.
[0044] For any pair of pixel points p(x) and p(y), if p(x) < p(y), then the corresponding position of the descriptor is 1, otherwise it is set to 0.
[0045] In the above formula, f n (p) represents the value of the descriptor at point p; n represents the number of point pairs selected according to a specific method around the feature point; x i 、y i represent the random points selected in the neighborhood window of the feature point.
[0046] S3: The Hamming distance is used to obtain the similarity between feature points, and the sub-optimal matching method is used to screen out matching errors. When the Iterative Closest Point algorithm is used to match feature points of 2 frames of images, an Euler transformation formula needs to be constructed and a least squares problem needs to be established for each group of images. The specific expression is:
[0047] J represents the target to be optimized; p i 、p′ i both represent the matched feature points; R represents the rotation matrix of 2 frames of images; t represents the translation vector; n represents the number of feature points detected during the movement.
[0048] S4: Based on the above feature point matching, the singular value decomposition method can be used to solve the opposition least squares problem, and finally the posture change of the two frames of images can be obtained. Based on the camera posture distance and the time difference between the two frames of images, some of the collected images are selected as key frames to reduce the amount of back-end optimization calculations and realize local map construction. Among them, the distance between images can be expressed as:
[0049] R a , R b and t a ,t b They represent the selection matrix and translation vector of the two image frames respectively; r represents the weight coefficient, and the value is set to 1.6 according to the change in camera posture corresponding to the rotation.
[0050] Step 3: Use the K nearest neighbor algorithm with the LSH search algorithm to match the image feature points. The hash function family is denoted as H, h1, h2…, h n ∈H. The K-bit hash function is defined as f(p) = [h1(p), h2(p)…h k (p)],h1(p),h2(p)…h k (p) are k functions randomly selected from the function family. After this step, there are q f(m) functions generated: f1(p), f2(p)…f q (p), where p is a point in space, s is a query sub, and each f(p) function corresponds to a hash table that maps point p to a hash bucket.
[0051] At the same time, query s is processed using hash mapping through the g(s) function. After processing, if point p and query s are mapped to the same hash bucket, point p can be used as a candidate point set, and the distance between s and each point in the set is calculated, and 1 or k closest points are selected.
[0052] In the matching process, mismatched points are still inevitable. In order to improve the matching accuracy of the image, the random sampling consensus (RANSAC) algorithm is used for secondary elimination.
[0053] Determine the initial optimal number of internal points Qi and the number of sampling times N. Using the formula n represents the minimum amount of data required to solve the model, W represents the proportion of inliers, and P represents the probability that a matching point pair is an inlier.
[0054] Randomly select 4 pairs of non-collinear matching points from the rough matching points to solve the transformation matrix H; use the transformation matrix H to calculate the Euclidean distance D between the corresponding matching points i ; Set a threshold δ, which will satisfy the Euclidean distance Di The point pairs with a value <δ are selected and recorded as interior points, and their number is counted; the number of interior points obtained is compared with Q i Compare, if the number of internal points is greater than Q i , then the current H matrix is taken as the best estimation matrix, and the Qi value is updated; random sampling is repeated many times until the number of internal points no longer changes, and the matrix at this time is taken as the optimal transformation matrix.
[0055] Step 4: Reference Figure 4 As shown, the specific two-dimensional ICP algorithm steps of the present invention are as follows:
[0056] Considering that the planar wheeled robot belongs to the category of motion, the two-dimensional ICP algorithm is used. The point set of the three-dimensional space points corresponding to the pixel matching points of the first frame image projected onto the X and Z planes is set as P, and the point set corresponding to the second frame image is set as P', as shown in the formula:
[0057] The coordinates of any point in P are (p ix , p iz ), the coordinates of any point in p′ are (p′ ix , p′ iz ). The camera motion solution required includes the rotation matrix R and the translation vector t. The above two sets of points satisfy:
[0058] The rotation matrix R is a two-dimensional rotation matrix, and the translation vector t is composed of the translation components on the X-axis and the Z-axis:
[0059] According to the error term e i =p i -(Rp′ i +t)
[0060] where e i It represents the coincidence error of the corresponding matching points in the two-dimensional space in the two-frame images. Construct the least squares problem to solve the rotation matrix R and the translation vector t, as shown below:
[0061] Find the centroid corresponding to the two sets of points, using p c and p c ′ is expressed as:
[0062] From the above two formulas
[0063] The first term above is only related to the rotation matrix R. Minimize the first term to find the value corresponding to the rotation matrix R. Then substitute the second term and make it zero to find the translation vector t.
[0064] Step 5: After the binocular camera, i.e. the image and the camera pose, is prepared, the binocular stereo matching algorithm needs to be used to calculate the disparity value, and then converted into a depth map. Figure 5 As shown, the specific steps of building a neural radiation field network of the present invention are as follows:
[0065] The data collected based on the intrinsic and extrinsic parameters of the above cameras is used to train a neural radiance field (NeRF); by rendering a batch of light colors from the collected image positions And optimize the L2 loss based on the pixel color C(r) in the collected frames: Stereo paired rendering using Instant-NGP is used to train a separate NeRF for each scene.
[0066] After NeRF is trained, multiple virtual stereo cameras (virtual extrinsics) are defined for each trained NeRF model, which can simultaneously render countless binocular stereo pairs (left and right eyes, with the left side as the reference frame and the right side as the target frame) of any viewpoint at any spatial resolution. A stereo training set is created to generate a virtual set of stereo extrinsic parameters S = I|b.
[0067] The rotation is represented by the 3×3 identity matrix I and the translation vector b=(b, 0, 0)T with a magnitude b along the x-axis in the camera reference frame, defining the baseline of the virtual stereo camera.
[0068] A third image (the second target frame) is rendered to the left of the reference frame of each stereo pair, resulting in a perfectly aligned stereo image triplet (I l , I c , I r ). The middle camera can be rendered to generate I c , E k =R k |t k ; Left camera, can render to generate I l , Right camera, can render and generate I r , Finally, the rendered depth map z r , plus the camera internal parameters, under the condition that the disparity map is aligned with the central image of the stereo triplet, the rendering disparity map d can be extracted by the following formula r , in order to train the stereo matching network.
[0069] The data generated above will then be used to train a stereo matching model. Given a rendered stereo image triplet (I l , I c , I r ), by providing (I l , I c ) as a stereo pair to predict disparity
[0070] Then, a self-supervised NeRF training loss is proposed for training: image reconstruction is used to supervise disparity estimation and a photometric loss is estimated. First, the estimated disparity map To map the target frame I r , get the reconnected reference frame calculate and I c The brightness difference between:
[0071] However, this formulation lacks reasonable supervision in occluded regions, such as the left boundary of the frame image or the left side of each depth discontinuity, which is invisible in the right image.
[0072] To overcome this limitation, the third image I mentioned in the previous section is used l , an image reconstruction was also performed, I l To I c The network estimates disparity and disparity compensates the target frame to obtain the reconstructed reference value.
[0073] By calculation The supervision of the occluded area of the middle left binocular will be compensated by the middle right binocular, and the final calculated triplet photometric loss is: Textureless areas are discarded by the mask μ The photometric loss is further assisted by an additional loss that exploits the difference between the disparity map predicted by the matching network and the disparity map rendered by NeRF:
[0074] To address the problem that depth maps rendered by NeRF often show artifacts and large errors, a filtering mechanism based on rendering uncertainty is adopted to retain only the most reliable pixels, and ambient occlusion (AO) is used to measure d c Confidence of:
[0075] The above two losses are added together, and the weights balance the effects of photometric and parallax losses, completing NeRF supervised training:
[0076] γdisp , γ 3ρ is the weight to balance the photometric loss and parallax loss, η disp Defined as: According to the threshold th on AO, it is normalized in [0, 1].
[0077] After obtaining the disparity map, the disparity map and depth map conversion formula are:
[0078] Among them, depth is the depth map to be calculated, f is the focal length of the camera, baseline is the baseline distance of the binocular camera, and disp is the disparity map previously calculated, in pixels. After obtaining the depth, create a point cloud for the pixels on the photo, and convert a pixel point (u, v) on the image into a three-dimensional point P = (x, y, z, b, g, r) in the point cloud. Its basic parameters are 6, namely the world coordinate system and the color value. The last three parameters directly correspond to the color paper assignment of the pixel point. The coordinate values are as follows: z=depth
[0079] Project the 3D point cloud image to the 3D coordinate system to obtain the reconstruction of the 3D scene.
[0080] Step 6: Reference Figure 6 As shown, the specific loop detection steps of the present invention are as follows:
[0081] Since the collected key frames may have errors, loop detection is used to help correct the accumulated errors.
[0082] Furthermore, in order to facilitate the description of the collected images, the bag-of-words model is constructed using the K-ary tree method. The specific steps are as follows:
[0083] First, prepare the feature points extracted from the image by the ORB algorithm, then call the dictionary in the DBoW3 library to convert the obtained feature points into corresponding functions, and use the bifurcation tree number k used in this paper is 10, the corresponding depth value d is 5, and the bag of words model is established; obtain the database image; obtain the offline dictionary based on the database image; match the historical frame image with the offline dictionary to obtain the feature vector recorded as v p ; The current frame image is matched with the offline dictionary to obtain the feature vector v q ; Calculate the similarity between the current frame and the historical frame Loop closures are detected based on similarity.
[0084] Using the graph optimization principle to deal with the back-end nonlinear optimization problem, the robot’s position at a certain moment in the movement process is x k , the data observed by the binocular camera is z k , where the observation equation is: z k =h(x k , p)
[0085] Due to the error e k Rewrite the observation error: e k =z k -h(x k , p)
[0086] The optimization objective function is expressed as:
[0087] The nonlinear function F(x) can be expressed as
[0088] Since the derivation process of nonlinear functions is relatively complicated, the Levenberg-Marquardt (LM) method is used to converge: (H+λv)Δx=b
[0089] Where I is the unit matrix and λ is a positive real number. It is adjusted in real time according to the state of optimization, controlling the direction and step size of the optimization variables to achieve a better convergence effect.
[0090] Step 7: g2o (General Graph Optimization) is used to optimize the visual SLAM related content. The specific steps are as follows: the first step is to select a solver from PCG, CSparse, Choldmod to solve the linear equation; the second step is to select a BlockSolver module; the third step is to select one from the three iterative strategies GN, LM, Doglog to complete the preparation of graph optimization. The optimization of pose and map is completed.
[0091] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.
[0092] In summary, the present invention first needs to correct the camera used to obtain internal and external parameters, extract key frames and poses collected after feature extraction and feature matching, pre-process the collected images and input them into the Nerf neural radiation field multi-layer feature network for rendering, train the stereo matching model with the rendered data, and further assist the photometric loss based on the additional loss of the difference between the disparity map predicted by the matching network and the NeRF rendered disparity map to obtain the final three-dimensional scene reconstruction result, optimize the three-dimensional scene features with high-frequency detail loss, and solve the network forgetting problem caused by the scene being too large, and improve the overall accuracy and robustness of the three-dimensional scene reconstruction. Next, the optimization and positioning of the three-dimensional reconstruction are achieved through loop detection and back-end optimization. Compared with the traditional SLAM method, the scheme of the present invention is more suitable for indoor lighting conditions after combining the Nerf neural radiation field, and achieves higher rendering effect quality and accuracy in the construction and positioning of the actual three-dimensional map.
Claims
1. A mapping and positioning algorithm based on self-supervised neural radiation field combined with SLAM, Its characteristics are: The algorithm consists of image preprocessing, front-end pose estimation, neural network of neural radiation field, loop detection, and back-end optimization. It includes the following steps: S1: According to the mapping and positioning algorithm of claim 1, it is characterized by: using a binocular camera to shoot the scene from different perspectives to obtain a full-view image, using ORB-SLAM2 to perform feature matching and extraction, obtaining key frames, and using the KNN algorithm and LSH algorithm to remove mismatched points through the image, as well as secondary elimination of the RANSAC algorithm. S2: According to the mapping and positioning algorithm of claim 1, it is characterized by: using a two-dimensional ICP algorithm, drawing an error curve by difference between the two-dimensional ICP algorithm and the standard value to solve the mean and variance to solve the posture. S3: The mapping and positioning algorithm according to claim 1 is characterized in that: taking the left view as the world coordinate system, collecting multi-view images and combining the camera pose and internal and external parameters to input into the NeRF network structure for training, generating triple pairs, and through a stereo matching network, adding photometric loss and an additional loss of the difference between the disparity map predicted by the matching network and the disparity map rendered by NeRF to compensate for the light loss, and learning the mapping relationship from light and sampling position to color and density through a multi-layer perceptron network MLP, finally obtaining a processed disparity map, and obtaining three-dimensional point cloud data by converting it into a depth map. S4: According to the mapping and positioning algorithm described in claim 1, it is characterized by: using loop detection to help correct the accumulated error, building a bag-of-words model, and using a K-ary tree method to achieve it, obtaining an offline dictionary based on the images in the database, and performing detection based on the calculated similarity of the feature vectors of the current frame and the historical frame. S5: According to the mapping and positioning algorithm of claim 1, Its characteristics are: The back-end optimization part uses the graph optimization principle to deal with the back-end nonlinear problems, and uses the Levenberg-Marquardt (LM) method convergence algorithm to derive the nonlinear function; then g2o (General Graph Optimization) is used to optimize the visual SLAM related content.
Citation Information
Cited By
Visual dominant multi-mode SLAM method for indoor office environment based on NeRF
CN119206118A
Airplane auxiliary power device pipeline three-dimensional reconstruction method based on vision measurement
CN119762677A
Method for determining preset position of video monitoring camera
CN119850720A
Passive positioning method for passengers in railway passenger transport station based on multi-modal image fusion
CN120339570A
XR equipment image synthesis method and system based on multiple key frames
CN120997055A
