36 results about "Monocular slam" patented technology

The invention discloses a real-time dense monocular simultaneous localization and mapping (SLAM) method based on an online learning depth prediction network. The method comprises: optimization of a luminosity error of a minimized high gradient point is carried out to obtain a camera attitude of a key frame and the depth of the high gradient point is predicted by using a trigonometric survey methodto obtain a semi-dense map of a current frame; an online training image pair is selected, on-line training and updating of a CNN network model are carried out by using a block-by-block stochastic gradient descent method, and depth prediction is carried out on the current frame of picture by using the trained CNN network model to obtain a dense map; depth scale regression is carried out based on the semi-dense map of the current frame and the predicted dense map to obtain an absolute scale factor of depth information of the current frame; and with an NCC score voting method, all pixel depth prediction values of the current frame are selected based on two kinds of projection results to obtain a predicted depth map, and Gaussian fusion is carried out on the predicted depth map to obtain a final depth map. In addition, the invention also provides a corresponding real-time dense monocular SLAM system based on an online learning depth prediction network.

The invention discloses a dynamic scene visual positioning method based on image semantic segmentation, and belongs to the field of SLAM (Simultaneous Localization and Mapping). The method comprises the following steps: firstly, segmenting a dynamic object in an original image by adopting a supervised learning mode in deep learning to obtain a semantic image; on the basis, extracting ORB feature points from the original image, and removing the feature points of the dynamic object according to the semantic image; and finally, based on the removed feature points, positioning and tracking the camera motion by adopting a monocular SLAM method based on point features. A positioning result shows that compared with a traditional method, the method disclosed by the invention has the advantage thatthe positioning precision in a dynamic scene is improved by 13% to 30%.

The invention provides an initialization method and system of a monocular SLAM algorithm. The method comprises the following steps of selecting pixel points in the image overlapping area from the acquired image; converting the selected pixel points into pixel points corresponding to the binocular SLAM algorithm; and calculating the three-dimensional coordinates of the converted pixel points in theimage overlapping area based on a binocular SLAM algorithm, converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system, and initializing the monocular SLAM algorithm according to the coordinate information of the camera coordinate system. Therefore, according to the embodiment of the invention, the monocular SLAM algorithm can be assisted to realize initialization according to the binocular SLAM algorithm, so that the monocular system corresponding to the monocular SLAM algorithm can realize initialization faster and better, andthe stability and the calculation precision of the monocular SLAM algorithm are improved.

A method for improving geometry-based monocular structure from motion (SfM) by exploiting depth maps predicted by convolutional neural networks (CNNs) is presented. The method includes capturing a sequence of RGB images from an unlabeled monocular video stream obtained by a monocular camera, feeding the RGB images into a depth estimation/refinement module, outputting depth maps, feeding the depth maps and the RGB images to a pose estimation/refinement module, the depths maps and the RGB images collectively defining pseudo RGB-D images, outputting camera poses and point clouds, and constructing a 3D map of a surrounding environment displayed on a visualization device.

The invention discloses an improved nonlinear optimization method of monocular inertial navigation SLAM. On the basis of the VINS-Mono engineering, the nonlinear optimization method is used for coupling IMU data in a tight coupling manner. The method comprises the steps of: image and IMU preprocessing; initialization; back-end sliding window optimization; and closed-loop detection and optimization. According to the improved nonlinear optimization method disclosed by the invention, algorithm improvement is performed based on the existing VINS.mono, a test is performed on a public data set EuRoC, and contrastive analysis is performed on the previous VINS.mono scheme. The results show that the IMU initialization can well restore a scale factor of the system, solve the problem of monocular SLAM scale uncertainty, can complete the initialization faster and more accurately, and can robustly perform real-time motion estimation and sparse map construction, and the accuracy and the robustness of the improved system are effectively improved.

The invention relates to a feature based monocular SLAM (Simultaneous Localization and Mapping) quick initialization method, which comprises the steps of 1) starting SLAM, acquiring a first frame picture and extracting ORB feature points P(x, y) for the picture; 2) performing image distortion removing on the ORB feature points P(x, y); 3) performing normalization on image coordinates of the distortion removed feature points in the step 2); 4) building a random depth for each feature point in the step 3); 5) combining results of the step 3) and the step 4) so as to build map points corresponding to the feature points, and acquiring an initial map; and 6) performing optimization on a matching result of subsequent adjacent frames and an existing map, and executing the normal feature SLAM process for each next newly increased frame so as to continuously adjust and expand the map and realize continuous tracking for SLAM. The feature based monocular SLAM quick initialization method provided by the invention is high in speed, small in calculation amount and not restricted by the depth of field.

The invention discloses a monocular SLAM initialization method and system based on a wheel type encoder, and the method comprises the steps of selecting a first key frame and a second key frame in anSLAM initialization image sequence, carrying out the feature matching, calculating the world coordinate system coordinates of all matching points in the first key frame, and building an initial pointcloud map; and determining the camera pose of the second key frame by using the displacement information of the wheel type encoder, fixing the camera poses of the first key frame and the second key frame during optimization, and not performing local bundling adjustment on the camera poses of the two frames. According to the present invention, the displacement information of the wheel type encoderis combined with the point characteristics of image information of the visual sensor to play a role in information complementation, the camera pose of the second key frame can be determined very accurately, the scale information of the real world is reserved, the positioning precision of the camera is improved, the operation is simplified, and the response speed is increased.

The invention discloses a monocular SLAM initialization method. The method comprises: feature points of a template picture are extracted, poses of the camera and the template picture are transmitted to a configuration file; collecting a picture containing a template as a first frame of picture; extracting ORB feature points of the first frame of picture and matching the ORB feature points with thefeature points on the template picture; a homography matrix is calculated; projecting known points defined in the template picture into the first frame of picture by using a homography matrix; calculating the pose of the first frame of picture relative to the template picture, and finally calculating the spatial coordinates of the feature points reserved in the feature points, matched with the first frame of picture, on the template picture according to the established self-defined coordinate system, and taking the spatial coordinates as corresponding map points in the SLAM map to complete initialization. Compared with an existing method, the rapid initialization method provided by the invention has higher precision and faster initialization speed, and greatly reduces the calculation costof initialization.

The invention provides a binocular point cloud generation method and system. The method comprises the following steps of: selecting pixel points in an image overlapping area from the acquired image according to a monocular SLAM algorithm; converting the selected pixel points into pixel points corresponding to a binocular SLAM algorithm; calculating the three-dimensional coordinates of the converted pixel points in the image overlapping region based on a binocular SLAM algorithm; and converting the three-dimensional coordinates of the pixel points into coordinate information of a camera coordinate system, calculating the three-dimensional coordinates of the pixel points in the non-overlapping area of the image based on the coordinate information of the camera coordinate system by using themonocular SLAM algorithm, and generating a point cloud according to the three-dimensional coordinates of the pixel points in the non-overlapping area and the overlapping area of the image. According to the scheme of the invention, scale calculation is carried out according to the pixel point information in the image overlapping area of the collected image, and the pixel point information of the non-overlapping area of the image is obtained. The binocular SLAM algorithm is used for assisting the monocular SLAM algorithm to obtain mature points in the image faster and more accurately, and the point cloud generation stability and the object real scale obtaining accuracy are improved.

The embodiment of the invention provides a monocular SLAM initialization method and device and electronic equipment, and belongs to the technical field of image processing. The method comprises the following steps: acquiring multiple frames of continuous images collected by a target camera; calculating a homography matrix among multiple frames of continuous images; obtaining a homography matrix between every two images by using the homography matrix between the multiple frames of continuous images; and obtaining pose data and a plane normal vector of the target camera according to all the homography matrix optimization variables. In the initialization process, multi-frame information is used, and an original method for solving the camera pose and the plane normal vector through matrix decomposition is replaced with variable optimization. According to the method, the parameter quantity is small, the camera pose and the plane normal direction are calculated by using the result of variable optimization, the spatial position of the feature point is calculated by using the plane normal direction and the normalized distance from the camera to the plane, triangularization and PnP are avoided, and the efficiency of monocular SLAM initialization is improved.

The invention discloses a monocular SLAM scale recovery method, which comprises the following steps: manufacturing a calibration box, wherein a rectangular through hole is formed in the calibration box in a penetrating mode; placing a calibration box at the visual field of the monocular camera, wherein the monocular camera models a target object after passing through a through hole of the calibration box; enabling the monocular camera to perform image acquisition on the inner wall of the calibration box, and performing region segmentation to obtain upper and lower surface images of the inner wall of the calibration box in the first few key frames generated by the SLAM; obtaining three-dimensional map point coordinates of the upper and lower images of the inner wall of the calibration box according to the corresponding relation between the feature points and the three-dimensional map points in the SLAM; fitting three-dimensional map point coordinates of the upper and lower surfaces of the inner wall of the calibration box by using two-plane horizontal behavior constraint conditions, and calculating and obtaining distances between the upper and lower surfaces of the inner wall of thecalibration box in a three-dimensional map; calculating a scale factor; and recovering the whole three-dimensional map scale by using the scale factor F. The monocular SLAM scale in the human intestinal environment can be automatically recovered, the effect is good, and the precision is high.

The invention discloses a method and system for recovering a monocular SLAM scale through detection and calibration, and the method comprises the following steps: a camera is calibrated by a calibration module; a calculation module calculates the depth of the point distance camera on the plane; a detection module is used for detecting the marker based on deep learning; and a scale recovery moduleperforms SLAM back-end optimization to obtain the scale of the monocular SLAM. The invention has the advantages that the calibration and detection method is adopted for scale recovery, the markers areplaced on the road for scale recovery, and the calculated amount and complexity of the algorithm are greatly reduced.

The invention discloses a monocular SLAM system initialization algorithm based on a dot-line unified framework. The algorithm comprises the following steps: 1, setting an index container with unifieddot-line features, and enabling the obtained dot-line features to be matched and unified in a subsequent random sampling consistency algorithm for calculating F and H matrixes; 2, unifying the line features in a matrix calculation framework, and calculating F and H matrixes and corresponding scores by threads according to the midpoints of the preprocessed line features; and 3, determining a current effective matrix according to the score, and recovering corresponding 3D dot-line features based on the matrix, thereby completing initialization of the monocular SLAM system. The invention providesan initialization method for unifying dot-line characteristics, which ensures that a monocular SLAM system can fully utilize image information, reduces the system initialization difficulty and realizes high-precision initialization.

The invention relates to the technical field of power line inspection, and discloses a monocular SLAM system for high-voltage power transmission line inspection. The system is characterized in that anoverhead power transmission line inspection robot uses a visible light and infrared double-light fusion nacelle, so that a visible light image and an infrared image of the same scene can be obtainedat the same time, wherein pixel points of the visible light image and the infrared image are in one-to-one correspondence; an inspector or an automatic fault identification algorithm checks whether faults such as rusting, cracking, falling and the like exist on the surface of hardware, whether foreign matters exist on a high-voltage power transmission line and whether bird nests and the like existon a tower to influence the power transmission safety according to the images; whether the temperature of the power transmission line is abnormal is judged through the infrared image so as to determine whether the power transmission line is abnormal; three-dimensional reconstruction is performed on a power transmission line channel through a visual SLAM technology; and whether the power transmission safety is affected can be known through a three-dimensional map.

The invention relates to an on-line luminosity calibration method based on direct method monocular SLAM. The method comprises the steps that feature point tracking is conducted on an on-line image sequence, and a luminosity error model is constructed for an initial frame image to obtain initial optimization parameters; the initial optimization parameters are written into an online parameter database, an online luminosity error model for an online frame window is subsequently constructed according to the online parameter database to perform online luminosity calibration, and meanwhile, parameter updating is performed on the online parameter database in the background. According to the method, online luminosity calibration is carried out on the online image sequence, the luminosity calibration effect is more accurate and robust, and the performance of the direct method monocular SLAM can be remarkably improved.

The invention discloses a monocular SLAM method capable of creating a large-scale map, and the method comprises the steps: obtaining the upper end image information of a space needing to be mapped through an image collection device, obtaining the non-upper end image information of the space needing to be mapped through a 3D laser radar, and carrying out the data processing of an environment image, and constructing an initial environment map and identifying the pose of the image acquisition device. Wide-angle rotary scanning and fixed-point detection are achieved through the 3d laser radar, a large-distance blind area is avoided in the detection process, the comprehensiveness and high resolution of detection data are guaranteed, the positioning precision of monocular SLAM is optimized, richer map information is created, two initial images are obtained through shooting of the two image acquisition devices, the initial SLAM map is constructed by utilizing the mutually matched feature points in the initial image, and after the initialization is successful, the image is shot by utilizing the image acquisition device so as to perform monocular SLAM mapping, so that the mapping success rate is improved, and the information loss in the map is reduced.

The present invention discloses an SIFT (Scale Invariant Feature Transform) feature detection optimization method based on local area substantial parameter indexes. The method comprises the steps of:detecting corresponding SIFT feature points; equally dividing an image into local areas, and selecting local substantial areas; for each local substantial area, selecting features points from the local substantial areas; and performing matching of the selected SIFT feature points with a current feature map. The SIFT feature detection optimization method based on the local area substantial parameter indexes is helpful for the convergence speed of a monocular SLAM (Simultaneous Localization and Mapping) system and is helpful for accurate description of a scene environment.

The invention discloses a monocular SLAM robust initialization method based on multiple frames, and the method comprises the steps: extracting feature points of image frames in an initial video stream, carrying out the mutual matching, screening out matching points, and obtaining an initial matching point pair; screening out a three-view pair according to the initial matching point pair, then screening out matching points in the three-view pair based on a random sampling consensus algorithm of a trifocal tensor, and constructing a three-frame matching graph; solving relative rotation among the image frames according to a double-vision geometric principle; solving global rotation according to relative rotation among the image frames; solving global displacement based on global rotation; integrating the global rotation and the global displacement to obtain an initial pose of each frame, and performing nonlinear optimization adjustment according to the initial pose; calculating depth of field of the feature points, and recovering three-dimensional coordinates of the feature points; according to the method, the convergence speed can be increased, scatter points are reduced, and therefore the precision of the initial map is improved.

The present application discloses a heterogeneous binocular SLAM method, including: respectively constructing a first monocular SLAM and a second monocular SLAM for the first monocular camera and the second monocular camera, wherein the first monocular camera and The second monocular camera is a heterogeneous camera; using feature points located in the binocular common field of view of the first monocular camera and the second monocular camera to respectively correct the first monocular SLAM and the The scale of the second monocular SLAM; and performing data fusion on the corrected first monocular SLAM and the second monocular SLAM to construct a binocular SLAM. Also disclosed are a heterogeneous binocular SLAM device, electronic equipment and a non-temporary storage medium. Through the heterogeneous binocular SLAM method of this application, binocular SLAM can be constructed more accurately, and a wider viewing angle and deeper depth of field can be obtained, and items located in different depths of field can be detected, which improves the accuracy and accuracy of binocular SLAM Spend.