75 results about "Structure from motion" patented technology

Dense range data obtained at real-time rates is employed to estimate the pose of an articulated figure. In one approach, the range data is used in combination with a model of connected patches. Each patch is the planar convex hull of two circles, and a recursive procedure is carried out to determine an estimate of pose which most closely correlates to the range data. In another aspect of the invention, the dense range data is used in conjunction with image intensity information to improve pose tracking performance. The range information is used to determine the shape of an object, rather than assume a generic model or estimate structure from motion. In this aspect of the invention, a depth constraint equation, which is a counterpart to the classic brightness change constraint equation, is employed. Both constraints are used to jointly solve for motion estimates.

Simultaneous localization and mapping (SLAM) utilizes multiple view feature descriptors to robustly determine location despite appearance changes that would stifle conventional systems. A SLAM algorithm generates a feature descriptor for a scene from different perspectives using kernel principal component analysis (KPCA). When the SLAM module subsequently receives a recognition image after a wide baseline change, it can refer to correspondences from the feature descriptor to continue map building and/or determine location. Appearance variations can result from, for example, a change in illumination, partial occlusion, a change in scale, a change in orientation, change in distance, warping, and the like. After an appearance variation, a structure-from-motion module uses feature descriptors to reorient itself and continue map building using an extended Kalman Filter. Through the use of a database of comprehensive feature descriptors, the SLAM module is also able to refine a position estimation despite appearance variations.

Determining three-dimensional structure in a road environment using a system mountable in a host vehicle including a camera connectible to a processor. Multiple image frames are captured in the field of view of the camera. In the image frames, a line is selected below which the road is imaged. The line separates between upper images essentially excluding images of the road and lower images essentially including images of the road. One or more of the lower images is warped, according to a road homography to produce at least one warped lower image. The three-dimensional structure may be provided from motion of a matching feature within the upper images or from motion of a matching feature within at least one of the lower images and at least one warped lower image.

The invention relates to an unmanned aerial vehicle aerial photography sequence image-based slope three-dimension reconstruction method. The method includes the following steps that: feature region matching and feature point pair extraction are performed on un-calibrated unmanned aerial vehicle multi-view aerial photography sequence images through adopting a feature matching-based algorithm; the geometric structure of a slope and the motion parameters of a camera are calculated through adopting bundle adjustment structure from motion and based on disorder matching feature points, and therefore, a sparse slope three-dimensional point cloud model can be obtained; the sparse slope three-dimensional point cloud model is processed through adopting a patch-based multi-view stereo vision algorithm, so that the sparse slope three-dimensional point cloud model can be diffused to a dense slope three-dimensional point cloud model; and the surface mesh of the slope is reconstructed through adopting Poisson reconstruction algorithm, and the texture information of the surface of the slop is mapped onto a mesh model, and therefore, a vivid three-dimensional slope model with high resolution can be constructed. The unmanned aerial vehicle aerial photography sequence image-based slope three-dimension reconstruction method of the invention has the advantages of low cost, flexibility, portability, high imaging resolution, short operating period, suitability for survey of high-risk areas and the like. With the method adopted, the application of low-altitude photogrammetry and computer vision technology to the geological engineering disaster prevention and reduction field can be greatly prompted.

Robust techniques for self-calibration of a moving camera observing a planar scene. Plane-based self-calibration techniques may take as input the homographies between images estimated from point correspondences and provide an estimate of the focal lengths of all the cameras. A plane-based self-calibration technique may be based on the enumeration of the inherently bounded space of the focal lengths. Each sample of the search space defines a plane in the 3D space and in turn produces a tentative Euclidean reconstruction of all the cameras that is then scored. The sample with the best score is chosen and the final focal lengths and camera motions are computed. Variations on this technique handle both constant focal length cases and varying focal length cases.

The present invention presents a framework for separating specular and diffuse reflection components in images and videos. Each pixel of the an M-channel input image illuminated by N light sources is linearly transformed into a new color space having (M-N) channels. For an RGB image with one light source, the new color space has two color channels (U,V) that are free of specularities and a third channel (S) that contains both specular and diffuse components. When used with multiple light sources, the transformation may be used to produce a specular invariant image. A diffuse RGB image can be obtained by applying a non-linear partial differential equation to an RGB image to iteratively erode the specular component at each pixel. An optional third dimension of time may be added for processing video images. After the specular and diffuse components are separated, dichromatic editing may be used to independently process the diffuse and the specular components to add or suppress visual effects. The (U,V) channels of images can be used as input to 3-D shape estimation algorithms including shape-from-shading, photometric stereo, binocular and multinocular stereopsis, and structure-from-motion.

The inventions herein relate generally to improvements in photogrammetry and devices suitable for obtaining such improvements. Some embodiments use only a single passive image-capture device to obtain overlapping 2D images, where such images at least partially overlap with regard to at least one object of interest in a scene. Such images can be processed using methods incorporating structure from motion algorithms. Accurate 3D digital representations of the at least one object of interest can be obtained. Substantially accurate measurements and other useful information regarding the at least one object of interest are obtainable from the methodology herein.

A method based on Structure from Motion for processing a plurality of sparse images acquired by one or more acquisition devices to generate a sparse 3D points cloud and of a plurality of internal and external parameters of the acquisition devices includes the steps of collecting the images; extracting keypoints therefrom and generating keypoint descriptors; organizing the images in a proximity graph; pairwise image matching and generating keypoints connecting tracks according maximum proximity between keypoints; performing an autocalibration between image clusters to extract internal and external parameters of the acquisition devices, wherein calibration groups are defined that contain a plurality of image clusters and wherein a clustering algorithm iteratively merges the clusters in a model expressed in a common local reference system starting from clusters belonging to the same calibration group; and performing a Euclidean reconstruction of the object as a sparse 3D point cloud based on the extracted parameters.

The invention relates to a target reconstruction method based on a geometric constraint and belongs to the computer vision field. The method comprises the following steps of through a structure from motion (SFM) method, acquiring initial point cloud; through image characteristic point clustering, acquiring a classification result of characteristic points, wherein the classification result means aneighborhood relation of similar portions in an image; carrying out normal characteristic clustering of the initial point cloud, and using a corresponding relation between the classification result ofthe image characteristic points and an initial point cloud clustering result to define a geometric structure of the initial point cloud; using the geometric structure to acquire a sparse portion in the initial point cloud, defining the portion as a ''hole'', and then using a combined structure constraint of a ''hole'' area to carry out fitting of a space plane and a curved surface through an RANSAC method and a least square method; and sampling a fitted surface, adding an acquired three-dimensional point into the initial point cloud so as to acquire a dense point cloud model, and finally using a Poisson surface to reconstruct and acquire a three-dimensional model of a target. Through an experiment result, implementation of the method is verified and a good effect is achieved.

A system provides camera position and point cloud estimation 3D reconstruction. The system receives images and attempts existing structure integration to integrate the images into an existing reconstruction under a sequential image reception assumption. If existing structure integration fails, the system attempts dictionary overlap detection by accessing a dictionary database and searching to find a closest matching frame in the existing reconstruction. If overlaps are found, the system matches the images with the overlaps to determine a highest probability frame from the overlaps, and attempts existing structure integration again. If overlaps are not found or existing structure integration fails again, the system attempts bootstrapping based on the images. If any of existing structure integration, dictionary overlap detection, or bootstrapping succeeds, and if multiple disparate tracks have come to exist, the system attempts reconstructed track merging.

Plane detection and tracking algorithms are described that may take point trajectories as input and provide as output a set of inter-image homographies. The inter-image homographies may, for example, be used to generate estimates for 3D camera motion, camera intrinsic parameters, and plane normals using a plane-based self-calibration algorithm. A plane detection and tracking algorithm may obtain a set of point trajectories for a set of images (e.g., a video sequence, or a set of still photographs). A 2D plane may be detected from the trajectories, and trajectories that follow the 2D plane through the images may be identified. The identified trajectories may be used to compute a set of inter-image homographies for the images as output.

The invention provides a super pixel-based target reconstruction method. The method is formed by structure-from-motion (SFM) projection matrix estimation, multi-view stereoscopic vision matching undersuper pixels and synthesis and fusion of depth graphs. A specific process includes the six major steps of: step 1, reading an image sequence, and utilizing a structure-from-motion method to estimatea camera projection matrix; step 2, carrying out super-pixel segmentation on image pairs; step 3, calculating possible depth values for each super pixel; step 4, utilizing an MRF model to select optimal depth values of the super pixels; step 5, creating a multi-scale super-pixel framework; and step 6, carrying out depth graph fusion and surface meshing. According to the method, a disadvantage of insufficient precision of super pixel-based stereoscopic vision matching is overcome, advantages that the same is high in robustness for noises and brightness deviations, can accurately provide targetcontour information, and is low in calculation complexity are utilized, better reconstruction results can be achieved for both texture regions and non-texture regions, and universality is high. The method has broad application backgrounds.

The invention discloses a real-time video camera tracking method based on key frames, which comprises the following steps: (1) capturing an index image sequence, and restoring a sparse three-dimensional characteristic point structure of a scene by a method of inferring a structure from movement; (2) giving the index image sequence and the sparse three-dimensional characteristic point structure, and automatically selecting the key frames by optimizing an energy function related to the key frames; (3) in the real-time video camera tracking process, for each frame of real-time input image, firstly, quickly positioning a candidate key frame similar to the real-time input image from the key frame set by a characteristic word tree method of image recognition; and (4) matching the characteristic points abstracted on the real-time input image with the characteristic points on the candidate key frame, obtaining the corresponding three dimensional coordinates of the characteristic points on the image, and calculating the directionality parameters of the video camera. The invention has high calculating efficiency and stable solving result, and the video camera tracking result obtained by the method can be directly used for the applications of reality enhancement, virtual interaction and the like.

A method for improving geometry-based monocular structure from motion (SfM) by exploiting depth maps predicted by convolutional neural networks (CNNs) is presented. The method includes capturing a sequence of RGB images from an unlabeled monocular video stream obtained by a monocular camera, feeding the RGB images into a depth estimation/refinement module, outputting depth maps, feeding the depth maps and the RGB images to a pose estimation/refinement module, the depths maps and the RGB images collectively defining pseudo RGB-D images, outputting camera poses and point clouds, and constructing a 3D map of a surrounding environment displayed on a visualization device.

The invention discloses a point cloud hole repairing method based on SFM (Structure From Motion), which includes the following steps: (1) acquiring point cloud data through the grating projection method and the SFM method, and extracting the boundary points of a 3D point cloud hole according to point cloud 2D phase information acquired using the grating projection method; (2) registering a point cloud data set acquired using the SFM method and a point cloud data set acquired using the grating projection method; (3) acquiring the supplement points of the hole area in the point cloud acquired using the grating projection method from the data set acquired using the SFM method; and (4) based on the supplement points, using a radial basis function to further repair the hole. The algorithm is robust. The repair effect is highly precise. More detail information of an object can be restored.

The invention relates to a sequence image's automatic splicing method based on three-dimension reconstruction. The method comprises the following steps: extracting the scale-invariant feature transform (SIFT) feature points respectively from N inputted images; based on the matching condition of the feature points, selecting m candidate matching images corresponding to each image to structure a candidate matching image set; performing three-dimension reconstruction to the candidate matching image set through the use of the structure from motion (SfM) algorithm to obtain a reflected and projected three-dimension plane; seeking the two-dimension reference plane corresponding to the three-dimension plane and projecting it to a designated two-dimension coordinate plane; seeking the mirror distortion parameter of each image and optimizing the splicing effect between adjacent images. Compared with the prior art, the invention is based on the three-dimension point cloud reconstruction method to restore the three-dimension structure of a photographed object and is able to solve the problems with the sequence image splicing when the images and the photographed object cannot meet the homograph restrain conditions, eliminates the homograph distortion of the inputted images and improves the image splicing quality.

The invention relates to the technical field of stereo vision in computer vision, and particularly to a depth restoring method for three-dimensional reconstruction. After depths of sparse characteristic points in an image are obtained through a structure from motion (SFM), depth is diffused based on the sparse characteristic points and a gray scale image through multilayer down-sampling and a double-side filter. An accurate depth map is quickly restored from low resolution to high resolution and from rough to fine in a layered manner. The method has advantages of accurate result and low calculating amount. The sparse characteristic points which are obtained through calculation in the system can be restored to the dense depth map by means of a simultaneous localization and mapping (SLAM) system based on a characteristic point method, thereby reconstructing the three-dimensional dense map.

Volume determining method for an object (1), particularly a stock pile, on a construction site, comprising –  moving a mobile camera (8) along a path (10) around the object (1) while orienting the camera (8) repeatedly or continuously, onto the object (1), –  in the course of this, capturing a series of images of the object (1) with the camera (8), the series comprising a plurality of images captured from different points on the path (10) and with different orientations of the camera (8), the series being represented by an image data set collected thereby, –  performing a structure from motion evaluation with a defined algorithm using the series of images and generating a point cloud spatial representation comprising a surface of the object (1) therefrom, –  scaling the spatial representation with help of a known absolute reference regarding scale, the scale being imaged by the camera (8) together with the object (1), –  defining a ground surface for the object (1) and applying it onto the spatial representation and –  calculating and outputting the absolute volume of the object (1) based on the scaled spatial representation and the defined ground surface.

A method for extracting keyframes from a sequence of frames for a computer vision application using structure from motion, the keyframes being a subset of representative frames from the complete sequence of frames, and an apparatus configured to perform the method are described. A subset selector selects a subset of keyframes that closely match a current camera position from already available keyframes. A determination unit then determines whether a current frame should be included a bundle adjustment keyframe set and/or a triangulation keyframe set.