8406 results about "Point cloud" patented technology

The present invention provides a system (method and apparatus) for creating photorealistic 3D models of environments and/or objects from a plurality of stereo images obtained from a mobile stereo camera and optional monocular cameras. The cameras may be handheld, mounted on a mobile platform, manipulator or a positioning device. The system automatically detects and tracks features in image sequences and self-references the stereo camera in 6 degrees of freedom by matching the features to a database to track the camera motion, while building the database simultaneously. A motion estimate may be also provided from external sensors and fused with the motion computed from the images. Individual stereo pairs are processed to compute dense 3D data representing the scene and are transformed, using the estimated camera motion, into a common reference and fused together. The resulting 3D data is represented as point clouds, surfaces, or volumes. The present invention also provides a system (method and apparatus) for enhancing 3D models of environments or objects by registering information from additional sensors to improve model fidelity or to augment it with supplementary information by using a light pattern projector. The present invention also provides a system (method and apparatus) for generating photo-realistic 3D models of underground environments such as tunnels, mines, voids and caves, including automatic registration of the 3D models with pre-existing underground maps.

A method and apparatus for automatically generating a three-dimensional computer model from a “point cloud” of a scene produced by a laser radar (LIDAR) system. Given a point cloud of an indoor or outdoor scene, the method extracts certain structures from the imaged scene, i.e., ceiling, floor, furniture, rooftops, ground, and the like, and models these structures with planes and/or prismatic structures to achieve a three-dimensional computer model of the scene. The method may then add photographic and/or synthetic texturing to the model to achieve a realistic model.

A method and apparatus for performing two-dimensional video alignment onto three-dimensional point clouds. The system recovers camera pose from camera video, determines a depth map, converts the depth map to a Euclidean video point cloud, and registers two-dimensional video to the three-dimensional point clouds.

Systems and methods are provided for extracting various features from data having spatial coordinates. The systems and methods may identify and extract data points from a point cloud, where the data points are considered to be part of the ground surface, a building, or a wire (e.g. power lines). Systems and methods are also provided for enhancing a point cloud using external data (e.g. images and other point clouds), and for tracking a moving object by comparing images with a point cloud. An objects database is also provided which can be used to scale point clouds to be of similar size. The objects database can also be used to search for certain objects in a point cloud, as well as recognize unidentified objects in a point cloud.

Various embodiments are disclosed ladar point cloud compression. For example, a processor can be used to analyze data representative of an environmental scene to intelligently select a subset of range points within a frame to target with ladar pulses via a scanning ladar transmission system. In another example embodiment, a processor can perform ladar point cloud compression by (1) processing data representative of an environmental scene, and (2) based on the processing, selecting a plurality of the range points in the point cloud for retention in a compressed point cloud, the compressed point cloud comprising fewer range points than the generated point cloud.

A system comprises an encoder configured to compress attribute information and/or spatial for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. To compress the attribute and/or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud.

An integrated system generates a model of a three-dimensional object. A scanning laser device scans the three-dimensional object and generates a point cloud. The points of the point cloud each indicate a location of a corresponding point on a surface of the object. A first model is generated, responsive to the point cloud, that generates a first model representing constituent geometric shapes of the object. A data file is generated, responsive to the first model, that can be inputted to a computer-aided design system.

Camera motion is determined in a three-dimensional image capture system using a combination of two-dimensional image data and three-dimensional point cloud data available from a stereoscopic, multi-aperture, or similar camera system. More specifically, a rigid transformation of point cloud data between two three-dimensional point clouds may be more efficiently parameterized using point correspondence established between two-dimensional pixels in source images for the three-dimensional point clouds.

A method compresses a point cloud composed of a plurality of points in a three-dimensional (3D) space by first acquiring the point cloud with a sensor, wherein each point is associated with a 3D coordinate and at least one attribute. The point cloud is partitioned into an array of 3D blocks of elements, wherein some of the elements in the 3D blocks have missing points. For each 3D block, attribute values for the 3D block are predicted based on the attribute values of neighboring 3D blocks, resulting in a 3D residual block. A 3D transform is applied to each 3D residual block using locations of occupied elements to produce transform coefficients, wherein the transform coefficients have a magnitude and sign. The transform coefficients are entropy encoded according the magnitudes and sign bits to produce a bitstream.

A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute information for the point cloud. Attribute values for at least one starting point are included in a compressed attribute information file and attribute correction values used to correct predicted attribute values are included in the compressed attribute information file. Attribute values are predicted based, at least in part, on attribute values of neighboring points and distances between a particular point for whom an attribute value is being predicted and the neighboring points. The predicted attribute values are compared to attribute values of a point cloud prior to compression to determine attribute correction values. A decoder follows a similar prediction process as an encoder and corrects predicted values using attribute correction values included in a compressed attribute information file.

Disclosed is a method of generating a three-dimensional (3D) surface defined by a boundary of a 3D point cloud. The method comprises generating density and depth maps from the 3D point cloud, constructing a 2D mesh from the depth and density maps, transforming the 2D mesh into a 3D mesh, and rendering 3D polygons defined by the 3D mesh.

Provided herein are methods and systems for implementing three-dimensional perception in an autonomous robotic system comprising an end-to-end neural network architecture that directly consumes large-scale raw sparse point cloud data and performs such tasks as object localization, boundary estimation, object classification, and segmentation of individual shapes or fused complete point cloud shapes.

The invention discloses a parallax optimization algorithm-based binocular stereo vision automatic measurement method. The method comprises the steps of 1, obtaining a corrected binocular view; 2, matching by using a stereo matching algorithm and taking a left view as a base map to obtain a preliminary disparity map; 3, for the corrected left view, enabling a target object area to be a colorized master map and other non-target areas to be wholly black; 4, acquiring a complete disparity map of the target object area according to the target object area; 5, for the complete disparity map, obtaining a three-dimensional point cloud according to a projection model; 6, performing coordinate reprojection on the three-dimensional point cloud to compound a coordinate related pixel map; 7, using a morphology method to automatically measure the length and width of a target object. By adopting the method, a binocular measuring operation process is simplified, the influence of specular reflection, foreshortening, perspective distortion, low textures and repeated textures on a smooth surface is reduced, automatic and intelligent measuring is realized, the application range of binocular measuring is widened, and technical support is provided for subsequent robot binocular vision.

The invention discloses a binocular stereo vision based intelligent three-dimensional human face rebuilding method and a system; the method comprises: preprocessing operations including image normalization, brightness normalization and image correction are carried out to a human face image; a human face area in the human face image which is preprocessed is obtained and human face characteristic points are extracted; the object is rebuilt by projection matrix, so as to obtain internal and external parameters of a vidicon; based on the human face characteristic points, gray level cross-correlation matching operators are expanded to color information, and a parallax image generated by stereo matching is calculated according to information including polar line restraining, human face area restraining and human face geometric conditions; a three-dimensional coordinate of a human face spatial hashing point cloud is calculated according to the vidicon calibration result and the parallax image generated by stereo matching, so as to generate a three-dimensional human face model. By adopting the steps, more smooth and vivid three-dimensional human face model is rebuilt in the invention.

A scene point cloud is processed and a solution to an inverse-function is determined to determine its source objects. A primitive extraction process and a part matching process are used to compute the inverse function solution. The extraction process estimates models and parameters based on evidence of cylinder and planar geometry in the scene. The matching process matches clusters of 3D points to models of parts from a library. A selected part and its associated polygon model is used to represent the point cluster. Iterations of the extraction and matching processes complete a 3D model for a complex scene made up of planes, cylinders, and complex parts from the parts library. Connecting regions between primitives and/or parts are processed to determine their existence and type. Constraints may be used to ensure a connected model and alignment of its components.

Methods, apparatus and computer program products provide efficient techniques for reconstructing surfaces from data point sets. These techniques include reconstructing surfaces from sets of scanned data points that have preferably undergone preprocessing operations to improve their quality by, for example, reducing noise and removing outliers. These techniques include reconstructing a dense and locally two-dimensionally distributed 3D point set (e.g., point cloud) by merging stars in two-dimensional weighted Delaunay triangulations within estimated tangent planes. The techniques include determining a plurality of stars from a plurality of points p_i in a 3D point set S that at least partially describes the 3D surface, by projecting the plurality of points p_i onto planes T_i that are each estimated to be tangent about a respective one of the plurality of points p_i. The plurality of stars are then merged into a digital model of the 3D surface.

The invention discloses a three-dimensional enhancing realizing method for multi-viewpoint free stereo display, which comprises the following steps: 1) stereoscopically shooting a natural scene by using a binocular camera; 2) extracting and matching a characteristic point of an image of a main camera, generating a three-dimensional point cloud picture of the natural scene in real time, and calculating a camera parameter; 3) calculating a depth image corresponding to the image of the main camera, drawing a virtual viewpoint image and a depth image thereof, and performing hollow repairing; 4) utilizing three-dimensional making software to draw a three-dimensional virtual model and utilizing a false-true fusing module to realize the false-true fusing of the multi-viewpoint image; 5) suitably combining multiple paths of false-true fused images; and 6) providing multi-viewpoint stereo display by a 3D display device. According to the method provided by the invention, the binocular camera is used for stereoscopically shooting and the characteristic extracting and matching technique with better instantaneity is adopted, so that no mark is required in the natural scene; the false-true fusing module is used for realizing the illumination consistency and seamless fusing of the false-true scenes; and the multi-user multi-angle naked-eye multi-viewpoint stereo display effect is supplied by the 3D display device.

Disclosed in the invention is an improved method of a RGB-D-based simultaneously localization and mapping (SLAM) algorithm. The method comprises two parts: a front-end part and a rear-end part. The front-end part is as follows: feature detection and descriptor extraction, feature matching, motion conversion estimation, and motion conversion optimization. And the rear-end part is as follows: a 6-D motion conversion relation initialization pose graph obtained by the front-end part is used for carrying out closed-loop detection to add a closed-loop constraint condition; a non-linear error function optimization method is used for carrying out pose graph optimization to obtain a global optimal camera pose and a camera motion track; and three-dimensional environment reconstruction is carried out. According to the invention, the feature detection and descriptor extraction are carried out by using an ORB method and feature points with illegal depth information are filtered; bidirectional feature matching is carried out by using a FLANN-based KNN method and a matching result is optimized by using homography matrix conversion; a precise inliners matching point pair is obtained by using an improved RANSAC motion conversion estimation method; and the speed and precision of point cloud registration are improved by using a GICP-based motion conversion optimization method.

A computer-implemented method for estimating a volume of at least one food item on a food plate is disclosed. A first and second plurality of images are received from different positions above a food plate, wherein angular spacing between the positions of the first plurality of images is greater than angular spacing between the positions of the second plurality of images. A first set of poses of each of the first plurality of images is estimated. A second set of poses of each of the second plurality of images is estimated based on at least the first set of poses. A pair of images taken from each of the first and second plurality of images is rectified based on at least the first and second set of poses. A 3D point cloud is reconstructed based on at least the rectified pair of images. At least one surface of the at least one food item above the food plate is estimated based on at least the reconstructed 3D point cloud. The volume of the at least one food item is estimated based on the at least one surface.

A method for ascertaining the axis of rotation of a vehicle wheel in which a light pattern is projected at least onto the wheel during the rotation of the wheel and the light pattern reflected from the wheel is detected by a calibrated imaging sensor system and analyzed in an analyzer device. Accurate and robust measurement of the axis of rotation and, optionally, of the axis and wheel geometry, in particular when the vehicle is passing by, is achieved in that a 3D point cloud with respect to the wheel is determined in the analysis and a parametric surface model of the wheel is adapted thereto; normal vectors of the wheel are calculated for different rotational positions of the wheel for obtaining the axes of rotation; and the axis of rotation vector is calculated as the axis of rotation from the spatial movement of the normal vector of the wheel.

The invention discloses an urban environment composition method for unmanned vehicles. Under the condition of being independent of external positioning sensors such as speedometers, GPS and inertial navigators, the trajectory tracking and environment map building of an unmanned vehicle are completed by just using few particles for 3D laser-point cloud data returned by a vehicle-mounted laser radar, thereby providing basis for the autonomous running of the unmanned ground vehicle in an unknown environment; and according to the invention, an ICP algorithm is applied to adjacent two frames of data so as to obtain a coarse estimation on the real position and posture of a vehicle, and then redundance is performed near the coarse estimation based on gaussian distribution. Although the coarse estimation is not the real position and posture of the vehicle, the coarse estimation is a high-probability area of the real position and posture of the vehicle, so that an effect of relatively accurate positioning and composition is achieved by using a small amount of particles in the process of subsequent redundance, thereby avoiding the fitting of a vehicle trajectory by using a large amount of particles in a traditional method, improving the efficiency of the algorithm, and effectively restraining a phenomenon of particle degeneracy caused by bad particle estimation.

Disclosed are a 3D laser scanning system and a method of obtaining a 3D image by using the system that detect, with a linear array type photo detector, a reflected light reflected from a target after rotation-emitting a line-shaped pulsed laser light through 360 degrees and obtain a 3D image through point cloud data obtained by measuring a distance to the target.

The present invention relates to a navigation device. The navigation device is arranged to dynamically generate multidimensional video signals based on location and directional information of the navigation device by processing at pre-generated 3D data set (3D Model or 3D point cloud). The navigation device is further arranged to superimpose navigation directions and/or environment information about surrounding objects onto the generated multidimensional view.

The invention discloses an autonomous location and navigation method and an autonomous location and navigation system of a robot. The method comprises the following steps: acquiring the RGB image of the robot in the current position; carrying out characteristic point matching on the RGB image of the current position and an indoor three-dimensional point cloud map, and determining the current position of the robot in the indoor 3D point cloud map; carrying out characteristic point matching on a pre-stored RGB image of a destination and the indoor three-dimensional point cloud map, and determining the position of the destination in the indoor 3D point cloud map; and searching an optimal path from the current position to the destination position in the indoor 3D point cloud map, and driving the robot to run according to the optimal path. The method and the system have the advantages of completion of autonomous location and navigation through using a visual sensor, simple device structure, low cost, simplicity in operation, and high path planning real-time property, and can be used in the fields of unmanned driving and indoor location and navigation.

The invention discloses an object detection and tracking method of fusing laser point clouds and images. The method comprises the steps of: S1, respectively collecting laser point cloud data and imagedata of an object, carrying out first object detection according to the collected point cloud data to obtain a first object detection result, and carrying out second object detection according to thecollected image data to obtain a second object detection result; S2, carrying out fusion judgment on the first object detection result and the second object detection result on the basis of Bayesiandecision to obtain final object detection result output; and S3, carrying out object tracking according to a final object detection result. The method has the advantages of a simple and highly-efficient realization method, high object detection and tracking accuracy, high environment adaptability and fault tolerance, stability, reliability and the like.

A method for extracting a 3D terrain model for identifying at least buildings and terrain from LIDAR data is disclosed, comprising the steps of generating a point cloud representing terrain and buildings mapped by LIDAR; classifying points in the point cloud, the point cloud having ground and non-ground points, the non-ground points representing buildings and clutter; segmenting the non-ground points into buildings and clutter; and calculating a fit between at least one building segment and at least one rectilinear structure, wherein the fit yields the rectilinear structure with the fewest number of vertices. The step of calculating further comprises the steps of (a) calculating a fit of a rectilinear structure to the at least one building segment, wherein each of the vertices has an angle that is a multiple of 90 degrees; (b) counting the number of vertices; (c) rotating the at least one building segment about an axis by a predetermined increment; and (d) repeating steps (a)-(c) until a rectilinear structure with the least number of vertices is found.

A method and apparatus are disclosed for finding a triangle mesh that interpolates a set of points obtained from a scanning system. A ball-pivoting algorithm computes a triangle mesh interpolating a given point cloud. The disclosed ball-pivoting algorithm triangulates a set of points by “rolling” a ball of radius r on the point cloud. The points are surface samples acquired with multiple range scans of an object. The ball-pivoting algorithm starts with a seed triangle, and pivots the ball of a given radius, r, around an edge of the triangle. During the pivoting operation, the ball revolves around the edge while keeping in contact with the edge's endpoints. The ball pivots until it touches another scan point, forming another triangle. The ball-pivoting operation continues until all reachable edges have been tried, and then starts from another seed triangle, until all scan points have been considered.

A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute information for the point cloud. Attribute values for at least one starting point are included in a compressed attribute information file and attribute correction values used to correct predicted attribute values are included in the compressed attribute information file. Attribute values are predicted based, at least in part, on attribute values of neighboring points and distances between a particular point for whom an attribute value is being predicted and the neighboring points. The predicted attribute values are compared to attribute values of a point cloud prior to compression to determine attribute correction values. A decoder follows a similar prediction process as an encoder and corrects predicted values using attribute correction values included in a compressed attribute information file.

The invention discloses a mobile robot three-dimensional mapping and obstacle avoidance method based on a space bag of words model. The method comprises following steps: 1) collecting Kinect sensor data, and using a space bag of words model which fuses spatial relationships to describe scene image features 2) describing robot three-dimensional SLAM by means of the SDBoW2 model of the scene image to realize closed loop detection, three-dimensional point cloud registration, and graph structure optimization and therefore creating a global environmental three-dimensional point cloud density map; 3) the robot using the created global three-dimensional map and the Kinect sensor information to perform indoor real-time obstacle avoidance guiding. The method is aimed at low cost mobile robots without speedometers or laser distance measuring sensors; reliable real-time three-dimensional map creation and obstacle avoidance can be realized depending only on Kinect sensors; the method can be applied in long time mobile robot operation service at large area of indoor environment such as household places and office rooms, etc.

The invention relates to a range image-based 3D spatial data processing method and a device. In the method, ground laser radar is adopted to scan an object to obtain 3D point cloud data on the surface of the object; a database object table is established by a PL/SQL programming approach; a reference datum is fit according to point cloud which is segmented in advance to generate a minimum bounding box (MBB) of the point cloud and subsequently generate the range image according to the reference datum obtained by fitting. Later, data of the MBB and the range image are written into the database object table and the relation between the MBB and a range image object is established, subsequently, R tree 3D spatial database index is established for the database objects which enter the database, finally, visualization is realized based on Oracle Objects for OLE (OO4O) and OpenGL.