33 results about "Stereo pair" patented technology

A system for implementing a shared mixed reality experience to participants in a same physical room having a plurality of VR headsets, each of which is adapted to be worn by a participant in the room. Each VR headset having a forward-facing color camera stereo pair. The system includes a computer in communication with at least one of the VR headsets. The system includes a memory in communication with the computer. The memory storing an original 3D digital representation of the room. The computer calculates a rendered VR scene for the at least one of the headsets from a point of view of each of the participant's two eyes, based on the VR headset's current position and orientation. A method for implementing a shared mixed reality experience to participants in a same physical room.

A computer that includes a processor and a memory, the memory including instructions executable by the processor to select first side stereo images of first and second pairs of stereo images acquired at first and second time steps, respectively, and mask first and second first side stereo images, determine point features in masked first and second first side stereo images and determine line features in masked first and second first side stereo images. Matching point features in the masked first and second first side stereo images can be determined using a first attention / graph neural network (attn / GNN) based on keypoints determined based on the line features. Matching line features in the masked first and second first side stereo images can be determined using a second attn / GNN based on keypoints determined based on the line features. Three-dimensional (3D) locations in a scene can be determined by determining stereo disparity based on the matched point features included in the first first side stereo image and point features determined in a first second side stereo image of first and second stereo pairs of images and a 3D stereo camera pose can be determined by determining a perspective-n-point and line algorithm on the 3D locations.

An image processing device according to an embodiment generates a stereoscopic image presented to a plurality of users from an original image. The image processing device is a computer including a processor. The processor is configured to: discretely divide an assumed viewpoint position of a tracked user who is a main viewer of the stereoscopic image; acquire an actual viewpoint position of the tracked user; generate left and right parallax inducing patterns from viewpoint images obtained by capturing an object included in the original image from a plurality of viewpoint positions, with a viewpoint image at the actual viewpoint position as a reference; and generate a stereo pair image including an image obtained by adding the parallax inducing patterns to a reference image to be presented and an image obtained by subtracting the parallax inducing patterns from the reference image.

The invention discloses a depth estimation method and device based on a multi-view fisheye camera, equipment and a medium, and relates to the technical field of computer vision, and the method comprises the steps: obtaining a fisheye image covering a 360-degree panorama through a plurality of fisheye cameras; constructing a three-dimensional pair of Karisi projection in a view overlapping area in fisheye images shot by two adjacent fisheye cameras; performing stereo matching on each stereo pair to obtain a disparity map and a confidence map of each stereo pair; converting each disparity map into a depth map; converting each confidence map and each depth map from Karisy projection to equidistant columnar projection, and fusing each depth map according to each confidence map to obtain a fused omnidirectional depth map; and carrying out refining processing on the fused omnidirectional depth map to eliminate a difference error generated by conversion from the Karzeni projection to the equidistant columnar projection and a detail error generated during fusion so as to obtain a target omnidirectional depth map. According to the method, the accuracy and the calculation efficiency are improved by constructing the three-dimensional pair of the Karisi projection.

The invention discloses an enhanced 3D (three-dimensional) surface reconstruction method based on 3D (three-dimensional) Gaussian Splitting. Global consistent depth priori is obtained through virtual stereo pair rendering; constructing a factor graph and introducing a cross-view geometry / luminosity consistency constraint to form local beam adjustment loss; the prior is used as a learnable parameter to be combined with 3DGS to be optimized, and meanwhile, the Pull loss is assisted to pull low-credibility pixels; and finally, multi-loss function end-to-end training is adopted. The method comprises the following steps of: in Tanksamp; the method has the advantages that F1 is equal to 0.58 and DTU Chamfer is equal to 0.48 mm on a Temples data set, the training time is only 20 min, compared with the prior art, geometric accuracy SOTA and speed magnitude improvement are achieved at the same time, and the method is suitable for VR / AR, robot and industrial measurement scenes.

The invention relates to a local-to-global deep space orbiter image stereo image pair optimization method, which comprises the following steps of: acquiring deep space orbiter images, performing image preprocessing, determining an initial stereo image pair by judging an overlapping area between the images, and performing image pair preliminary screening to obtain a preliminarily screened stereo image pair set; obtaining information of a region of interest, carrying out rasterization and gridding processing on the region of interest, calculating the local quality cost of each stereo image pair for each grid, and obtaining a local optimal image pair and a local alternative image pair set corresponding to the current grid based on the local quality cost; calculating the global quality cost of each stereo image pair based on the local optimal image pair and the local alternative image pair set corresponding to each grid; and combining the local quality cost and the global quality cost of the stereo image pair to obtain a final optimized stereo image pair. Compared with the prior art, the method can automatically and effectively select the stereo image pair meeting the requirement of photogrammetry mapping.

The application discloses a kind of railway three-dimensional geographic scene data service production methods based on red blue stereo sheet, comprising: S1, preparation basic data, S2, organization data, S3, production stereo sheet, S4, associated railway line position data, S5, stereo scene map service release.The method of the application constructs stereo pair using aerial survey original aerial photograph, generates red blue stereo sheet, realizes that only low-cost red blue glasses can be viewed stereo scene;Through the analysis of stereo pair coverage and railway mileage point positioning information, the association of stereo pair and railway line position is realized, and the interested area is quickly searched and positioned;By making red blue stereo sheet into standard format map service data and publishing service, multiple users can be quickly called and observed through network transmission.

A method and a system are disclosed for maintaining accuracy of a photogrammetry system comprising a stereo pair of cameras and characterized by calibration parameters determined at initialization, the system for tracking one of a touch probe and a 3D sensor, the method comprising in use, continuously detecting a presence of a reduced number of 3D target points comprising at least one pair of 3D target points selected in a group comprising at least two 3D target points; measuring image position data associated with the at least one pair of 3D target points of the reduced number of 3D target points; computing at least one updated calibration parameter using the measured image position data and corresponding reference distance data associated with the at least one pair of 3D target points of the reduced number of 3D target points; and updating at least one calibration parameter of the photogrammetry system.

Eyewear having asynchronous rolling shutter (RS) cameras such that the images produced by each camera are misaligned and a state including velocity and gravity direction in the eyewear frame of reference is computed. The state is not limited to these two parameters as other parameters such as acceleration bias, gyroscope bias, or both can be included. A mathematical solver is used such that the processing time to compute the velocity and gravity direction is acceptable. Arranging the RS cameras in an asynchronous configuration allows the motion of the eyewear to be estimated from only one stereo pair of images and does not require more images to be processed.

The application discloses a multi-view fisheye camera-based depth estimation method and device, equipment and medium, relates to the technical field of computer vision, and the method comprises the following steps: acquiring fisheye images covering 360-degree panorama through multiple fisheye cameras; constructing a stereo pair of Cassini projection in the visual field overlapping area of fisheye images shot by adjacent two fisheye cameras; performing stereo matching on each stereo pair to obtain a disparity map and a confidence map of each stereo pair; converting each disparity map into a depth map; converting each confidence map and each depth map from Cassini projection to equirectangular projection, and then fusing each depth map according to each confidence map to obtain a fused omnidirectional depth map; and performing refinement processing on the fused omnidirectional depth map to eliminate the difference error generated when converting from Cassini projection to equirectangular projection and the detail error generated during fusion, so as to obtain a target omnidirectional depth map. The application improves the accuracy and calculation efficiency by constructing a stereo pair of Cassini projection.

A method for training a neural network to perform 3D object manipulation is described. The method includes extracting features from each image of a synthetic stereo pair of images. The method also includes generating a low-resolution disparity image based on the features extracted from each image of the synthetic stereo pair of images. The method further includes generating, by the neural network, a feature map based on the low-resolution disparity image and one of the synthetic stereo pair of images. The method also includes manipulating an unknown object perceived from the feature map according to a perception prediction from a prediction head.

A system for implementing a shared mixed reality experience to participants in a same physical room having a plurality of VR headsets, each of which is adapted to be worn by a participant in the room. Each VR headset having a forward-facing color camera stereo pair. The system includes a computer in communication with at least one of the VR headsets. The system includes a memory in communication with the computer. The memory storing an original 3D digital representation of the room. The computer calculates a rendered VR scene for the at least one of the headsets from a point of view of each of the participant's two eyes, based on the VR headset's current position and orientation. A method for implementing a shared mixed reality experience to participants in a same physical room.

In examples, system and methods for stereo disparity based hazard detection for autonomous machine applications are presented. Example embodiments may assist an ego-machine in detecting hazards within its path of travel. The systems and methods may use disparity between a stereo pair of images to generate a baseline path disparity model and further identify hazards from detected disparities that deviate from that path disparity model. A disparity map for the image pair is constructed in which each pixel represents a disparity for a corresponding element of the image captured. Blockwise division may be optionally used to subdivide the disparity map into a plurality of smaller disparity maps, each corresponding to a block of pixels of the disparity map. A V-space disparity map, where a first axis corresponds to disparity values and the second axis corresponds to pixel rows, may be used to simplify estimation of the path disparity model.

The invention discloses an audio rendering method and device, equipment and a storage medium, and relates to the technical field of audio signal processing, and the method comprises the steps: obtaining a target 2D audio, and separating a target stereo from the target 2D audio; performing sound source object type analysis on the target stereo to determine whether a target sound source object corresponding to the target stereo is a point sound source, if the target sound source object is the point sound source, converting the target stereo into a monaural audio signal, generating first rendering metadata based on the monaural audio signal, and generating second rendering metadata based on the first rendering metadata; rendering the monaural audio signal based on the first rendering metadata; and if the sound source is not the point sound source, generating second rendering metadata based on the left channel signal and the right channel signal of the target stereo, and rendering the left channel signal and the right channel signal based on the second rendering metadata. By judging the type of the sound source, the problems of sound field information loss and sound quality degradation are solved.

In various examples, systems and methods for machine learning based hazard detection for autonomous machine applications using stereo disparity are presented. Disparity between a stereo pair of images is used to generate a path disparity model. Using the path disparity model, a machine learning model can recognize when a pixel in the first image corresponds to a pixel in the second image even though the pixel in the two images does not have identical characteristics. Similarities in extracted feature vectors can be computed and represented by a vector similarity metric that is input to a machine learning classifier, along with feature information extracted from the stereo image pair, to differentiate hazard pixels from non-hazard pixels. In some embodiments, a V-space disparity map, where a first axis corresponds to disparity values and the second axis corresponds to pixel rows, may be used to simplify estimation of the path disparity model.

Techniques related to multi-level optical flow estimation are discussed. Such techniques include partitioning each pair of input images into one or more partitions, separately performing optical flow estimation on the partitions, and merging the separately generated optical flow results into a final optical flow map for the pair of input images.

A method for 3D object perception is described. The method includes extracting features from each image of a synthetic stereo pair of images. The method also includes generating a low-resolution disparity image based on the features extracted from each image of the synthetic stereo pair images. The method further includes predicting, by a trained neural network, a feature map based on the low-resolution disparity image and one of the synthetic stereo pair of images. The method also includes generating, by a perception prediction head, a perception prediction of a detected 3D object based on the feature map predicted by the trained neural network.

A method includes obtaining, using multiple imaging sensors of an electronic device, a left image frame and a right image frame forming a stereo pair of image frames. The method also includes identifying, using at least one processing device of the electronic device, extrinsic parameters associated with relative positions and orientations of the imaging sensors. The method further includes performing, using the at least one processing device, an online stereo rectification of the stereo pair of image frames based on the extrinsic parameters such that epipolar lines of the left and right image frames are horizontally aligned to generate a rectified stereo pair of image frames. In addition, the method includes rendering, using the at least one processing device, one or more images for display based on the rectified stereo pair of image frames.

Various aspects include portable speakers and methods of controlling such speakers. In a particular implementation, a portable loudspeaker includes: a controller configured to: control an audio output according to at least two distinct equalization profiles and in at least two distinct physical orientations, and switch between two of the distinct equalization profiles in response to detecting a change in physical orientation of the portable loudspeaker between two of the distinct physical orientations, where at least one of the equalization profiles comprises a pairing profile configured for outputting audio while the portable loudspeaker is in a stereo pair of loudspeakers or a stereo grouping of loudspeakers.

Embodiments are described for detecting optical discrepancies associated with image capture analyzing pixels in multiple images corresponding to common points of reference in a physical environment. In an embodiment, photometric error values are averaged over time to compute the mean error at each pixel. Once the estimate of the mean error has a sufficient number of updates above a specified value, the estimate is thresholded to provide a mask of any optical discrepancies occurring in the stereo pair of images. Applications include detecting optical discrepancies in images captured for use by a visual navigation system in guiding an autonomous vehicle (e.g., an unmanned aerial vehicle).

An active multiview 3D (AM3D) display is considered, in which detectors are used to recognize the observer's position both parallel and perpendicular to the display surface. Using various modes for shifting at least one of the objects of the AM3D display, such as parallax barriers, 2D displays, or the 3D image itself, the proposed method allows the viewing area of ​​the stereo pairs to be extended and prevents the observer's gaze from entering the 3D image breakup area. Furthermore, the proposed approach allows the parameters of a 3D image to be changed depending on the distance between the observer and the AM3D display, in order to make the perception of 3D objects on the AM3D display as realistic as possible.

In various examples, systems and methods for machine learning based hazard detection for autonomous machine applications using stereo disparity are presented. Disparity between a stereo pair of images is used to generate a path disparity model. Using the path disparity model, a machine learning model can recognize when a pixel in the first image corresponds to a pixel in the second image even though the pixel in the two images does not have identical characteristics. Similarities in extracted feature vectors can be computed and represented by a vector similarity metric that is input to a machine learning classifier, along with feature information extracted from the stereo image pair, to differentiate hazard pixels from non-hazard pixels. In some embodiments, a V-space disparity map, where a first axis corresponds to disparity values and the second axis corresponds to pixel rows, may be used to simplify estimation of the path disparity model.

A method includes obtaining images of a scene captured using a stereo pair of imaging sensors of an XR device and depth data associated with the images, where the scene includes multiple objects. The method also includes obtaining volume-based 3D models of the objects. The method further includes, for one or more first objects, performing depth-based reprojection of the one or more 3D models of the one or more first objects to left and right virtual views based on one or more depths of the one or more first objects. The method also includes, for one or more second objects, performing constant-depth reprojection of the one or more 3D models of the one or more second objects to the left and right virtual views based on a specified depth. In addition, the method includes rendering the left and right virtual views for presentation by the XR device.

A method includes obtaining, using multiple imaging sensors of an electronic device, a left image frame and a right image frame forming a stereo pair of image frames. The method also includes identifying, using at least one processing device of the electronic device, extrinsic parameters associated with relative positions and orientations of the imaging sensors. The method further includes performing, using the at least one processing device, an online stereo rectification of the stereo pair of image frames based on the extrinsic parameters such that epipolar lines of the left and right image frames are horizontally aligned to generate a rectified stereo pair of image frames. In addition, the method includes rendering, using the at least one processing device, one or more images for display based on the rectified stereo pair of image frames.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for detection of pixel correspondences in stereo imaging through the use of polarization data. One of the methods include: obtaining a first image from a first viewpoint of a stereo pair and a second image from a second viewpoint of the stereo pair; obtaining an angle of linear polarization (AOLP) map and a degree of linear polarization (DOLP) map having a field of view overlapping the first image and the second image; computing a surface normal map based on the AOLP map and the DOLP map; and detecting corresponding pixels between the first image and the second image by computing a disparity map that minimizes an energy function comprising a pixel matching cost term and a polarized smoothness regularization term computed based on pixels in a local neighborhood and based on the surface normal map.

In various examples, system and methods for stereo disparity based hazard detection for autonomous machine applications are presented. Example embodiments may assist an ego-machine in detecting hazards within its path of travel. The systems and methods may use disparity between a stereo pair of images to generate a baseline path disparity model and further identify hazards from detected disparities that deviate from that path disparity model. A disparity map for the image pair is constructed in which each pixel represents a disparity for a corresponding element of the image captured. Blockwise division may be optionally used to subdivide the disparity map into a plurality of smaller disparity maps, each corresponding to a block of pixels of the disparity map. A V-space disparity map, where a first axis corresponds to disparity values and the second axis corresponds to pixel rows, may be used to simplify estimation of the path disparity model.

Described herein are methods and systems for obtaining high-quality images, which may be suitable for various applications such as training and operating artificial intelligence (AI) systems. Specifically, a system may comprise multiple cameras and one or more actuators that are capable of moving these cameras relative to each other. For example, these cameras may form one or more stereo pairs, each pair having its stereo axis. The actuators can change baselines in these pairs and / or tilt these stereo axes relative to the imaged objects to address possible self-similarity issues associated with the shape of these objects and their orientation relative to the cameras. In some examples, the simultaneous images captured by these cameras are used to construct a three-dimensional (3D) model. The fidelity of this model is then used to determine the position of the cameras (as a camera unit or individually for each camera).

A method for 3D object perception is described. The method includes extracting features from each image of a synthetic stereo pair of images. The method also includes generating a low-resolution disparity image based on the features extracted from each image of the synthetic stereo pair images. The method further includes predicting, by a trained neural network, a feature map based on the low-resolution disparity image and one of the synthetic stereo pair of images. The method also includes generating, by a perception prediction head, a perception prediction of a detected 3D object based on the feature map predicted by the trained neural network.

Techniques related to multi-level optical flow estimation are discussed. Such techniques include partitioning each pair of input images into one or more partitions, separately performing optical flow estimation on the partitions, and merging the separately generated optical flow results into a final optical flow map for the pair of input images.

The information extraction and three-dimensional reconstruction method based on super-generalized stereo pair relates to the field of trajectory planning. The present application is to solve the problem that the existing stereo pair modeling method cannot meet the standard stereo pair limitation condition, and it is difficult to reconstruct the overall structure of the building with a small number of local points, resulting in the difficulty of extracting the missing information in the perspective blind area of the orthographic image. The present application comprises: obtaining a building oblique remote sensing image, and performing perspective instance segmentation on the building oblique remote sensing image to obtain a perspective instance segmentation result; performing information repair on the perspective instance segmentation result to obtain a single building image; matching the same repaired single building image from different building oblique remote sensing images; based on the matching result, using the image of each single building to realize super-generalized stereo pair three-dimensional reconstruction; the super-generalized stereo pair is any oblique remote sensing image covering a certain area. The present application is used for three-dimensional reconstruction of buildings.