15 results about "Stereo pair" patented technology

The invention discloses an enhanced 3D (three-dimensional) surface reconstruction method based on 3D (three-dimensional) Gaussian Splitting. Global consistent depth priori is obtained through virtual stereo pair rendering; constructing a factor graph and introducing a cross-view geometry / luminosity consistency constraint to form local beam adjustment loss; the prior is used as a learnable parameter to be combined with 3DGS to be optimized, and meanwhile, the Pull loss is assisted to pull low-credibility pixels; and finally, multi-loss function end-to-end training is adopted. The method comprises the following steps of: in Tanksamp; the method has the advantages that F1 is equal to 0.58 and DTU Chamfer is equal to 0.48 mm on a Temples data set, the training time is only 20 min, compared with the prior art, geometric accuracy SOTA and speed magnitude improvement are achieved at the same time, and the method is suitable for VR / AR, robot and industrial measurement scenes.

A method and a system are disclosed for maintaining accuracy of a photogrammetry system comprising a stereo pair of cameras and characterized by calibration parameters determined at initialization, the system for tracking one of a touch probe and a 3D sensor, the method comprising in use, continuously detecting a presence of a reduced number of 3D target points comprising at least one pair of 3D target points selected in a group comprising at least two 3D target points; measuring image position data associated with the at least one pair of 3D target points of the reduced number of 3D target points; computing at least one updated calibration parameter using the measured image position data and corresponding reference distance data associated with the at least one pair of 3D target points of the reduced number of 3D target points; and updating at least one calibration parameter of the photogrammetry system.

The application discloses a multi-view fisheye camera-based depth estimation method and device, equipment and medium, relates to the technical field of computer vision, and the method comprises the following steps: acquiring fisheye images covering 360-degree panorama through multiple fisheye cameras; constructing a stereo pair of Cassini projection in the visual field overlapping area of fisheye images shot by adjacent two fisheye cameras; performing stereo matching on each stereo pair to obtain a disparity map and a confidence map of each stereo pair; converting each disparity map into a depth map; converting each confidence map and each depth map from Cassini projection to equirectangular projection, and then fusing each depth map according to each confidence map to obtain a fused omnidirectional depth map; and performing refinement processing on the fused omnidirectional depth map to eliminate the difference error generated when converting from Cassini projection to equirectangular projection and the detail error generated during fusion, so as to obtain a target omnidirectional depth map. The application improves the accuracy and calculation efficiency by constructing a stereo pair of Cassini projection.

A method for training a neural network to perform 3D object manipulation is described. The method includes extracting features from each image of a synthetic stereo pair of images. The method also includes generating a low-resolution disparity image based on the features extracted from each image of the synthetic stereo pair of images. The method further includes generating, by the neural network, a feature map based on the low-resolution disparity image and one of the synthetic stereo pair of images. The method also includes manipulating an unknown object perceived from the feature map according to a perception prediction from a prediction head.

A system for implementing a shared mixed reality experience to participants in a same physical room having a plurality of VR headsets, each of which is adapted to be worn by a participant in the room. Each VR headset having a forward-facing color camera stereo pair. The system includes a computer in communication with at least one of the VR headsets. The system includes a memory in communication with the computer. The memory storing an original 3D digital representation of the room. The computer calculates a rendered VR scene for the at least one of the headsets from a point of view of each of the participant's two eyes, based on the VR headset's current position and orientation. A method for implementing a shared mixed reality experience to participants in a same physical room.

The invention discloses an audio rendering method and device, equipment and a storage medium, and relates to the technical field of audio signal processing, and the method comprises the steps: obtaining a target 2D audio, and separating a target stereo from the target 2D audio; performing sound source object type analysis on the target stereo to determine whether a target sound source object corresponding to the target stereo is a point sound source, if the target sound source object is the point sound source, converting the target stereo into a monaural audio signal, generating first rendering metadata based on the monaural audio signal, and generating second rendering metadata based on the first rendering metadata; rendering the monaural audio signal based on the first rendering metadata; and if the sound source is not the point sound source, generating second rendering metadata based on the left channel signal and the right channel signal of the target stereo, and rendering the left channel signal and the right channel signal based on the second rendering metadata. By judging the type of the sound source, the problems of sound field information loss and sound quality degradation are solved.

In various examples, systems and methods for machine learning based hazard detection for autonomous machine applications using stereo disparity are presented. Disparity between a stereo pair of images is used to generate a path disparity model. Using the path disparity model, a machine learning model can recognize when a pixel in the first image corresponds to a pixel in the second image even though the pixel in the two images does not have identical characteristics. Similarities in extracted feature vectors can be computed and represented by a vector similarity metric that is input to a machine learning classifier, along with feature information extracted from the stereo image pair, to differentiate hazard pixels from non-hazard pixels. In some embodiments, a V-space disparity map, where a first axis corresponds to disparity values and the second axis corresponds to pixel rows, may be used to simplify estimation of the path disparity model.

Embodiments are described for detecting optical discrepancies associated with image capture analyzing pixels in multiple images corresponding to common points of reference in a physical environment. In an embodiment, photometric error values are averaged over time to compute the mean error at each pixel. Once the estimate of the mean error has a sufficient number of updates above a specified value, the estimate is thresholded to provide a mask of any optical discrepancies occurring in the stereo pair of images. Applications include detecting optical discrepancies in images captured for use by a visual navigation system in guiding an autonomous vehicle (e.g., an unmanned aerial vehicle).

An active multiview 3D (AM3D) display is considered, in which detectors are used to recognize the observer's position both parallel and perpendicular to the display surface. Using various modes for shifting at least one of the objects of the AM3D display, such as parallax barriers, 2D displays, or the 3D image itself, the proposed method allows the viewing area of ​​the stereo pairs to be extended and prevents the observer's gaze from entering the 3D image breakup area. Furthermore, the proposed approach allows the parameters of a 3D image to be changed depending on the distance between the observer and the AM3D display, in order to make the perception of 3D objects on the AM3D display as realistic as possible.

In various examples, systems and methods for machine learning based hazard detection for autonomous machine applications using stereo disparity are presented. Disparity between a stereo pair of images is used to generate a path disparity model. Using the path disparity model, a machine learning model can recognize when a pixel in the first image corresponds to a pixel in the second image even though the pixel in the two images does not have identical characteristics. Similarities in extracted feature vectors can be computed and represented by a vector similarity metric that is input to a machine learning classifier, along with feature information extracted from the stereo image pair, to differentiate hazard pixels from non-hazard pixels. In some embodiments, a V-space disparity map, where a first axis corresponds to disparity values and the second axis corresponds to pixel rows, may be used to simplify estimation of the path disparity model.

A method includes obtaining images of a scene captured using a stereo pair of imaging sensors of an XR device and depth data associated with the images, where the scene includes multiple objects. The method also includes obtaining volume-based 3D models of the objects. The method further includes, for one or more first objects, performing depth-based reprojection of the one or more 3D models of the one or more first objects to left and right virtual views based on one or more depths of the one or more first objects. The method also includes, for one or more second objects, performing constant-depth reprojection of the one or more 3D models of the one or more second objects to the left and right virtual views based on a specified depth. In addition, the method includes rendering the left and right virtual views for presentation by the XR device.

In various examples, system and methods for stereo disparity based hazard detection for autonomous machine applications are presented. Example embodiments may assist an ego-machine in detecting hazards within its path of travel. The systems and methods may use disparity between a stereo pair of images to generate a baseline path disparity model and further identify hazards from detected disparities that deviate from that path disparity model. A disparity map for the image pair is constructed in which each pixel represents a disparity for a corresponding element of the image captured. Blockwise division may be optionally used to subdivide the disparity map into a plurality of smaller disparity maps, each corresponding to a block of pixels of the disparity map. A V-space disparity map, where a first axis corresponds to disparity values and the second axis corresponds to pixel rows, may be used to simplify estimation of the path disparity model.

The information extraction and three-dimensional reconstruction method based on super-generalized stereo pair relates to the field of trajectory planning. The present application is to solve the problem that the existing stereo pair modeling method cannot meet the standard stereo pair limitation condition, and it is difficult to reconstruct the overall structure of the building with a small number of local points, resulting in the difficulty of extracting the missing information in the perspective blind area of the orthographic image. The present application comprises: obtaining a building oblique remote sensing image, and performing perspective instance segmentation on the building oblique remote sensing image to obtain a perspective instance segmentation result; performing information repair on the perspective instance segmentation result to obtain a single building image; matching the same repaired single building image from different building oblique remote sensing images; based on the matching result, using the image of each single building to realize super-generalized stereo pair three-dimensional reconstruction; the super-generalized stereo pair is any oblique remote sensing image covering a certain area. The present application is used for three-dimensional reconstruction of buildings.

Example techniques relate to changing a playback device mode based on a device base. In an example implementation, a first playback device operates in a first mode where it is connected to a first network and plays back audio content while in the first mode. The playback device detects connection of the first playback device to a device base and while the first playback device is on the device base, detects connection to a second network. The first playback device switches from operating in the first mode to operating in a second mode. The first playback device forms a stereo pair configuration with a second playback device over the second network in the second mode. While in the second mode, the first playback device plays back a first channel of audio content in synchrony with the second playback device playing back a second channel of the audio content.

A method for training a neural network to perform 3D object manipulation is described. The method includes extracting features from each image of a synthetic stereo pair of images. The method also includes generating a low-resolution disparity image based on the features extracted from each image of the synthetic stereo pair of images. The method further includes generating, by the neural network, a feature map based on the low-resolution disparity image and one of the synthetic stereo pair of images. The method also includes manipulating an unknown object perceived from the feature map according to a perception prediction from a prediction head.