76 results about "Stereoscopic depth" patented technology

The invention discloses an image rendering method and device based on Gaussian splashing, equipment, a storage medium and a program product, and relates to the technical field of computer vision, and the method comprises the steps: obtaining a training image, selecting a reference visual angle of the training image, obtaining the multi-view stereoscopic depth of the training image under the reference visual angle through a multi-view stereoscopic model, and obtaining a multi-view stereoscopic depth of the training image under the reference visual angle; generating a multi-view three-dimensional point cloud through filtering fusion, inputting the multi-view three-dimensional point cloud into a three-dimensional Gaussian splash model, and initializing the model; performing forward mapping and loss calculation on the training image and the multi-view stereo depth to obtain mixed loss of the training image; the loss is returned to the three-dimensional Gaussian splash model, and the model is trained and updated; and giving a target visual angle and camera parameters corresponding to the target visual angle, performing image rendering under the target visual angle through the trained three-dimensional Gaussian splash model, and generating a target image under the target visual angle. According to the invention, through forward mapping, the problem of overfitting under a sparse view angle can be relieved.

Aspects relate to stereo depth estimation utilizing asymmetric down-sampling in different directions. A device may include one or more memories configured to store a plurality of images and a plurality of cameras. The plurality of cameras may be configured to capture a left and right image, in which, each of the images includes one or more patches, each patch including plurality of pixels. The device may include one or more processors coupled to one or more memories, in which, the one or more processors are configured to: down-sample in a first direction on a first set of pixels in a first patch of a first image to generate a first down-sample; and down-sample in a second direction on a second set of pixels in a second patch of a second image to generate a second down-sample, the second down-sample including a greater number of pixels.

The invention discloses a three-dimensional scene reconstruction method driven by monocular implicit depth features. The method comprises the following steps: extracting multi-view monocular depth features through a frozen monocular depth encoder; realizing cross-view feature alignment by using a geometry-vision similarity fusion module; generating a multi-scale monocular implicit depth feature through a multi-scale feature adaptive and processing module; a context modeling module and a feature pyramid network are adopted to extract multi-view context features; carrying out one-stage fusion with a monocular implicit depth feature to obtain a fusion feature; generating a depth map through the multi-view stereo depth estimation network; under the constraint of a depth map, generating spatial point features through multi-source feature projection sampling; performing two-stage fusion with the fusion features to generate point-level implicit features; optimizing the features through a frequency-space local-global gating mechanism; respectively inputting the optimization features into a Gaussian sputtering explicit rendering branch and a volume implicit rendering branch; and carrying out weighted fusion on the rendering result to generate a new view image.

An optical subsystem for use in a display system or an imaging system comprises a plurality of reflective surfaces collectively arranged to provide variable control of device-internal path lengths of light coming to an imaging sensor or traveling a path to an eye of a viewer. The optical subsystem can be used to provide multiple images concurrently at different apparent depths as perceived by the user.

A depth mapping apparatus includes a stereoscopic depth mapping module, including a first and second cameras configured to capture pairs of respective first and second images of a scene, and a range-sensing module, configured to measure respective ranges from the apparatus to multiple points in the scene. A controller processes a first pair of the first and second images of the scene to compute a first depth map of the scene and associates at least some of the points at which the range-sensing module measured respective ranges with corresponding pixels in the first depth map. The controller computes a correction function for the stereoscopic depth mapping module by comparing the respective ranges measured by the range-sensing module to respective depth coordinates of the corresponding pixels in the first depth map and applies the correction function in computing subsequent depth maps based on subsequent pairs of the first and second images.

The invention relates to a three-dimensional model reconstruction method and device based on a Gaussian sputtering model. The method comprises the following steps: acquiring a scene image of a sparse view angle of a target scene, and determining initialization parameter information of a Gaussian ellipsoid according to the scene image through a motion recovery structure technology and a Gaussian sputtering model; determining a priori depth, a color loss function and a gradient smoothing loss function through a monocular depth model according to a scene image; determining a reference depth according to the scene image through a multi-view three-dimensional depth estimation algorithm; determining a depth consistency loss function according to the prior depth and the reference depth; and updating the initialization parameter information through a Gaussian sputtering model based on a depth consistency loss function, a color loss function and a gradient smooth loss function, determining updated parameter information, and generating a three-dimensional model of the target scene according to the updated parameter information. The reliability and robustness of the three-dimensional model can be improved through depth prior constraint information and a dynamic gradient control method.

An example apparatus includes: a camera to record an image; memory to store instructions; and a processor in circuit with the memory, the processor to execute the instructions to: determine a depth based on: (a) the image and (b) a calibration parameter of the camera; and adjust the calibration parameter based on a temperature of the camera and the depth.

The application discloses a kind of super visual field 3D stereoscopic double camera technology systems (referred to as "super 3D"), belong to 3D image double camera technical field.Aiming at the problem of insufficient stereoscopic depth caused by fixed baseline parameter (6-6.5cm) in traditional 3D double camera, the parallax distortion and dizziness caused by 2D to 3D technology, this scheme proposes a kind of technical system based on double camera unit cooperative displacement and post parameter control.Through left camera unit horizontal transverse displacement (12-30cm) and angle rotation (1°-3°), combined with forced focal length ≥50mm optical parameter control module, expand double camera visual field 20%-50%, and use post reverse displacement compensation algorithm (close range 1-3%, medium 3-5%, long distance 5-10%), the main body of picture is aligned to natural focusing position of human eye, realize stereoscopic level enhancement 2-3 times, no distortion high immersive imaging effect.This scheme technical parameter is independent, and cannot be converted with traditional 3D video, ensure content quality from source, applicable to film and television, VR, metaverse and other fields.

The invention provides an end-to-end multi-task three-dimensional depth and semantic fusion visual perception system and an end-to-end multi-task three-dimensional depth and semantic fusion visual perception method. Comprising an image acquisition module used for outputting image data carrying calibration parameter identification; the fusion preprocessing module is used for processing the synchronous image pair to obtain a registration image pair; the multi-task network module comprises a shared feature extraction layer, a depth estimation branch and a semantic segmentation branch; the multi-task coupling unit is used for applying consistency constraint to the shared features and the branch features and transmitting geometric priori and category priori among branches; and the output interface module is used for packaging the depth map and the semantic mask according to a preset data structure, and outputting the depth map and the semantic mask to a real-time control interface of an external execution control system through an industrial real-time communication interface for calling. According to the invention, end-to-end low delay and deterministic output, time synchronization and calibration consistency maintenance, compact integration and simplified wiring can be realized, and a high-precision depth map and a semantic mask can be stably provided for a control system.

The invention provides a multi-view three-dimensional depth estimation method and device, equipment and a readable storage medium. The method comprises the following steps: acquiring a plurality of views of a target scene; performing multi-scale feature extraction on the views to obtain image features and edge features of each view; according to the image features and the edge features, depth estimation and edge depth optimization processing are carried out on the views through geometric perception, and a depth map of each view is generated; and fusing the depth maps of the plurality of views to obtain a three-dimensional point cloud map of the target scene. According to the embodiment of the invention, the depth estimation is more accurate, and the precision of the weak texture region is higher.

This disclosure provides systems, methods, and devices that facilitate image-based three-dimensional (3D) scanning and reconstruction of objects. In one aspect, a method is provided that includes capturing, by a computing device, a plurality of images of an object from respective viewpoints. The method further includes detecting a region containing an object in each image, extracting visual features within the region, and generating a segmentation mask using the features. The first machine learning model determines the volume three-dimensional representation of the object according to the image and the mask; a second representation, such as a grid or point cloud, is then derived. Other processes include attitude estimation of known objects and stereo depth prediction of transparent and opaque materials. Other aspects are also provided.

The invention provides a three-dimensional depth estimation method based on pulse streams, and belongs to the technical field of three-dimensional computer vision and neuromorphic imaging. The method comprises the following steps: (1) constructing a training and testing data set of a synthetic pulse stream; (2) constructing a pulse stream data set of a real scene; (3) constructing a three-dimensional depth estimation network model based on the pulse flow; (4) training a three-dimensional depth estimation network model based on the pulse flow; and (5) performing depth prediction by adopting the trained pulse stream-based stereo depth estimation network model. According to the invention, depth estimation can be directly carried out from the original pulse stream data, and a high-definition depth image is obtained without sequence framing or other post-processing; compared with an existing method, the method has better interpretability and higher prediction precision under a pulse flow data test set through iterative updating of the multi-layer and multi-scale cyclic pulse neural network and an algorithm of the pulse neural network.

Aspects of the subject technology relate to systems and methods for virtual stereo ranging in a vehicle. A visible-light image and a thermographic image of a surrounding environment of a vehicle are respectively received from a visible-light image sensor and a thermographic image sensor. A visible-light depth map and a thermographic depth map of the surrounding environment are respectively generated based on the visible-light image and the thermographic image using a monocular ranging method. A first boundary surrounding a first object and a second boundary surrounding a second object in the surrounding environment are respectively generated in the visible-light image and the thermographic image. Based on the first and second boundaries, a stereo depth map of the surrounding environment is generated using a stereo ranging method. Driving of the vehicle is controlled based on the visible-light depth map, the thermographic depth map, and the stereo depth map.

The invention discloses a super-vision-field 3D stereoscopic double-shooting technology system (hyper3D for short), and belongs to the technical field of 3D image double-shooting. Aiming at the problems of insufficient three-dimensional depth caused by a fixed baseline parameter (6-6.5 cm) and parallax distortion brightness loss and dizziness caused by a 2D-to-3D technology in the traditional 3D double camera, the invention provides a technical system based on cooperative displacement of double camera units and later parameter control. Through horizontal lateral displacement (12-30 cm) and angle rotation (1-3 degrees) of a left shooting unit, an optical parameter control module with the forced focal length larger than or equal to 50 mm is combined, the double-shooting vision field is expanded by 20%-50%, a later-stage reverse displacement compensation algorithm (close shot 1-3%, middle shot 3-5% and long shot 5-10%) is utilized, a picture body is aligned to the natural focusing position of human eyes, and the focusing effect is achieved. The high-immersion imaging effect that the three-dimensional layering sense is enhanced by 2-3 times and distortion is avoided is achieved. Technical parameters of the scheme are independent, mutual conversion with a traditional 3D video cannot be achieved, content quality is ensured from the source, and the method is suitable for the fields of movies and televisions, VR, element universe and the like.

The invention discloses an image depth estimation method, system and equipment based on knowledge mutual distillation and a medium, and relates to the technical field of computer vision, the method comprises the following steps: collecting a single-frame image, a multi-frame image and a three-dimensional image, and respectively obtaining corresponding depth maps; the pixels of the single-frame image are converted into a three-dimensional space, the scale between the inherent geometric plane and the space points corresponding to the pixels is calculated, and a depth map with the recovered scale is obtained according to the scale and the depth maps of the single-frame image and the multi-frame image; aligning a single-frame depth map, a multi-frame depth map and a depth map estimated by the stereoscopic model; reconstructing pixels by using depth values aligned with corresponding scales of a single frame, multiple frames and the three-dimensional depth estimation model, and constructing a knowledge mutual distillation model according to the original pixels and the reconstructed pixels; training a depth estimation model based on a knowledge mutual distillation model, and performing depth estimation on the to-be-detected image according to the trained model; according to the method, the advantages of a single-frame model, a multi-frame model and a three-dimensional depth estimation model are fused, and the independence of a single model is guaranteed.

Stereoscopic and multiscopic imaging methods and systems are disclosed. An example method can include capturing, with a first image capture device, first image data from a scene within a first field of view, and capturing, with a second image capture device, second image data from the scene within a second field of view overlapping with the first field of view over a stereoscopic overlap region, wherein either or both of the first and second image capture devices include a transmissive diffraction mask (TDM) configured to encode TDM depth information in the first and / or second image data. The method can also include determining stereoscopic depth information within the stereoscopic overlap region based on the first and second image data, determining the TDM depth information encoded in first and / or second image data, and generating combined depth information based on the stereoscopic depth information and the first and / or second TDM depth information.

Embodiments of the invention provide an image processing method and apparatus, and an electronic device. The method comprises the steps of obtaining images acquired by an image acquisition device when a lens is located at different optical axis positions in different illumination modes; generating a super depth-of-field depth map based on the obtained image; generating a photometric stereo normal vector diagram based on the obtained image, and generating a photometric stereo depth map based on the photometric stereo normal vector diagram; and correcting depth information in the same depth-of-field region in the super depth-of-field depth map according to a photometric stereo normal vector in the photometric stereo normal vector map, and correcting depth information in a low texture region in the super depth-of-field depth map according to a photometric stereo depth value in the photometric stereo depth map. The detail depth information in the same depth-of-field region can be corrected by the photometric stereo normal vector, and the missing depth information in the low-texture region can be restored by the photometric stereo depth value, so that the super depth-of-field depth map which can well reflect the depth details of the collection view field can be obtained.

The invention provides an online measurement method and system for the key size of a lock based on lightweight edge calculation. The method comprises the following steps: building a detection system of a cloud-edge collaborative architecture; after a lock to be detected is conveyed to a detection station through a conveying belt, a uniform illumination scene is constructed through a shadowless panel light source in an optical detection environment construction system, and RGB color images and infrared depth images of the lock are synchronously collected and detected through an Intel RealSense D405 stereoscopic depth camera; and the qualification of the lock is judged according to a preset dimensional tolerance standard, and unqualified product sorting is completed by controlling the mechanical arm. According to the technical scheme, the key size of the lock on an assembly line can be efficiently measured in real time.

The application relates to a three-dimensional model reconstruction method and device based on a Gaussian sputtering model. The method comprises the following steps: acquiring sparse-view scene images of a target scene, and determining initialization parameter information of a Gaussian ellipsoid according to the scene images by using a motion recovery structure technology and a Gaussian sputtering model; determining prior depth, a color loss function and a gradient smoothing loss function according to the scene images by using a monocular depth model; determining reference depth according to the scene images by using a multi-view stereo depth estimation algorithm; determining a depth consistency loss function according to the prior depth and the reference depth; updating the initialization parameter information based on the depth consistency loss function, the color loss function and the gradient smoothing loss function by using the Gaussian sputtering model, determining updated parameter information, and generating a three-dimensional model of the target scene according to the updated parameter information. The reliability and robustness of the three-dimensional model can be improved by using a depth prior constraint information and a dynamic gradient control method.

The invention provides a multi-view three-dimensional depth reasoning method for unmanned aerial vehicle aerial images, which comprises the steps of data preprocessing and input, frequency perception feature extraction, cost body construction, double regularization and depth map generation and loss calculation, and is used for a multi-view depth estimation task in an unmanned aerial vehicle aerial scene. A traditional method mainly depends on commercial software such as SURE, while a current deep learning method is insufficient in generalization ability and often generates more errors in a boundary region and a weak texture region when processing deep reasoning and reconstruction of multi-view aerial images of an unmanned aerial vehicle. According to the method, spatial features are extracted by adopting frequency adaptive feature extraction and BiFormer global feature modeling, and a depth map is generated and optimized by adopting a double regularization module, so that the problem of reconstruction errors caused by depth inference errors of a boundary region and a weak texture region is solved.

Aspects of the subject technology relate to systems and methods for virtual stereo ranging in a vehicle. A visible-light image and a thermographic image of a surrounding environment of a vehicle are respectively received from a visible-light image sensor and a thermographic image sensor. A visible-light depth map and a thermographic depth map of the surrounding environment are respectively generated based on the visible-light image and the thermographic image using a monocular ranging method. A first boundary surrounding a first object and a second boundary surrounding a second object in the surrounding environment are respectively generated in the visible-light image and the thermographic image. Based on the first and second boundaries, a stereo depth map of the surrounding environment is generated using a stereo ranging method. Driving of the vehicle is controlled based on the visible-light depth map, the thermographic depth map, and the stereo depth map.

The present invention relates a stereo depth camera and in particular to a stereo depth camera based on multiple detectors configured as event sensors and at least one illumination source oriented by 1-axis or 2-axis high-speed scanning for operation in optically clear to contaminated environments. A stereo depth camera (100) according to the present invention comprises an illumination unit (10) including a first illumination source (12-1) for emitting a first light beam, wherein the illumination unit (10) is configured for actively steering the first light beam to a first measurement position (P) in a surrounding of the stereo depth camera (100) within the field of view of the illumination unit (10); an imaging unit (20) including at least two detectors (22-1, 22-2) for detecting the first light beam reflected by a first object point at the first measurement position (P), wherein the detectors (22-1, 22-2) have different alignments and are configured to image a common field of view to enable stereoscopic imaging of the first object point; wherein the detectors (22-1, 22-2) include multiple pixels (S1, S2) that are configured as event sensors sharing a predefined pixel threshold trigger setting and which are adapted to collect multiple depth points from the first object point at the first measurement position (P) during a predetermined time interval Δt, wherein statistical analysis, filters, deep learning or machine learning algorithms, and / or a trained AI-model are applied to the collected depth points for determining and outputting the most probable distance of the first object point as pixel depth value for the first measurement position (P).

Deformable sensors having a stereo depth sensor and an infrared sensor and methods of their use are disclosed. In one embodiment, a deformable sensor includes an enclosure having a housing and a deformable membrane, wherein the deformable membrane is transparent to a wavelength band in the infrared spectrum, a stereo depth sensor that is disposed within the enclosure and is configured to view an underside of the deformable membrane, and output a deformation region of the deformable membrane as a result of contact with an object, wherein the deformation region includes depth information, and an infrared sensor that is disposed within the enclosure and is configured to view the object through the deformable membrane when the object contacts the deformable membrane and output a contact patch region corresponding with a region of the deformable membrane that contacts the object.

Aspects relate to stereo depth estimation utilizing asymmetric down-sampling in different directions. A device may include one or more memories configured to store a plurality of images and a plurality of cameras. The plurality of cameras may be configured to capture a left and right image, in which, each of the images includes one or more patches, each patch including plurality of pixels. The device may include one or more processors coupled to one or more memories, in which, the one or more processors are configured to: down-sample in a first direction on a first set of pixels in a first patch of a first image to generate a first down-sample; and down-sample in a second direction on a second set of pixels in a second patch of a second image to generate a second down-sample, the second down-sample including a greater number of pixels.

Deformable sensors having a stereo depth sensor and an infrared sensor and methods of their use are disclosed. In one embodiment, a deformable sensor includes an enclosure having a housing and a deformable membrane, wherein the deformable membrane is transparent to a wavelength band in the infrared spectrum, a stereo depth sensor that is disposed within the enclosure and is configured to view an underside of the deformable membrane, and output a deformation region of the deformable membrane as a result of contact with an object, wherein the deformation region includes depth information, and an infrared sensor that is disposed within the enclosure and is configured to view the object through the deformable membrane when the object contacts the deformable membrane and output a contact patch region corresponding with a region of the deformable membrane that contacts the object.

The disclosed embodiments relate to an optical system for an augmented reality (AR) head-up display (HUD) device. The implementation of the disclosed optical system in an AR HUD device can improve visual ergonomics by providing an enhanced stereoscopic depth of field (SDoF). The SDoF is created by forming a suitable shape and spatial direction of a virtual image surface (VIS), with the convex surface directed towards the user or observer. When the optical system of the AR HUD device is implemented in a vehicle, the driving comfort and safety can be improved through improved visual ergonomics.

Deformable sensors having a stereo depth sensor and an infrared sensor and methods of their use are disclosed. In one embodiment, a deformable sensor includes an enclosure having a housing and a deformable membrane, wherein the deformable membrane is transparent to a wavelength band in the infrared spectrum, a stereo depth sensor that is disposed within the enclosure and is configured to view an underside of the deformable membrane, and output a deformation region of the deformable membrane as a result of contact with an object, wherein the deformation region includes depth information, and an infrared sensor that is disposed within the enclosure and is configured to view the object through the deformable membrane when the object contacts the deformable membrane and output a contact patch region corresponding with a region of the deformable membrane that contacts the object.

The embodiment of the invention provides a power grid target body identification method and device. The method comprises the following steps: acquiring a plurality of images acquired by a plurality of image acquisition units at different shooting angles; wherein the image acquisition units are carried at different positions of the aircraft; performing three-dimensional recovery processing on the plurality of images to generate a three-dimensional depth map; identifying the stereo depth map by using a preset target identification model to obtain a target identification result; when the target identification result is unidentified, adjusting the shooting angle of each image acquisition unit by controlling the position of the aircraft until the target identification result obtained by identifying a plurality of images acquired based on the adjusted shooting angle comprises a target body; the accuracy of identifying the target body can be improved.

The present application discloses an image rendering method, apparatus, device, storage medium and program product based on Gaussian splattering, which relates to the field of computer vision technology, including: obtaining a training image, selecting its reference perspective, obtaining the multi-view stereo depth of the training image under the reference perspective through a multi-view stereo model, generating a multi-view stereo point cloud through filtering and fusion, inputting it into a three-dimensional Gaussian splattering model, and initializing the model; performing forward mapping and loss calculation on the training image and the multi-view stereo depth to obtain a mixed loss of the training image; transmitting the loss back to the three-dimensional Gaussian splattering model to train and update the model; given a target perspective and camera parameters corresponding to the target perspective, performing image rendering under the target perspective through the trained three-dimensional Gaussian splattering model to generate a target image under the target perspective. The present application can alleviate the problem of overfitting under sparse perspectives through forward mapping.