247 results about "3d surfaces" patented technology

In various examples, to support training a deep neural network (DNN) to predict a dense representation of a 3D surface structure of interest, a training dataset is generated using a simulated environment. For example, a simulation may be run to simulate a virtual world or environment, render frames of virtual sensor data (e.g., images), and generate corresponding depth maps and segmentation masks (identifying a component of the simulated environment such as a road). To generate input training data, 3D structure estimation may be performed on a rendered frame to generate a representation of a 3D surface structure of the road. To generate corresponding ground truth training data, a corresponding depth map and segmentation mask may be used to generate a dense representation of the 3D surface structure.

In various examples, systems and methods are disclosed relating to geometry estimation and dynamic object rendering for vehicle environment visualization. In embodiments, the environment surrounding an ego-machine may be visualized by extracting one or more depth maps from image data, converting the depth map(s) into a 3D surface topology of the surrounding environment, and / or texturizing the detected 3D surface topology with image data. Dynamic objects such as moving vehicles or pedestrians may be detected and masked from a first pass of texturization. Rigid dynamic objects may be visualized by warping corresponding depth values using corresponding trajectories, inserting or fusing the resulting warped 3D representation of each such object into the (e.g., texturized) 3D surface topology, and texturizing the warped 3D representation of each object using corresponding image data. Non-rigid dynamic objects may be represented as flat 2D surfaces and texturized with corresponding image data.

Disclosed is a computer-implemented method that includes receiving, by one or more processors, a three-dimensional (3D) representation of a user's mouth where the 3D representation is a mesh representation comprising a plurality of surfaces; mapping, by the one or more processors, the 3D representation of the user's mouth into a two-dimensional (2D) space; encoding, one or more 3D surface characteristics of the plurality of surfaces using one or more channels of a 2D representation; applying, by the one or more processors, a machine learning model to the representation of the user's mouth in 2D space, the machine learning model trained to enhance the representation of the user's mouth in 2D space; and mapping, by the one or more processors, the enhanced representation of the user's mouth in 2D space to an enhanced representation of the user's mouth in 3D space using the one or more surface characteristics.

The invention discloses depth-based vehicle environment visualization using generative AI. In various examples, systems and methods related to geometry estimation and dynamic object rendering for vehicle environment visualization are disclosed. The environment around the ego-vehicle may be visualized by extracting one or more depth maps from the image data, converting the depth maps to a 3D surface topology of the surrounding environment, and / or texturing the detected 3D surface topology with the image data. Dynamic objects may be detected and masked from a first pass of texturing. Rigid body dynamic objects may be visualized by warping a respective depth map using a respective trajectory, inserting or fusing the resulting warped 3D representation of each such object into a (textured) 3D surface topology, and texturing the warped 3D representation of each object using a respective image data. The non-rigid dynamic object may be represented as a flat 2D surface and textured with corresponding image data.

In various examples, systems and methods are disclosed relating to geometry estimation and dynamic object rendering for vehicle environment visualization. In embodiments, the environment surrounding an ego-machine may be visualized by extracting one or more depth maps from image data, converting the depth map(s) into a 3D surface topology of the surrounding environment, and / or texturizing the detected 3D surface topology with image data. Dynamic objects such as moving vehicles or pedestrians may be detected and masked from a first pass of texturization. Rigid dynamic objects may be visualized by warping corresponding depth values using corresponding trajectories, inserting or fusing the resulting warped 3D representation of each such object into the (e.g., texturized) 3D surface topology, and texturizing the warped 3D representation of each object using corresponding image data. Non-rigid dynamic objects may be represented as flat 2D surfaces and texturized with corresponding image data.

Systems and methods for performing surface matching using three-dimensional (3D) and color information. An example method includes obtaining, by a 3D camera, a first 3D image of a field of view. The 3D image includes (i) a plurality of voxels, or 3D points, of the field of view and (ii) color information of the field of view. A processor normalizes the color information into a common color space, and determines scene histograms from the normalized color information. Each of the scene histograms is determined for a voxel of the plurality of voxels. The method further includes determining a color score from at least the scene histograms and model histograms, the model histograms being indicative of color information of a model image of a 3D object, and determining the presence of a surface of an object in the first 3D image from the color score.

A method of generating a virtual 3D model of a dental arch is provided. The method includes receiving an intraoral scan of a dental arch, determining a first depth of a first intraoral 3D surface in the first intraoral scan, and determining a second depth of a second intraoral 3D surface in the first intraoral scan, where a fixed distance exists between the first intraoral 3D surface in the first intraoral scan and the second intraoral 3D surface. The method further includes stitching together the intraoral scans and generating a virtual 3D model of the dental arch from the intraoral scans, where a fixed distance between the first intraoral 3D surface and the second intraoral 3D surface is included in the virtual 3D model.

A method for generating a virtual model of an object in a patient's anatomy. The method employs a surface scan of an anatomical region of the patient's oral cavity and a volumetric density scan of the anatomical region. The method includes receiving a first dataset of labeled surface scan segments, receiving a second dataset of labeled volumetric density scan segments, cross-mounting in a coordinate system, labeled surface scan segments from the first dataset to labeled volumetric density scan segments from the second dataset, receiving identification of the targeted extraction object, identifying the 3D volumetric density model and the 3D surface model associated with the volumetric density scan segment label, and generating a third dataset comprising and 3D model of a socket, a 3D model of portions of a tooth crown, a 3D model of portions of a target extraction tooth, and / or a 3D model of the extraction contour.

A surgical guidance system has two cameras to provide stereo image stream of a surgical field; and a stereo viewer. The system has a 3D surface extraction module that generates a first 3D model of the surgical field from the stereo image streams; a registration module for co-registering annotating data with the first 3D model; and a stereo image enhancer for graphically overlaying at least part of the annotating data onto the stereo image stream to form an enhanced stereo image stream for display, where the enhanced stereo stream enhances a surgeon's perception of the surgical field. The registration module has an alignment refiner to adjust registration of the annotating data with the 3D model based upon matching of features within the 3D model and features within the annotating data; and in an embodiment, a deformation modeler to deform the annotating data based upon a determined tissue deformation.

A Kaleidoscopic Geometric Vision Platform (KGVP) that transforms three-dimensional (3D) machine vision through an optical system that creates multiple virtual laser projectors from a single moving component. A spindle mirror mechanism (SMM) rotates a reflective surface to sweep collimated laser beams in circular patterns. These beams strike a series of kaleidoscopic mirror facets (KMFs) arranged concentrically around the SMM in a concave configuration. Each KMF redirects the rotating beam, creating a virtual projector with a distinct origin point and sweep direction. As the SMM completes one rotation, it generates N distinct laser trajectories (where N equals the number of KMFs), each sweeping from a different virtual origin point. Event-based cameras / sensors positioned strategically around the KMFs detect laser light reflected from object surfaces with microsecond precision. The KGVP triangulates 3D surface coordinates by determining correspondence between detected light and specific virtual projectors based on precise timing information.

The invention discloses an automobile part defect detection method based on multispectrum, and relates to the technical field of spectrum detection. According to the method, an optimal wave band is screened from a data source through spectral response discrimination, and the spectral texture difference between scratches and normal textures is enhanced; normal texture and suspected scratch areas are accurately divided through density clustering, a target area is focused, irrelevant data interference is reduced, and it is ensured that double-feature extraction is higher in pertinence. Further synchronously extracting texture gray level change features and 3D surface height depression features of the suspected area, and capturing dual attributes of two-dimensional texture anomaly and three-dimensional physical form depression of the scratch; and finally, through deviation calculation and relevance verification with a normal region reference feature, the abnormal degree of the double features is quantified, two types of abnormalities are ensured to correspond to the same physical region through spatial relevance, pseudo defects are eliminated, and the problem that scratches are difficult to identify based on texture gray level change and surface height depression in the prior art is solved.

The present disclosure relates to an intraoral scanning system (102) that is configured to generate an overlay of correlated 2D inner geometry features on a 3D surface model for dental objects. The system is configured to receive visible light information (128) and IR information (130) from one or more sensors and generate a 3D surface model (118) of a dental object (!46) based on the visible light information and generate a plurality of 2D internal feature images (302, 306, 310, 314) based on the visible light information and the IR information, process the plurality of 2D internal feature images to correlate the 2D inner geometry features of the dental object with at least one reference frame (712) of the dental object in the 3D surface model; and output an overlay (120) of the correlated 2D inner geometry features of the dental object on the 3D surface model. The plurality of 2D internal feature images indicates 2D inner geometry features for the dental object.

Aspects of this technical solution can allocate one or more portions of a mesh in a three-dimensional (3D) space to one or more processing units associated with the one or more circuits, the mesh associated with a surface of an object in the 3D space, transform, by the one or more of the processing units, one or more of the portions of the mesh from the 3D space into corresponding second meshes in a two-dimensional (2D) space, segment, by the one or more of the processing units, according to a determination that a distortion of a portion of the mesh among the one or more of the portions of the mesh is below a threshold of parameterization, the portion of the mesh into two further portions of the mesh, where the further portions are among the one or more portions of the mesh, and generate, according to a determination that the distortion of the portion of the mesh among the one or more of the portions of the mesh is at or above the threshold of parameterization, an output mesh including the one or more portions of the mesh.

The invention belongs to the field of visual inspection, and particularly relates to a chip packaging multi-dimensional defect intelligent detection system which comprises a plurality of modules and a subsystem. Wherein the electric-thermal linkage detection module consists of ATE (automatic test equipment) and a thermal infrared imager, and is used for acquiring an electric parameter matrix and a thermal imaging matrix of a chip; the multi-dimensional vision detection module comprises an X-ray tomography unit, a 3D structured light imaging unit and a plane vision unit; the reconstruction module is used for reconstructing an internal structure of the chip, acquiring 3D surface topography and extracting plane defect features; the multi-modal feature fusion module is based on a cross-modal attention mechanism and a multi-task joint learning model to fuse electric-thermal data, X-ray voxel features and 3D morphology features; the hierarchical defect decision-making subsystem locates a defect area through FastR-CNN, classifies defect types through a graph convolutional network, and outputs a final defect judgment result based on Bayesian reasoning network by integrating multi-modal evidence; through multi-modal data linkage, joint detection of cavities, heat dissipation and appearance defects can be realized.

The invention discloses depth-based vehicle environment visualization using generative AI. In various examples, systems and methods related to geometry estimation and dynamic object rendering for vehicle environment visualization are disclosed. The environment around the ego-vehicle may be visualized by extracting one or more depth maps from the image data, converting the depth maps to a 3D surface topology of the surrounding environment, and / or texturing the detected 3D surface topology with the image data. Dynamic objects may be detected and masked from a first pass of texturing. Rigid body dynamic objects may be visualized by warping a respective depth map using a respective trajectory, inserting or fusing the resulting warped 3D representation of each such object into a (textured) 3D surface topology, and texturing the warped 3D representation of each object using a respective image data. The non-rigid dynamic object may be represented as a flat 2D surface and textured with corresponding image data.

The invention discloses an enhanced 3D (three-dimensional) surface reconstruction method based on 3D (three-dimensional) Gaussian Splitting. Global consistent depth priori is obtained through virtual stereo pair rendering; constructing a factor graph and introducing a cross-view geometry / luminosity consistency constraint to form local beam adjustment loss; the prior is used as a learnable parameter to be combined with 3DGS to be optimized, and meanwhile, the Pull loss is assisted to pull low-credibility pixels; and finally, multi-loss function end-to-end training is adopted. The method comprises the following steps of: in Tanksamp; the method has the advantages that F1 is equal to 0.58 and DTU Chamfer is equal to 0.48 mm on a Temples data set, the training time is only 20 min, compared with the prior art, geometric accuracy SOTA and speed magnitude improvement are achieved at the same time, and the method is suitable for VR / AR, robot and industrial measurement scenes.

A Kaleidoscopic Geometric Vision Platform (KGVP) that transforms three-dimensional (3D) machine vision through an optical system that creates multiple virtual laser projectors from a single moving component. A spindle mirror mechanism (SMM) rotates a reflective surface to sweep collimated laser beams in circular patterns. These beams strike a series of kaleidoscopic mirror facets (KMFs) arranged concentrically around the SMM in a concave configuration. Each KMF redirects the rotating beam, creating a virtual projector with a distinct origin point and sweep direction. As the SMM completes one rotation, it generates N distinct laser trajectories (where N equals the number of KMFs), each sweeping from a different virtual origin point. Event-based cameras / sensors positioned strategically around the KMFs detect laser light reflected from object surfaces with microsecond precision. The KGVP triangulates 3D surface coordinates by determining correspondence between detected light and specific virtual projectors based on precise timing information. This approach eliminates the computational complexity typically required for multi-view feature matching, as the KGVP can unambiguously identify which virtual projector created each detected event. With calibration, the KGVP can establish the exact 3D positions of all virtual projector origins and their beam trajectories and build look-up tables that optimize triangulation accuracy. The KGVP enables real-time 3D surface reconstruction with high precision while using far fewer mechanical components than conventional machine vision systems.

In various examples, an environment visualization pipeline may determine whether to generate or otherwise enable a visualization using an environmental modeling pipeline that models an environment as a 3D bowl or using an environmental modeling pipeline that models the environment using some other 3D representation, such as a detected 3D surface topology. The determination may made based on various factors, such as ego-machine state, (e.g., one or more detected features indicative of a designated operational scenario, proximity to a detected object, speed of ego-machine, etc.), estimated image quality of a corresponding environment visualization, and / or other factors. Accordingly, an environment around an ego-machine, such as a vehicle, robot, and / or other type of object, may be visualized in systems such as parking visualization systems, Surround View Systems, and / or others.

A method of operation of a compute system includes: generating a digital model from a input image; segmenting a 3D-polygon from the input image by applying a triangular mesh; assembling a 3D surface, based on the digital model of a zone of interest from the 3D-polygon; calculating an average threshold value of pixels within the zone of interest for identifying a condition over the 3D surface; and assembling a skin condition display from analyzing by a skin artificial intelligence (AI) of the 3D surface for displaying on a device.

A method of and an apparatus for motion tracking of a subject located in a scanner are presented. The method comprises generating a baseline 3D surface representation of a surface region of the subject at a first point of time (T(0)); generating a subsequent 3D surface representation of the surface region of the subject at a subsequent point of time (T(s)); determining a best-fit registration of the subsequent 3D surface representation with at least one constraint relative to the baseline 3D surface representation and determining at least one motion tracking parameter. The method may include selecting at least one virtual feature and associating the at least one virtual feature to the baseline 3D surface representation, wherein the constraint comprises a restriction of at least one parameter of the at least one virtual feature associated to the best-fit subsequent 3D surface representation relative to the at least one parameter of the at least one virtual feature associated to the baseline 3D surface representation.

The present invention belongs to the field of visual inspection, in particular to an intelligent multi-dimensional defect detection system for chip packaging, including multiple modules and a subsystem; among them, the electro-thermal linkage detection module is composed of an ATE automatic test equipment and an infrared thermal imager, which collects the electrical parameter matrix and thermal imaging matrix of the chip; the multi-dimensional visual detection module includes an X-ray tomography imaging unit, a 3D structured light imaging unit and a planar vision unit; it is used to reconstruct the internal structure of the chip, obtain the 3D surface topography and extract the planar defect features; the multi-modal feature fusion module is based on the cross-modal attention mechanism and multi-task joint learning model to fuse the electro-thermal data, X-ray voxel features and 3D topography features; the hierarchical defect decision subsystem locates the defect area through FastR-CNN, classifies the defect types through the graph convolutional network, and outputs the final defect determination result based on the Bayesian inference network by integrating multi-modal evidence; through the multi-modal data linkage, the joint detection of voids, heat dissipation and appearance defects can be realized.

The present disclosure relates to the field of medical image processing and, more in particular, to a method for determining an anatomical fracture reconstruction based on one or more medical images of a broken bone, and a system configured for the same. According to an aspect the method comprises the steps of: receiving medical image data associated with the broken bone, identifying the plurality of bone fragments in the medical image data, generating a plurality of 3D surface models that virtually represent the bone fragments; registering a plurality of landmarks matching the surface models; generating a reconstructed model representing a reconstructed form of the broken bone by selecting at least two landmarks; generating a template model representing an optimal alignment of the surface models; repositioning the surface models to align at the selected landmarks set using the template model as reference; iteratively refining the reconstructed model by selecting different landmarks; changing the position of the surface models to align at the different landmarks; evaluating a quality measure that quantifies how changing the position affects the quality of the reconstructed model until a stopping condition is satisfied; and determining an anatomical reconstruction of the broken bone by comparing the position of the identified bone fragments in the medical image data with the position of the corresponding surface models in the reconstructed model.

The invention provides a path texture driven 3D surface processing track generation method and device, and the method comprises the steps: 1, obtaining the coordinates of three vertexes of a triangular patch and a first-order adjacency list of each vertex based on an STL model; step 2, constructing mapping from a curved surface path to a two-dimensional image path based on the vertex coordinates and the first-order adjacency list obtained in the step 1; 3, processing the path texture image to obtain a point sequence, and then reversely mapping the point sequence to the STL model based on mapping to obtain an initial track point set; and step 4, performing optimization processing on the initial track point set to generate a path point set. According to the method, the path texture image and the STL model serve as input, and the continuous track capable of being used for precision machining tasks such as industrial grinding is generated.

Methods and apparatuses for automating the retopologization of 3D meshes including the automated selection and adjustment of correspondence points are described. The automated selection of correspondence points may be performed to refine locations of correspondence points using a matching score that is computed based on surface normal similarity between surfaces corresponding with a candidate correspondence point on an input scan mesh and a point on a morphable model of 3D surfaces. The matching score may also take into account a distance between a candidate correspondence point on the input scan mesh and a corresponding point on the morphable model of 3D surfaces and similarities in surface features, such as similarities in surface curvature at the candidate correspondence point on the input scan mesh and the corresponding point on the morphable model of 3D surfaces.

This invention provides an automatic calibration method and apparatus for a ship panoramic surround view system based on UAV collaboration and 3D surface mapping. A virtual 3D surface projection model is established in the ship's coordinate system, and the surface normal vectors of sampling points on this model are mapped outwards to generate the UAV's flight trajectory. The UAV, equipped with an RTK positioning module, acts as a dynamic calibration target with precise three-dimensional coordinates as it flies along this trajectory, its spatial distribution naturally matching the geometry of the surface projection model. During flight, images from the UAV are simultaneously acquired by the ship's onboard camera, and pixel coordinates are extracted using a target detection algorithm. Simultaneously, the time synchronization data between the UAV and the ship's onboard RTK module is used to establish the correspondence between pixel coordinates and three-dimensional coordinates in the ship's coordinate system, minimizing reprojection errors and jointly calculating the extrinsic parameters of each camera. This eliminates the dependence on fixed planar references on shore, enabling automated calibration with precise adaptation to the 3D surface projection model in any water area.

An intraoral scanning system (102) includes a handheld intraoral scanner (104), a processor (106), and a continuous volumetric ML model (108) for generating a 3D internal geometry (120) of a tooth (306). The intraoral scanner includes a sensor (214D) for detecting NIR and visible light. The processor receives the visible light and NIR information from the sensor, generates a 3D surface model (118) of the tooth using the visible light information, and determines input parameters of a continuous volume ML model by capturing a 2D NIR image (114) of an interior region of the tooth from the NIR information. Each NIR image contains pixels (722) having a projection object. The input parameters include spatial position and viewing angle, and the projected object contains point coordinates (726, 732). The continuous volume ML model processes the input parameters to determine intensity and density values for pixels in the 3D internal geometry of the 3D surface model.

This invention belongs to the field of contact lens technology, specifically providing a method for designing scleral lenses based on ocular surface and scleral topography. Based on the topography of the anterior surface of the patient's cornea and sclera, the method divides the anterior surface of the cornea and sclera into meridians at equal angles, and designs the surface profile curves of the anterior and posterior surfaces of the scleral lens according to the morphology of the patient's cornea and sclera. Finally, 3D reconstruction is used to obtain the 3D surface profile data of the scleral lens. The optical zone of the scleral lens can be designed in various ways depending on the different shapes of the patient's eyeball; the transition zone can be designed in diverse ways to ensure the thickness of the intraocular fluid at certain locations based on the ocular surface morphology; and the landing zone smoothly conforms to the anterior surface of the sclera. This invention maximizes conformity to the patient's ocular surface morphology, ensures the uniformity of the intraocular fluid, and maintains a high degree of matching with the ocular surface, thereby achieving trial-free scleral lens fitting, improving clinical accuracy and efficiency, and enhancing patient safety and satisfaction.

Disclosed is a printing apparatus for a 3D surface, which performs printing by ejecting a droplet onto a 3D surface and controlling an electric field on an impact path of the droplet, the printing apparatus including: a ejecting environment information provider configured to provide ejecting environment information between a nozzle and an impact point; and a controller configured to predict a result of printing on an actual substrate by accumulating previous printing results according to printing conditions and the ejecting environment information into a database, and perform the printing on the 3D surface while changing the printing conditions provided by the database based on the ejecting environment information provided by the ejecting environment information provider.

The present disclosure relates to an intraoral scanning system configured to determine a three-dimensional surface model and a plurality of focus images. The intraoral scanning system comprises: a projector unit configured to emit light onto a dental object, wherein the emitted light comprises an infrared wavelength and a structured light, the structured light comprising a first visible wavelength; a first image sensor unit comprising: a first image sensor configured to acquire first reflected light from the dental object, wherein the first reflected light comprises the infrared wavelength, and wherein the first image sensor comprises a pixel array, a microlens array arranged in front of the pixel array and configured to transmit the first reflected light to the pixel array, wherein one or more microlenses of the microlens array direct the first reflected light to one or more pixels of the pixel array; a second image sensor unit comprising: a second image sensor configured to acquire second reflected light from the dental object, wherein the second reflected light comprises the structured light; a processor unit configured to: determine three-dimensional (3D) data of the dental object based on the second reflected light, determine a three-dimensional (3D) surface model of the dental object by merging the 3D data, and determine a plurality of focus images at different focus depths into the dental object by selecting a set of pixels from the pixel array associated with each microlens of the microlens array, and wherein the processor unit is configured to align a position of each focus image of the plurality of focus images to a position in the 3D surface model based on the 3D data.

A modular, multi-functional, and reusable three-dimensional (“3D”) mounting surface for use in robotic automation to facilitate changing from one robot type to another (e.g., COTS (commercial off the shelf) robot or component) is described. A change involves removing only the surface plates without the entire mounting base plate for each product, which make the manufacture process less costly and faster to deploy. The modular design of the present invention enables a flexible mounting base plate configured to incorporate different types of robot sizes, robot cabinets, and applications. This invention is not only expandable on the three-dimensional (“3D”) tessellated surface for larger robot applications, the 3D surfaces may be combined together to form even larger robot solutions for factory automation. The present invention is reconfigurable with inserts that fit precisely in the mounting base plate.