966 results about "2d images" patented technology

A method and apparatus for reconstructing a three-dimensional (3D) scene from a two-dimensional (2D) input image of the scene using a fully-differentiable transformer-based encoder-decode. A 2D input image encoded into a set of image features using a pre-trained vision transformer model, wherein the vision transformer model is pre-trained with multi-view RGB image supervision and point cloud supervision. The set of image features is projected onto a 3D triplane representation using a transformer decoder to obtain output triplane tokens. A triplane representation is created from the tokens and queried. 3D point features of color and density for volumetric rendering re predicted using a multi-layer perceptron. The geometry of the generated 3D asset is represented with a surface mesh including vertices and triangular faces. A texture map by is created with a multichannel image in UV space. Multiple views of the 3D scene are simultaneously generated based on the surface mesh.

Methods and processors for rendering a 3D object are disclosed. The method includes acquiring multi-camera image input including first image frames of the 3D object generated by a first camera and second image frames of the 3D object generated by a second camera, acquiring an initial 3D Gaussian Splatting (3DGS) model having a plurality of initial parameters including an initial frame-wise GS parameter and an initial camera-wise GS parameter, generating an adjusted 3DGS model by adjusting, based on the multi-camera image input, at least one of: the initial frame-wise GS parameter, the initial camera-wise GS parameter, generating, by the adjusted 3DGS model, a 3DGS output and rendering a 2D image of the 3D object using the 3DGS output.

The invention discloses a tunnel fracture recognition method based on three-dimensional live-action reconstruction and ordered re-acquisition of a virtual camera. The method specifically comprises the following steps: acquiring a multi-view rock fracture image; acquiring three-dimensional point cloud data of a tunnel wall surface according to the acquired rock mass fracture image, reconstructing a geometrical shape of the tunnel, mapping acquired texture information to a three-dimensional grid vertex through an image processing algorithm, and generating a three-dimensional tunnel model; setting parameters of the virtual camera, orderly collecting to form a virtual image, and enabling the position of the virtual camera to correspond to the real position of the model; and extracting information of the crack of the virtual image, and projecting the coordinates of the two-dimensional image back to the three-dimensional space through the projection matrix to obtain the space coordinates of the crack. A virtual image is generated by combining a texture mapping technology of a three-dimensional model, so that the limitation of a visual angle, illumination and space in actual image acquisition is overcome, and high-precision identification of tunnel cracks is realized.

The invention discloses a shield segment dislocation detection method and system fusing visual and geometric features, and the method comprises the steps: employing a mobile track three-dimensional laser scanning system, and rapidly obtaining the three-dimensional point cloud data of a tunnel segment; projecting the three-dimensional point cloud data of the tunnel segment to a two-dimensional image according to the scanning parameters and the image preset resolution, and mapping the laser point reflection intensity into a pixel gray value; shield segment inter-ring joints and shield segment in-ring joints are extracted respectively, and segment joint positioning is completed; and according to the seam positioning information, extracting local point clouds at the two sides of the seam, respectively carrying out circular model fitting, calculating the height difference of circular models at the two sides at the seam, and obtaining an in-ring slab staggering value. According to the method, the visual information of the point cloud reflection intensity and the geometric information of the spatial position are fused, rapid and accurate shield segment in-ring slab staggering detection is realized, and the efficiency and the accuracy are high.

The invention discloses a laser welding seam online quality detection system and method based on multi-mode visual fusion, and particularly relates to the technical field of laser welding seam quality detection. Comprising a data acquisition and synchronous control unit, a 2D image processing and defect detection unit, a 3D point cloud processing and size measurement unit, a multi-modal information fusion and collaborative analysis unit and a comprehensive quality judgment and output unit. According to the laser welding seam online quality detection system and method based on multi-modal visual fusion, the 2D and 3D multi-modal visual fusion technology is adopted, an improved defect detection algorithm and an accurate size measurement method are combined, welding seam surface defects are accurately recognized through 2D images, key size data are obtained by means of 3D point cloud, and the detection accuracy is improved. And the limitation of'heavy defects and light sizes' or'heavy sizes and light defects' in single-modal detection is avoided, and comprehensive and accurate evaluation of the welding seam quality is realized.

The invention discloses a dynamic digital twinning system based on 3D Gaussian splashing. The dynamic digital twinning system comprises a flexible electric power inspection robot, a 3DGS dynamic enhancement module and a fusion digital twinning background centralized control system. The flexible electric power inspection robot is used for collecting 2D images and environment point cloud data of substation equipment; the 3DGS dynamic enhancement module is used for carrying out 3DGS dynamic enhancement and generating a 3D Gaussian model; the standard digital twinborn centralized control platform is used for storing and calling data and digital twinborn bodies of the traditional static substation digital model; the model fusion port carries out space alignment and attribute interpolation fusion on the 3D Gaussian model and the static digital twin; and the visual terminal is used for displaying the fused enhanced digital twinborn body and the equipment state evaluation result. According to the invention, the dynamic enhancement of the digital twinborn model of the substation equipment can be realized, and the operation state and change condition of the equipment can be reflected more truly and intuitively.

A system comprises an encoder configured to compress attribute information and / or spatial for a point cloud and / or a decoder configured to decompress compressed attribute and / or spatial information for the point cloud. To compress the attribute and / or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. Additionally, an encoder is configured to signal and / or a decoder is configured to receive a supplementary message comprising volumetric tiling information that maps portions of 2D image representations to objects in the point. In some embodiments, characteristics of the object may additionally be signaled using the supplementary message or additional supplementary messages.

In various examples, systems and methods are disclosed relating to generating an output 3D latent representation by encoding, using a text encoder, a text prompt and encoding, using a 2D / 3D encoder, a 2D image of an object or a 3D representation of the object. A 3D output is generated by applying the output 3D latent representation to a decoder. A reconstruction loss and a SDS loss are determined for the 3D output. At least one of the text encoder, the 2D / 3D encoder, and the decoder is updated using the reconstruction loss and the SDS loss.

The invention relates to a semantic simultaneous localization and mapping method and system based on Gaussian splashing. The method comprises the following steps: firstly, collecting a frame of RGB-D image, modeling a scene into a 3D semantic Gaussian field containing a plurality of 3D semantic gausses according to the RGB-D image, and rendering the 3D semantic gausses by using a tile rasterization technology to obtain 2D image plane gausses; rendering results of RGB color, depth and semantic features are extracted from the 2D image plane in a Gaussian mode, and the semantic features are decoded into semantic tags; constructing a mapping and tracking loss function by using an RGB color rendering result, a depth rendering result, a semantic tag and a truth value, and jointly optimizing a camera pose and a semantic Gaussian field based on a tracking stage and a mapping stage; and repeating the steps for each new frame of RGB-D image to complete the construction of the incremental semantic Gaussian map. Compared with the prior art, the method has the advantages of realizing robust camera tracking, real-time high-quality rendering, accurate 3D semantic reconstruction and the like.

Techniques are discussed herein for generating three-dimensional (3D) representations of an environment based on two-dimensional (2D) image data, and using the 3D representations to perform 3D object detection and other 3D analyses of the environment. 2D image data may be received, along with depth estimation data associated with the 2D image data. Using the 2D image data and associated depth data, an image-based object detector may generate 3D representations, including point clouds and / or 3D pixel grids, for the 2D image or particular regions of interest. In some examples, a 3D point cloud may be generated by projecting pixels from the 2D image into 3D space followed by a trained 3D convolutional neural network (CNN) performing object detection. Additionally or alternatively, a top-down view of a 3D pixel grid representation may be used to perform object detection using 2D convolutions.

The invention discloses a 3D generative model training method and device based on a hybrid training framework, equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: pre-training a 3D generation model by utilizing a preset 3D data set; quickly generating initial 3D assets according to the text cue words through a 3D generation model; a strategy based on two-dimensional knowledge distillation is adopted, a text-to-2D image generation model is used as a teacher model to carry out iterative distillation optimization on the initial 3D assets, and the 3D assets with improved visual fidelity are obtained; in the distillation process, dynamically adjusting the teacher model to adapt to the distribution of the initial 3D assets by using a self-adaptive teacher model guide strategy; and taking the optimized high-fidelity 3D assets as enhanced training samples, and training the 3D generation model again to improve the generation quality. According to the method, the 3D generation model is pre-trained, so that the 3D generation model is initially converged; according to the distillation method, strong text-to-2D image generation model knowledge is transferred to initial 3D assets and serves as an enhanced sample to train a 3D generation model again for knowledge internalization. The adaptive teacher model guiding strategy reduces the distribution difference between the teacher model and the student model, and ensures the generation quality of the 3D generation model.

The invention relates to the technical field of computer vision and three-dimensional modeling, and discloses a 3DGS (three-dimensional ground structure) equipment monomer method based on dynamic segmentation of an SAM2 large model, which comprises the following steps of: calling a visual large model to segment a multi-view two-dimensional image to obtain a component contour mask bearing semantic information; the method comprises the following steps of: establishing a mask of a three-dimensional point cloud, establishing space mapping of the mask and the three-dimensional point cloud, executing structure segmentation on the point cloud under double dimensions of fusing geometry and semantics, and further generating a three-dimensional monomer model for each separated equipment component. The segmentation processing of the three-dimensional point cloud is no longer limited to fuzzy geometric inference, but is carried out under clear semantic affiliation, so that the accuracy and robustness of monomer separation in the component dense adjacent region are improved.

A method and system for eliminating 2D features from planar surfaces in 2D images. Three digital images are taken, from a single camera at a fixed position, of a subject such as a pallet of boxes. One image (IA) is taken with ambient lighting, one image (I1) has ambient lighting plus a first added light source, and one image (I2) has ambient lighting plus a second added light source. An output image Q is then computed by Q=(I1−IA) / (I2−IA). Subtracting the ambient image removes ambient diffuse and specular reflections. Division eliminates all variations in the output image caused by color. The only variations that remain are those due to the angle between each surface point's normal direction and the direction from the light to that point. The output image Q, devoid of all colors and 2D features, is well suited for computing a robot grasp of an object in the image.

A computer-implemented method for generating a 2D image of a 3D scene. The method comprises obtaining arrangement data comprising a layout of the 3D scene and at least one conditioning signal. Each conditioning signal has a type among a predetermined set of at least two types. The method comprises applying a machine-learning function to the obtained arrangement data and viewpoint. The function comprises a scene encoder and a generative image model. The scene encoder takes as input the obtained arrangement data and viewpoint and outputting a scene encoding tensor. The generative image model takes as input the scene encoding tensor outputted by the scene encoder and outputting the generated 2D image. Such a generating method forms an improved solution for controllably generating a 2D image of a 3D scene.

Zero-shot open-vocabulary 3D auto-labeling is performed using visual foundation models (VFMs). Multi-view 2D images of an environment and corresponding 3D LiDAR points of the environment are received. 2D semantic knowledge is extracted from the multi-view 2D images in close-set and open-set detection branches. 3D spatial-temporal prompts are generated via clustering and tracking of the 3D LiDAR points. The 3D spatial-temporal prompts and the 2D semantic knowledge are used for mapping the 2D semantic knowledge to a plurality of clusters of the 3D LiDAR points, thereby producing labeled 3D LiDAR points defining a 3D semantic segmentation of the 3D LiDAR points. One or more downstream applications are performed using the labeled 3D LiDAR points.

The invention is notably directed to a computer-implemented method for automatically charging an electric vehicle via an end effector (10) of a robotic arm (40) of an automated vehicle charging robot. The end effector is assumed to be structured so as to be able to connect to a charge port (220) of a vehicle. In addition, the automated vehicle charging robot further includes a camera system (102) having depth sensing capability. The method comprises the following steps. First, a reference position of a reference feature (210) of the vehicle is estimated thanks to the camera system. Next, a pose of the charge port of the vehicle is determined based on the estimated reference position. The robotic arm is subsequently instructed to actuate the end effector, based on the determined pose of the charge port, to connect the end effector to the charge port with a view to charging the vehicle. The reference position is estimated as follows. Both a 2D image and a depth image of a surface portion of the vehicle are obtained. This surface portion includes the reference feature, i.e., the feature of interest. Contour points of the reference feature are then extracted from the 2D image obtained. The 3D coordinates of the extracted contour points are subsequently reconstructed based on the depth image obtained. A geometric object (such a 2D plane) is then matched to the reconstructed 3D coordinates, e.g., by fitting the geometric object to the reconstructed 3D coordinates. Eventually, the reference position of the reference feature is determined based on the matched geometric object. The invention is further directed to related automated vehicle charging robots and computer program products.

A computer-implemented process for creating 3D objects from 2D images includes receiving an input image, conditioning a generative model using image-derived, voxelized three-dimensional features, generating, by a transformer-based rectified-flow generative model parameterized as a base network optionally coupled to one or more low-rank adapter modules activatable at inference, a volumetric latent of the target object, the volumetric latent including a sparse, feature-augmented volumetric lattice obtained by transporting an initial random sample toward a learned manifold via a rectified-flow sampling process, decoding the volumetric latent by mapping the volumetric latent to a feature-bearing sparse volumetric field consistent with the volumetric lattice, decoding the field to a continuous implicit surface function, and extracting a watertight mesh by isosurface extraction, estimating camera-pose parameters by render-and-compare alignment between silhouettes rendered from the mesh and silhouettes of the input image, and, performing style-preserving inverse rendering on the mesh that updates UV-space albedo and material maps.

Provided in the embodiments of the present application are a 3D video generation method and an electronic device. The method is applied to the electronic device. The method comprises: on the basis of captured data, acquiring 3D video material data, wherein the captured data comprises a 2D video and / or a 2D image; on the basis of the 3D video material data, analyzing a 3D video scene type; on the basis of the 3D video scene type, acquiring a new-view generation model corresponding to the 3D video scene type; and using the new-view generation model corresponding to the 3D video scene type to generate 3D video data on the basis of the 3D video material data. On the basis of the method of the embodiments of the present application, at a 3D video data generation stage, a corresponding new-view generation model is called on the basis of a 3D video scene type, and thus a more complete scene synthesis view can be acquired on the basis of a new-view synthesis algorithm that uses deep learning and artificial intelligence, such that when 3D video data is played, a user can immersively view 3D video content from different angles, thereby providing a stronger sense of 3D and richer 3D video content.

A head-worn augmented reality (AR) device system includes cameras, display devices, and processors, along with a memory that stores specific instructions. When these instructions are executed by the processors, they enable the device to perform several operations. First, the device accesses a two-dimensional (2D) camera image taken by its camera. The device then generates a first set of three-dimensional (3D) tracked points using the device's odometry system applied to this 2D image. Optionally, a second set of tracked 3D points is created based on one or more images captured by the camera. These 3D points are projected onto the 2D camera image to create a sparse depth image. Finally, this 2D camera image, along with the newly formed depth image, is fed into a first machine learning model to generate a metric depth estimation.

An embodiment provides a method of producing a three-dimensional (3D) model of an object based on a single, frontal input two-dimensional (2D) image. In one example a method includes obtaining an actual, frontal 2D image of an object and generating a pair of synthetic 2D multiview images of the object based on the actual, frontal 2D image of the object. A 3D model of the object is produced based on at least the pair of synthetic 2D multiview images. The 3D model conserves an asymmetry of the object. An output using the 3D model of the object is produced that conserves the asymmetry.

The invention belongs to the technical field of industrial automation, and particularly relates to a snake-shaped pipe grabbing point positioning method based on 2D vision and 3D vision fusion, and the method comprises the steps: collecting an original image set of a snake-shaped pipe, the original image set comprises original images of the snake-shaped pipe at a plurality of different visual angles, and the original images comprise depth images and texture images; based on each texture image in the original image set and the trained target detection model, determining a planar clip-free area; based on the original image set and the planar clamping piece-free area, determining a complete point cloud of the coiled pipe and a point cloud of the area to be positioned; based on the complete point cloud of the coiled pipe and the point cloud of the to-be-positioned area, determining a target positioning area; and determining a grabbing point of the coiled pipe based on the target positioning area. According to the technical scheme provided by the invention, the grabbing point is positioned in combination with the 2D image and the 3D image of the serpentine pipe, so that the accuracy and efficiency of positioning the grabbing point of the serpentine pipe are improved.

Methods and systems for assessing, determining, or quantifying structural properties of a transport structure are provided, including methods and systems for capturing, using first and second image capture sensors of an inspection system, a plurality of 2-dimensional (2D) images of an inspection area of the inspection system; detecting, using an AI engine, a transport structure in a first image from the plurality of 2D images; extracting, using the AI engine and based on the first image, a second image from the plurality of 2D images; generating, using the AI engine and based on the first image and the second image, a computing model representing the transport structure; analyzing, using the AI engine, the computing model thereby generating analysis data; generating, using a data processing unit, a report that indicates the analysis data; and initiate formatting, using the data processing unit, for display on a graphical interface, the report.

The invention discloses a virtual projection 3D oral and maxillofacial key point detection method based on deep learning, and aims to solve the problem of insufficient 3D image key point detection precision and efficiency in the prior art. The method comprises the following steps: projecting 3D image data into a plurality of 2D projection images by using a virtual projection technology; then key point detection and marking are carried out on the 2D image by using a deep learning network model, and 2D mark point coordinates are obtained; and finally, performing 3D reconstruction by adopting a virtual back projection method, and back-projecting the 2D mark point information to a 3D space to obtain 3D mark point coordinates. According to the method, 2D key point detection and 3D projection and reconstruction technologies are combined, rapid, high-precision and automatic detection of 3D image mark key points is realized, and the method can be widely applied to the fields of oral medical image analysis and the like.

The invention discloses a three-dimensional target detection method based on view cone point cloud clustering and 2D detection, and the method comprises the steps: generating a plurality of target bounding boxes with categories according to a 2D image, converting the target bounding boxes to an image coordinate system, and carrying out the screening of 3D point clouds, so as to obtain view cone point clouds; performing a point cloud clustering operation on the view cone point cloud, and calculating a result of the point cloud clustering operation to obtain a 3D frame meeting a preset requirement; and performing IOU calculation of different visual angles on the 3D frames to merge the 3D frames of the same object under different visual angles so as to obtain a 3D bounding box. According to the method, the problem of geometric information loss can be effectively avoided, and the accuracy of the finally obtained 3D bounding box is ensured; the problem that global search is needed due to the fact that 3D point cloud is directly adopted is avoided, the complexity of the calculation process is reduced, and the overall calculation efficiency is effectively improved; the preset frame requirement can be set according to the requirement of a use scene, so that different requirements are met, and the wide use range is ensured.

The embodiment of the invention provides a quality evaluation method and device, a storage medium and a computer program product. The method comprises the steps of obtaining first data; wherein the first data comprises point cloud data corresponding to one or more Three Dimensional (3D) models, and the point cloud data comprises point cloud data corresponding to one or more Three Dimensional (3D) models; generating a quality evaluation result corresponding to the 3D model based on the point cloud data and the first model; wherein the first model comprises a first module, and the first module is at least used for converting the point cloud data into a first representation form and a second representation form; wherein the first representation form comprises a voxel grid form, and the second representation form comprises a two-dimensional (Two Dimensional, 2D) image form, that is, according to the embodiment of the invention, the quality of the 3D model can be evaluated based on the voxel grid data and the 2D image data at the same time, so that the advantages of the multi-modal data can be fully utilized, and the accuracy of the quality evaluation of the 3D model is improved.

The invention relates to a laser radar point cloud densification method and device fusing image information, and a medium. The method comprises the following steps: fixing a visual camera and a laser radar on a rigid tool, and carrying out joint calibration and data alignment; a laser radar is used to collect three-dimensional point cloud data, and a visual camera is used to shoot a scene; performing super-pixel segmentation on the image by using an image brightness linear iterative clustering algorithm; mapping the three-dimensional point cloud data into a segmented two-dimensional image superpixel pattern spot region by using a jointly calibrated camera model conversion relationship to realize feature clustering of the original three-dimensional point cloud of the laser radar; performing curved surface fitting on a clustered result by using a random sampling consistency algorithm; and linear interpolation is carried out on the fitted curved surface, dense points which are not covered by the original point cloud of the laser radar are supplemented and generated, and densification of the point cloud of the laser radar is realized. According to the method, the point cloud densification precision and practicability are improved, and technical support is provided for automatic driving, robot navigation and three-dimensional reconstruction.

The present invention relates to a method for evaluating a rock mass facility by an electronic device. The method comprises the steps of: acquiring first data which is 2D image data obtained by capturing a rock mass facility by using a camera sensor; acquiring second data which is 3D point cloud data obtained by capturing the rock mass facility by using a LIDAR sensor; and analyzing the second data on the basis of the first data and the second data in order to evaluate the risk of the rock mass facility.

The invention relates to the technical field of visual inspection, in particular to a sneaker appearance flaw intelligent detection system and method based on small sample learning, and the system comprises a flaw intelligent detection center, a flaw analysis unit, a check acquisition unit, a difficulty evaluation unit and a visual feedback unit. According to the method, defect type identification and bounding box coordinate output of the sports shoes are realized in a mode of combining the model and the 2D image, meanwhile, along with precision analysis of the bounding box coordinates, data support is provided for follow-up, a defect area is obtained based on analysis of the bounding box coordinates, and meanwhile, 3D and spectral image analysis is performed on the defect area, so that the detection accuracy of the sports shoes is improved. The obtained flaw type is checked with the flaw type analyzed by the 2D image, so that the consistency of flaw results is ensured, the error rate of flaw detection results is reduced, and repair feasibility judgment processing is carried out, so that repair difficulty classification of the sneakers with flaws is facilitated, namely, objective classification is realized in combination with multi-index weighted fusion.

A computer-implemented method for recovering a three-dimensional (3D) mesh of N humans in a 3D scene, includes: receiving a two-dimensional (2D) image of the scene captured by an image capturing device, the 2D image including a plurality of regions; by one or more processors, encoding the received image and extracting features for each of the plurality of regions; by one or more processors, detecting the N humans in N respective regions among the plurality of regions; by one or more processors, based on the features in the N respective regions and the features for each of the plurality of regions, determining distribution parameters for distributions of body model and depth parameters for each of the N detected humans using a probabilistic network; providing the distribution parameters for the body model parameters for each of the N detected humans to a 3D parametric model for generating N 3D meshes.