1656 results about "2d images" patented technology

The invention discloses a ceramic product defect detection and analysis method and system, and relates to the technical field of intelligent detection and analysis, and the method comprises the steps: collecting the multi-modal data of a ceramic product, and carrying out the data preprocessing; for the preprocessed data, identifying a defect area existing on the surface of the ceramic by adopting an image analysis method, and marking the position and the range of the defect area; constructing a classification model, taking the identified defect region data as input, dividing defect types, and extracting features of each defect; the classified defect features are compared with a preset production process parameter rule base, and possible causes of defect formation are analyzed; and generating a detection report according to classification and analysis results. According to the invention, accurate identification and classification of the surface defects of the ceramic product are realized. 2D images and 3D shape data are combined, so that the comprehensiveness and the accuracy of detection are greatly improved; and meanwhile, through a cause analysis model, defect characteristics are associated with production process parameters, and clear cause inference and optimization suggestions are provided.

The invention relates to a building prefabricated part quality detection method based on multi-modal vision. According to the method, multi-modal data including 2D image data and 3D point cloud data are obtained, an improved YOLOv8 model is used for performing defect coarse positioning on the 2D image data, defect parameters are calculated, defect areas such as cracks and exposed ribs can be quickly locked, and the parameters of the defect areas can be obtained. And by means of SIFT feature matching and ICP point cloud registration technologies, comparing with a two-dimensional template and three-dimensional geometric parameters of the BIM standard component model to obtain a two-dimensional registration difference chart and a three-dimensional deviation thermodynamic chart. And finally, according to the defect confidence coefficient, the two-dimensional registration difference chart and the three-dimensional deviation thermodynamic diagram, a preset dynamic weighting rule is adopted to carry out joint decision making, and a quality detection result is obtained. According to the method, through the multi-modal data, the improved YOLOv8 model, the point cloud registration technology and the preset dynamic weighting rule, the false detection problem can be effectively solved, the detection reliability and accuracy under the complex working condition are improved, and the quality management level of the building prefabricated part is improved.

An illustrative point cloud segmentation system generates an instance-wise semantic mask for a particular source image of a set of source images that has been used to construct a 3D point cloud representing a scene that includes one or more objects. The point cloud segmentation system maps a set of 3D points from the 3D point cloud to corresponding 2D points of the particular source image, then labels, based on contours defined by the instance-wise semantic mask to demarcate the one or more objects, the mapped set of 3D points in accordance with where the corresponding 2D point for each mapped 3D point is positioned with respect to the contours. Based on the labeling of the mapped 3D points, the point cloud segmentation system produces a segmented 3D point cloud including an instance-wise segmentation of the one or more objects at the scene. Corresponding methods and systems are also disclosed.

A method and apparatus for reconstructing a three-dimensional (3D) scene from a two-dimensional (2D) input image of the scene using a fully-differentiable transformer-based encoder-decode. A 2D input image encoded into a set of image features using a pre-trained vision transformer model, wherein the vision transformer model is pre-trained with multi-view RGB image supervision and point cloud supervision. The set of image features is projected onto a 3D triplane representation using a transformer decoder to obtain output triplane tokens. A triplane representation is created from the tokens and queried. 3D point features of color and density for volumetric rendering re predicted using a multi-layer perceptron. The geometry of the generated 3D asset is represented with a surface mesh including vertices and triangular faces. A texture map by is created with a multichannel image in UV space. Multiple views of the 3D scene are simultaneously generated based on the surface mesh.

A system accesses, during a camera scan of a 3D physical object, video feed data of a user computing device including a plurality of frames. The system generates camera scan data including a set of 2D images of the physical object generated from a subset of the plurality of frames. Generating the camera scan data can include, responsive to determining that a translation between a previous frame and a current frame is greater than a threshold, including the current frame in the camera scan data. Generating the camera scan data can include excluding the current frame from the camera scan data responsive to determining an undesired camera movement type associated with the current frame. Generating the camera scan data can include indicating, in a surface coverage preview model, points of the 3D physical object included in at least a predefined number of consecutive frames of the subset of frames.

A method includes receiving 2D image data corresponding to a 2D image of a human face. The method further includes determining a textual identifier that describes a facial feature of the human face based on the 2D image data. The method further includes providing, to a generative machine learning model, a first prompt including information identifying the textual identifier that describes the facial feature of the human face. The method further includes obtaining, from the generative machine learning model, a first output identifying, among a plurality of beauty products, a subset of the plurality of beauty products, the subset of the plurality of beauty products related to the facial feature of the human face.

Methods and processors for rendering a 3D object are disclosed. The method includes acquiring multi-camera image input including first image frames of the 3D object generated by a first camera and second image frames of the 3D object generated by a second camera, acquiring an initial 3D Gaussian Splatting (3DGS) model having a plurality of initial parameters including an initial frame-wise GS parameter and an initial camera-wise GS parameter, generating an adjusted 3DGS model by adjusting, based on the multi-camera image input, at least one of: the initial frame-wise GS parameter, the initial camera-wise GS parameter, generating, by the adjusted 3DGS model, a 3DGS output and rendering a 2D image of the 3D object using the 3DGS output.

The invention discloses a tunnel fracture recognition method based on three-dimensional live-action reconstruction and ordered re-acquisition of a virtual camera. The method specifically comprises the following steps: acquiring a multi-view rock fracture image; acquiring three-dimensional point cloud data of a tunnel wall surface according to the acquired rock mass fracture image, reconstructing a geometrical shape of the tunnel, mapping acquired texture information to a three-dimensional grid vertex through an image processing algorithm, and generating a three-dimensional tunnel model; setting parameters of the virtual camera, orderly collecting to form a virtual image, and enabling the position of the virtual camera to correspond to the real position of the model; and extracting information of the crack of the virtual image, and projecting the coordinates of the two-dimensional image back to the three-dimensional space through the projection matrix to obtain the space coordinates of the crack. A virtual image is generated by combining a texture mapping technology of a three-dimensional model, so that the limitation of a visual angle, illumination and space in actual image acquisition is overcome, and high-precision identification of tunnel cracks is realized.

The invention provides an open vocabulary 3D segmentation method based on a three-dimensional Gaussian sputtering technology, and the method comprises the steps: S1, obtaining a multi-view 2D image set, and carrying out the initialization three-dimensional reconstruction; s2, segmenting the multiple views by using SAM to obtain a 2D mask; s3, associating masks of the same instance by using a video tracking segmentation model (SAM2); s4, extracting open vocabulary features of each mask by using a visual language model (CLIP); s5, performing average pooling and dimension reduction on the CLIP feature of the mask of each instance to form a representative open vocabulary feature of the instance; s7, training the model, and supervising the embedding of a low-dimensional embedding feature # imgabs0 #; and S8, comparing the CLIP feature of the query text with the semantic feature of the Gaussian body to obtain the Gaussian body with high correlation, and generating a mask of the query object through rasterization rendering. According to the method, only a multi-view 2D image is needed, and the three-dimensional object and the scene are segmented by combining the three-dimensional Gaussian sputtering technology and the open vocabulary technology, so that the problems of difficulty in 3D data acquisition and high cost are solved.

A system generates, based on a displacement bounds data structure and a triangle mesh modeling a surface of a 3D virtual object within a 3D virtual scene, a displaced triangle mesh including one or more displaced surface bounding prisms, each of the one or more displaced surface bounding prisms displaced from a respective base triangle of a plurality of base triangles of the triangle mesh structure based on displacement bounds defined in a displacement bounds data structure for an area of a 2D texture space corresponding to a location of the respective base triangle defined by the 3D virtual scene. The system performs, using the displaced triangle mesh structure, a ray tracing process for a ray associated with a pixel of a 2D image of the virtual scene including determining, responsive to determining the ray intersects the particular displaced surface bounding prism, a location of an intersection of the ray.

The invention discloses a shield segment dislocation detection method and system fusing visual and geometric features, and the method comprises the steps: employing a mobile track three-dimensional laser scanning system, and rapidly obtaining the three-dimensional point cloud data of a tunnel segment; projecting the three-dimensional point cloud data of the tunnel segment to a two-dimensional image according to the scanning parameters and the image preset resolution, and mapping the laser point reflection intensity into a pixel gray value; shield segment inter-ring joints and shield segment in-ring joints are extracted respectively, and segment joint positioning is completed; and according to the seam positioning information, extracting local point clouds at the two sides of the seam, respectively carrying out circular model fitting, calculating the height difference of circular models at the two sides at the seam, and obtaining an in-ring slab staggering value. According to the method, the visual information of the point cloud reflection intensity and the geometric information of the spatial position are fused, rapid and accurate shield segment in-ring slab staggering detection is realized, and the efficiency and the accuracy are high.

A three-dimensional target detection and tracking method fusing multi-sensor information belongs to the technical field of computer vision, and can make up for the defect of a single mode by collecting camera and radar image information and fusing camera images and laser radar point clouds. Multi-target tracking is carried out by using a multi-data coupling method of 2D + 3D data, a remote object is identified through a camera, when the object enters an induction range of the laser radar, the laser radar can carry out more accurate positioning, and position estimation of the target in a 2D image is corrected by using a more accurate 3D position provided by the laser radar. The method is used for solving the problems of inaccurate target identification, target shielding and the like under complex road conditions; a traditional single-mode three-dimensional target detection algorithm cannot effectively fuse the advantages of multi-sensor data and is difficult to accurately identify objects in a distance or under a complex road condition.

The invention discloses an implicit image enhancement and optical flow estimation method based on multi-mode collaborative optimization. The method comprises the following steps: firstly, acquiring an RGB image in a normal illumination scene and a depth map of a corresponding view angle; and calculating corresponding three-dimensional point cloud data according to the depth map and the camera internal reference. And synthesizing low-illumination image data by using the RGB images in the normal illumination scene. Decomposing high-frequency features and low-frequency features of the low-illumination image by using a high-low-frequency feature enhancement network to enhance the low-illumination image, and then extracting image features and context features of the low-illumination image; and extracting 2D image features and 3D point cloud features by using a 2D-3D feature fusion network, and performing alignment fusion to obtain image features and context features of a normal RGB image for supervising the feature extraction process of the high and low frequency feature enhancement network. And finally, based on the image features and context features of the low-illumination image, constructing a 4D correlation volume table, and reasoning the optical flow by using GRU.

In a method of using a structured-light based system, real-time 2D images of a portion of a field of view are captured using an endoscope. A portion of an object in the field of view is illuminated with a structured light pattern, and light reflected from the field of view is detected.From the reflected light, a 3D image of the field of view is constructed, and 3D locations of points on a surface of the object are determined. The real time 3D spatial position of the endoscope and / or a surgical tool is determined. If a distance between the surface the endoscope and / or surgical tool, as determined using the 3D spatial position, falls below a predetermined distance, an alert is generated to notify a user.

A method for recovering a 3D mesh of N humans in a 3D scene comprises: encoding a 2D image from an image capturing device to extract embedded features for each of a plurality of regions; detecting N humans in N respective regions among the plurality of regions; processing the embedded features in the N respective regions and the embedded features for each of the plurality of regions to predict body model and depth parameters using a decoder comprising a cross-attention module; providing the predicted body model parameters to a 3D parametric model for generating 3D meshes; and placing the generated 3D meshes at respective 3D spatial locations based on the predicted depth parameters.

The invention discloses a laser welding seam online quality detection system and method based on multi-mode visual fusion, and particularly relates to the technical field of laser welding seam quality detection. Comprising a data acquisition and synchronous control unit, a 2D image processing and defect detection unit, a 3D point cloud processing and size measurement unit, a multi-modal information fusion and collaborative analysis unit and a comprehensive quality judgment and output unit. According to the laser welding seam online quality detection system and method based on multi-modal visual fusion, the 2D and 3D multi-modal visual fusion technology is adopted, an improved defect detection algorithm and an accurate size measurement method are combined, welding seam surface defects are accurately recognized through 2D images, key size data are obtained by means of 3D point cloud, and the detection accuracy is improved. And the limitation of'heavy defects and light sizes' or'heavy sizes and light defects' in single-modal detection is avoided, and comprehensive and accurate evaluation of the welding seam quality is realized.

A system to capture a two dimensional digital source image of a scene by a user, including a smart device having a memory device for storing an instruction, a processor in communication with the memory and configured to execute the instruction, a digital image capture device in communication with the processor, said processor configured to capture a first two dimensional digital source image of the scene, said processor configured to execute an instruction to generate a second two dimensional digital image of the scene from said first two dimensional digital image of the scene via a camera angle rotation of between 1-180 degrees of said first two dimensional digital image of the scene, and a display in communication with the processor, the display configured to display a multidimensional digital image.

Various implementations relate to methods, systems and computer readable media to provide body tracking from monocular video. According to one aspect, a computer-implemented method includes obtaining a video including a set of video frames depicting movement of a human subject; extracting 2D images of the human subject from the video frames; providing the 2D images as input to a pre-trained neural network model. The method further includes determining a pose of the subject based on the 2D images. The method further includes generating a 3D pose estimation of upper body joint positions of the human subject. The method further includes determining confidence scores, and selecting a set of keypoints of the upper body joints of the human subject based on the confidence scores. The method further includes animating a 3D avatar using at least the selected set of keypoints, and displaying the animated 3D avatar in a user interface.

The invention is suitable for the technical field of mechanical arm control, and provides a self-adaptive mechanical arm clamping control method and system based on image recognition, the system comprises a mechanical arm, and the mechanical arm comprises a mechanical arm body and a mechanical clamping jaw; the image acquisition module is used for acquiring a 2D image and depth information of an object; the image processing module is used for carrying out preprocessing, contour extraction and pose coordinate determination on the acquired image; the data transmission module is used for transmitting the contour, size and pose coordinate information of the object processed by the image processing module to the controller; the controller is used for receiving information sent by the data transmission module and controlling movement and clamping force of the mechanical arm; according to the mechanical arm, the path can be automatically planned and proper force can be applied according to the characteristics of different objects, the flexibility, the accuracy and the safety of the clamping operation of the mechanical arm are improved, and the mechanical arm is suitable for various automatic scenes.

The invention discloses a dynamic digital twinning system based on 3D Gaussian splashing. The dynamic digital twinning system comprises a flexible electric power inspection robot, a 3DGS dynamic enhancement module and a fusion digital twinning background centralized control system. The flexible electric power inspection robot is used for collecting 2D images and environment point cloud data of substation equipment; the 3DGS dynamic enhancement module is used for carrying out 3DGS dynamic enhancement and generating a 3D Gaussian model; the standard digital twinborn centralized control platform is used for storing and calling data and digital twinborn bodies of the traditional static substation digital model; the model fusion port carries out space alignment and attribute interpolation fusion on the 3D Gaussian model and the static digital twin; and the visual terminal is used for displaying the fused enhanced digital twinborn body and the equipment state evaluation result. According to the invention, the dynamic enhancement of the digital twinborn model of the substation equipment can be realized, and the operation state and change condition of the equipment can be reflected more truly and intuitively.

Disclosed herein are methods, systems, and aircraft for performing image analysis for aiding refueling operations. A tanker aircraft includes a camera system configured to generate a two-dimensional (2D) image of a receiving aircraft and a depth image of the receiving aircraft, a processor, and non-transitory computer readable storage media storing code. The code is executable by the processor to perform operations including detecting the receiving aircraft within the 2D image based on 2D image values, determining 2D keypoints of the receiving aircraft located within the 2D image based on the depth image, the 2D image values, and a previously determined keypoint model, determining a 6 degree-of-freedom (6DOF) pose using the 2D keypoints and corresponding three-dimensional (3D) keypoints, and outputting a position of at least a component of the receiving aircraft based on the 6DOF pose.

The invention discloses a deburring method combining a 2D image and a 3D point cloud, and relates to the technical field of image processing, and the method comprises the steps: carrying out the point cloud registration under the verification of the surface orientation consistency of a standard point cloud and an actual point cloud; constructing a pixel mapping matrix between the workpiece image and the RGB image; semantic-level edge coarse extraction is carried out on the workpiece image, and extraction of a sub-pixel-level actual edge contour is carried out on the basis of coarse extraction in combination with a traditional edge detection algorithm; aligning the actual edge contour to the actual point cloud through the pixel mapping matrix; and the target polishing surfaces are aligned to the actual point cloud, to-be-polished edge contours are screened based on the distances between the contours of all the target polishing surfaces in the actual point cloud and all the actual edge contours, and a polishing track is obtained in the workpiece image. According to the method, the sub-pixel edge positioning capability of the 2D image and the spatial topology information of the 3D point cloud are fused, and the inherent defect of a single mode is overcome.

A method and an apparatus for 4D road scene annotation based on time series data includes: obtaining point cloud data and multi-view 2D image data of a same road scene at a same time series, as well as a rotation matrix R and a translation vector t obtained through sensors; merging all the point cloud data into a combined point cloud using the rotation matrix R and the translation vector t; performing dynamic and static object annotation and lane line annotation on the combined point cloud in the 4D road scene; and mapping annotation information in the 4D road scene to all the 2D image data at the time series based on camera parameter information, to obtain annotation information on 2D images. The method fully utilizes time series information, avoiding the loss of important information.

The present disclosure describes techniques for generating three-dimensional models using machine learning models. A two-dimensional (2D) image is input into a machine learning model. The machine learning model is configured to generate three-dimensional (3D) models with accurate geometry and detailed textures. A set of multi-view images is generated based at least in part on the 2D image by a first sub-model of the machine learning model. The first sub-model comprises a multi-level image prompt controller configured to implement hierarchical controls over generating multi-view images by the first sub-model based at least in part on an input image. A 3D model is generated based at least in part on the set of multi-view images by a second sub-model of the machine learning model. The second sub-model is configured to implement a background alignment and a camera alignment for improving quality and geometric accuracy of the generated 3D models.

The invention provides a graph rendering method and system based on a GLSL shader, and relates to the technical field of three-dimensional model rendering, and the rendering method comprises the steps: loading a 3D model, extracting data of a control point of the 3D model in a VBO of a GPU, generating a first geometric primitive, extracting vertex data in the first geometric primitive, generating an index array of the vertex data, and creating an EBO, based on an index array stored in the EBO, a vertex shader is used for sequentially processing vertex data, obtaining the position and interpolation data of each vertex, cutting the vertex data, regenerating a second geometric primitive, converting the second geometric primitive into a fragment, and a fragment shader calculates color data of each fragment; and the GPU performs parallel rendering on the color data to obtain a final output image, so that resource consumption and memory occupation are reduced while the 3D model is converted into the 2D image, and the rendering efficiency is improved.

The invention provides a spacecraft-oriented multi-task data set construction method. A high-precision satellite geometric model is created. The image rendered based on the PBR not only has a bright-eye visual effect, but also conforms to a physical optics principle, and particularly, a normal mapping simulates a star body wrinkle effect. A space simulation scene is constructed by using an unreal engine 4, and a vivid rendering effect is provided. A high-quality satellite image is generated by combining a scene capture component and a target rendering function of an unreal engine, and a semantic segmentation tag, an instance segmentation tag, a depth tag, a normal tag and a pose tag are automatically generated based on a 3D model 2D image automatic labeling method. And a large-scale data set can be quickly generated in a frame-by-frame rendering and automatic labeling mode. Semantic segmentation is carried out on the generated image through the deep learning model DeepLab V3 +, the accuracy and efficiency of model prediction are further improved, and the rendering trueness is greatly improved through detailed analysis and optimization.

A method for aligning a laser point cloud and an image based on deep learning includes: collecting data by using a laser ladar and a camera, to obtain laser point cloud data and a two-dimensional (2D) image respectively; dividing preprocessed point cloud data into triangular meshes to obtain a three-dimensional (3D) mesh model of the point cloud data; determining a relative location relationship between a laser radar and the multi-function camera, and constructing a virtual camera in the 3D mesh model to capture a simulated 2D image at a same location and angle as those of the camera; and aligning, by using a convolution neural network (CNN), the 2D image captured by the camera and the simulated 2D image captured by the virtual camera to generate aligned images that map onto each other.

The invention discloses an imaging equipment positioning guiding method, system and equipment based on sparse reconstruction, and the method comprises the steps: employing imaging equipment, and automatically collecting 2D images of a designated region of a simulation patient from a plurality of sparse angles; selecting a target tissue to be modeled on the 2D image; performing 3D modeling on the selected target tissue by adopting an extremely sparse reconstruction algorithm to form a 3D modeling image of the target tissue; registering the 3D modeling image and the 2D image, and calculating a first conversion relation between a visual coordinate system and a motion coordinate system of the imaging equipment; adjusting the initial pose of the 3D modeling image, and calculating the target position of the imaging device corresponding to the adjusted pose based on the first conversion relation and the adjusted pose of the 3D modeling image; and controlling the imaging equipment to reach the target position so as to obtain an expected imaging view angle of the target tissue.

A three-dimensional (3D) texture is generated for an input 3D model based on input text describing the desired texture. Example methods include rendering the 3D model and trainable 3D texture to generate a first two-dimensional (2D) image, adding noise to the first 2D image to generate a first 2D image with added noise, and inputting the first 2D image with added noise and input text into a trained neural network to generate a predicted noise of the first 2D image with added noise. The methods further include determining a loss between the first 2D image with added noise and the predicted noise and updating the trainable 3D texture based on the loss. The method is repeated for a number of time or until a loss between the first 2D image and the predicted noise transgresses a threshold.

A system comprises an encoder configured to compress attribute information and / or spatial for a point cloud and / or a decoder configured to decompress compressed attribute and / or spatial information for the point cloud. To compress the attribute and / or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. Additionally, an encoder is configured to signal and / or a decoder is configured to receive a supplementary message comprising volumetric tiling information that maps portions of 2D image representations to objects in the point. In some embodiments, characteristics of the object may additionally be signaled using the supplementary message or additional supplementary messages.