477 results about "3d coordinates" patented technology

The invention discloses a building electromechanical BIM model information rapid retrieval method and system, and the method comprises the steps: generating composite retrieval parameters fusing semantic keywords and three-dimensional coordinate constraints according to a multi-mode retrieval instruction inputted by a user; on the basis of the composite retrieval parameters, constructing a dynamic search space by utilizing a hierarchical graph convolutional network, and generating a candidate model index structure of multi-dimensional feature coding; inputting the candidate model index into a multi-objective optimization engine, performing real-time optimization on a search path by adopting a dynamic pruning algorithm driven by reinforcement learning, and outputting a candidate model set of which the confidence coefficient is higher than a preset confidence threshold after pruning; and on the basis of the candidate model set, associated equipment nodes are expanded through a knowledge graph embedding and complementing technology, and an enhanced retrieval result set containing the hidden associated equipment is generated. By utilizing the embodiment of the invention, efficient, multi-dimensional and multi-modal information accurate positioning and quick retrieval can be realized in a large-scale complex BIM model.

The invention relates to the field of image processing, and particularly discloses a welded pipe surface defect detection method based on robot visual inspection. The method comprises the steps that a robot carries a binocular camera and an annular LED light source and moves at a constant speed in the axial direction of a welded pipe to collect orthographic and inclined views, and a three-dimensional point cloud is constructed; and establishing a parameterized mapping function based on the point cloud, and converting the 3D coordinate into a 2D expansion surface coordinate. In the convolutional neural network, a first layer is inserted into a spatial transformation network to correct distortion of the expanded image, deformable convolution is adopted to extract edge, local deformation and specific defect response features, and standard convolution is combined to extract global features; and fusing multi-scale features and adding an attention mechanism to improve the weight of a defect region, and outputting a defect category and a bounding box offset after generating a candidate box. The method effectively solves the problems of stretching, deformation and defect distortion of welded pipe curved surface imaging, reduces the imaging difference of the same defect, and remarkably improves the defect positioning precision and recognition accuracy.

The invention provides an adaptive welding seam detection and three-dimensional reconstruction method based on deep learning and binocular vision. The adaptive welding seam detection and three-dimensional reconstruction method comprises the steps of S1, collecting samples and making a training data set; s2, the picture of the sample to be welded is processed, a feature region is recognized, the image quality of the region to be welded is analyzed and evaluated through wavelet transform and local variance, and the noise level and the contrast ratio are calculated; s3, dynamically generating edge detection parameters and model fitting parameters according to the image quality; s4, using an edge detection algorithm to extract edge point cloud of the welding seam area; s5, performing RANSAC linear fitting, weighted least square fitting and polynomial curve fitting on the edge point cloud in parallel; s6, selecting an optimal fitting result based on an image quality adaptive dynamic scoring model; and S7, carrying out three-dimensional coordinate conversion in combination with the three-dimensional matching model IGEV-Stereo, and outputting a final welding seam three-dimensional coordinate. According to the invention, automatic detection of the position and size of the welding seam can be efficiently and accurately realized.

The invention discloses an RGB-D-based van carton stacking 3D point cloud recognition and positioning method, and belongs to the field of machine vision and automatic logistics. An existing visual detection algorithm is poor in recognition and positioning accuracy, complex in calculation and low in detection efficiency. The method comprises the following steps: acquiring RGB images of a target object under two visual angles and depth images of the target object under the two visual angles, performing denoising processing on the acquired images, combining the denoised images of the same visual angle to obtain a three-dimensional point cloud image under the corresponding visual angle, and fusing the three-dimensional point cloud images under the two visual angles to generate a complete three-dimensional point cloud image; the method comprises the steps of optimizing point cloud data in a point cloud image, inputting an optimized two-dimensional RGB point cloud image into a pre-trained YOLO target detection model, outputting a detection image, correcting center coordinates of sub-objects in the detection image, obtaining corrected two-dimensional center point coordinates of all the sub-objects, further obtaining three-dimensional coordinates, and finally obtaining a three-dimensional point cloud image. The method is used for identifying coordinate points of each sub-object.

Devices, systems, and methods that present a virtual assistant that provides natural assistant interactions in an extended reality (XR) environment. For example, an example process may include presenting a view of a three-dimensional (3D) environment with a virtual assistant. The process may further include receiving data corresponding to first user activity in the 3D coordinate system and identifying a user interaction event associated with the virtual assistant based on the data corresponding to the user activity. The process may further include providing a graphical indication corresponding to one or more attributes associated with the virtual assistant based on identifying the user interaction event. The process may further include generating one or more user interface elements that are positioned at 3D positions based on the 3D coordinate system associated with the 3D environment in accordance with receiving data corresponding to a second user activity.

A method includes generating, using at least one processing device, a ground truth optical flow map for displacement of pixels within a reference image based on a motion model. The motion model determines 3D coordinates of pixels within the reference image based on estimated depths of the pixels within the reference image. The method also includes performing, using the at least one processing device, 3D to 2D reprojection of the pixels within the reference image based on the motion model to generate a reprojected image view corresponding to a shifted camera perspective. The method further includes generating, using the at least one processing device, an occlusion mask for the reference image. The occlusion mask corresponds to occluded pixels within the reprojected image view. In addition, the method includes performing, using the at least one processing device, occlusion region inpainting of the occluded pixels to generate an inpainted reprojected image view.

The invention discloses a pedestrian re-recognition method based on a space-time hybrid Transform and a skeleton diagram attention mechanism, and belongs to the field of computer vision and pattern recognition. The invention aims to solve the problem that the existing pedestrian re-identification method based on a skeleton cannot fully utilize spatial-temporal characteristics to represent pedestrian identity information. Comprising the following steps: processing 3D coordinates of each articulation point in a skeleton diagram by adopting a track branch to obtain track characteristics of all articulation points of each frame of skeleton diagram; an adjacent matrix is determined based on the hinging relation of all the joint points by adopting a structure branch, and then processing is carried out to obtain an integrated structure feature corresponding to each frame of skeleton diagram; fusing the trajectory features with the integrated structure features to obtain space-time mixed features; and processing the space-time mixed feature to obtain an identity feature vector so as to realize pedestrian re-identification. The method is used for pedestrian re-identification.

The invention discloses an iron tower operation safety early warning system and method, and relates to the technical field related to intelligent safety early warning, and the system comprises a data obtaining unit which is used for obtaining Beidou three-dimensional coordinate data and an operation site panoramic image in real time; the operation scene model establishing unit is used for establishing a dynamic three-dimensional operation scene model; the prediction model building unit is used for tracking the motion trail of each operation target based on the coordinate data and building a prediction model fusing kinematics rules and machine learning; and the risk level judgment unit is used for carrying out risk level judgment according to the spatial position and the attribute characteristics of the intersection area and generating operation early warning information. The technical problems of insufficient safety early warning accuracy and timeliness caused by insufficient high-altitude operation target dynamic track monitoring precision and risk early warning lagging in the prior art are solved, and the purposes of accurately identifying the movement track intersection risk in real time, dynamically evaluating the safety level and improving the safety early warning efficiency are achieved. And the early warning accuracy and timeliness of the operation safety of the iron tower are improved.

The invention discloses an isocenter calibration quality control system and method for a radiotherapy image guidance system, and relates to the technical field of medical image processing, and the method comprises the steps: S1, enabling radiotherapy rays to be imaged on a scintillator through an optical body surface imaging mode, and obtaining the radiation isocenter of a linear accelerator according to the imaging information of the scintillator; s2, calculating a 3D point cloud of the ball array by using an image processing module to obtain a 3D coordinate of the center position of the ball array; and S3, aligning the 3D coordinate of the center position of the ball array with the radiation isocenter of the linear accelerator to realize calibration of a point cloud coordinate system so as to accurately position the patient in a positioning and treatment stage. According to the invention, the isocenter position of the accelerator can be accurately positioned, the isocenter point of the accelerator is mapped to the point cloud three-dimensional space coordinate for coordinate alignment, accurate positioning and calibration of the system are realized, and the system can also be used for positioning verification before radiotherapy.

A method for automatically determining a canonical pose of a 3D object comprises: providing one or more blocks of voxels of a voxel representation of the 3D object to a first 3D deep neural network, the first 3D neural network being trained to generate canonical pose information; receiving canonical pose information from the first 3D deep neural network, the canonical pose information comprising for each voxel a prediction of a position of the voxel in the canonical coordinate system; using the canonical coordinates to determine an orientation and scale of the axes of the canonical coordinate system and a position of the origin of the canonical coordinate system relative to the axis and the origin of the first 3D coordinate system and using the orientation and the position to determine transformation parameters of the first coordinate system into canonical coordinates; and, determining a canonical representation of the 3D dental structure.

The invention relates to a cryoelectron microscope density map reconstruction method based on a multi-modal deep learning framework, which comprises the following steps: obtaining a two-dimensional projection image from a cryoelectron microscope experiment, screening a particle image, and carrying out preprocessing including denoising, normalization and pose and CTF estimation; then dispersing the initial density field into a three-dimensional grid, uniformly sampling 3D coordinates, rotating and translating the coordinates in combination with pose information obtained by preprocessing, and simulating experimental conditions; thirdly, carrying out Fourier pose coding on the 3D coordinates, mapping the 3D coordinates to a high-dimensional space, combining the 3D coordinates with latent variables extracted by an image encoder, inputting the 3D coordinates into an MLP, generating an initial low-resolution 3D density map, carrying out U-Net refining to enhance local details, projecting the 3D density map into a 2D image through a differentiable model, and calculating loss with an original particle image to optimize MLP parameters; a density map with a more accurate structure is generated; and finally, the image quality under different resolutions is analyzed and evaluated through FSC, and the model performance is verified. According to the invention, the reconstruction efficiency and precision are improved.

A robot interference checking motion planning technique using point sets. The technique uses CAD models of robot arms and obstacles and converts the CAD models to 3D point sets. The 3D point set coordinates are updated at each time step based on robot and obstacle motion. The 3D points are then converted to 3D grid space indices indicating space occupied by any point on any part. The 3D grid space indices are converted to 1D indices and the 1D indices are stored as sets per object and per time step. Interference checking is performed by computing an intersection of the 1D index sets for a given time step. Swept volumes are created by computing a union of the 1D index sets across multiple time steps. The 1D indices are converted back to 3D coordinates to define the 3D shapes of the swept volumes and the 3D locations of any interferences.

The invention relates to a multi-user holographic sand table collaborative resolving system and method based on 360-degree omni-directional perception gesture recognition. The method comprises the following steps: acquiring continuous multi-frame multi-view hand image data; hand instance separation and key point extraction are carried out on the multi-view hand image data of each frame, and spatial 3D coordinates of 21 key points are independently solved for each hand instance; based on the space 3D coordinate sequence of the continuous frames, space-time action features are extracted, and a gesture template library is matched to recognize gesture types and motion parameters; analyzing the gesture type, matching an interaction intention in combination with a predefined gesture-instruction mapping table, and generating an interaction control instruction according to a matching result; generating a three-dimensional holographic sand table image cooperatively operated by multiple persons in real time according to the interaction control instruction; through a multi-hand parallel processing technology, different hand instance feature extraction paths are separated, the problem of multi-hand ID jump is solved, occlusion area coordinate prediction is optimized, and interactive use experience is improved.

The invention is notably directed to a computer-implemented method for automatically charging an electric vehicle via an end effector (10) of a robotic arm (40) of an automated vehicle charging robot. The end effector is assumed to be structured so as to be able to connect to a charge port (220) of a vehicle. In addition, the automated vehicle charging robot further includes a camera system (102) having depth sensing capability. The method comprises the following steps. First, a reference position of a reference feature (210) of the vehicle is estimated thanks to the camera system. Next, a pose of the charge port of the vehicle is determined based on the estimated reference position. The robotic arm is subsequently instructed to actuate the end effector, based on the determined pose of the charge port, to connect the end effector to the charge port with a view to charging the vehicle. The reference position is estimated as follows. Both a 2D image and a depth image of a surface portion of the vehicle are obtained. This surface portion includes the reference feature, i.e., the feature of interest. Contour points of the reference feature are then extracted from the 2D image obtained. The 3D coordinates of the extracted contour points are subsequently reconstructed based on the depth image obtained. A geometric object (such a 2D plane) is then matched to the reconstructed 3D coordinates, e.g., by fitting the geometric object to the reconstructed 3D coordinates. Eventually, the reference position of the reference feature is determined based on the matched geometric object. The invention is further directed to related automated vehicle charging robots and computer program products.

The invention relates to the technical field of cable tunnel inspection, in particular to a cable tunnel inspection method and system based on deep learning, a cable image is collected through an inspection robot, automatic defect identification is realized based on a convolutional neural network, and further, temperature, humidity and gas concentration parameters of a defect position are combined, so that the defect position is identified. And splicing with image features to form a fusion feature vector, so that more accurate defect cause judgment is realized. And inputting the fusion feature vector into an inference model embedded with a prior rule, and outputting a cause category and a confidence score in combination with domain knowledge. And reversely positioning the cause source according to the cause label, and controlling the visual angle of the robot to obtain a corresponding image and identify the image to form closed-loop correlation of the defect and the cause source. And finally, in combination with the three-dimensional map and the pose data, the cause source is mapped into a three-dimensional coordinate point, a spatial causal relationship is established, and automatic and closed-loop intelligent inspection work of the cable tunnel is completed.

The invention provides a water leakage point positioning method and device based on resistance gradient calculation, and the method comprises the steps: obtaining a total resistance value and a tail end resistance value of a water leakage induction rope, and reading a preset resistivity parameter per unit length; calculating the distance of a water leakage point based on the total resistance value, the tail end resistance value and the resistivity parameter per unit length; according to the mapping relation between the length of the induction rope and the physical position established in the installation and debugging stage, the distance of the water leakage point is converted into the physical position described by corresponding 3D coordinates or characters; the physical position information is combined with real-time monitoring data to generate alarm prompt information containing a specific water leakage point location, and the alarm prompt information is output to a monitoring interface, so that accurate positioning of the water leakage point can be realized, the fault response time is remarkably shortened, and through a multi-thread data acquisition and dynamic alarm judgment mechanism, the real-time performance and the alarm accuracy of the system are improved; and the false alarm and repeated alarm frequency is reduced.

A computer-implemented method for 3D reconstruction of a target is provided, comprising obtaining an initial global reconstruction of the target in a 3D space, inferred by a global machine learning model; providing, to a user, an initial visualisation of the target based on the reconstruction; receiving, from the user, at least one indication of at least one point of interest in the visualisation; resampling at least one first subsection of the target based on the at least one point of interest to obtain local data, wherein the local data is associated with the subsection based on spatial information that associates the local data with a point in 3D space; inputting the resampled local data and spatial information into a local feature machine learning model to obtain at least one 3D reconstruction of the target, wherein the local feature machine learning model has been trained to output a target reconstruction from local data of resampled subsections, and wherein the 3D coordinate system of the local 3D reconstruction aligns with the global 3D reconstruction; and merging the global 3D reconstruction with the local 3D reconstruction. A corresponding computer system and computer readable medium may also be provided.

The invention provides a three-dimensional calibration method using a speckle image as a calibration object and combining weight parameters. The method comprises the following steps: acquiring a standard grid image and the speckle image through a camera; determining angular point positions in the standard grid image through an angular point detection algorithm; calibrating the stereoscopic vision system by using the obtained angular points to obtain an initial calibration result; calculating 3D coordinates of the camera calibration board in the world by using the disparity map; determining a subset of the reference image corresponding to the checkerboard angular points in the target image through a digital image cross-correlation algorithm; calculating a weight parameter of each calibration point based on the re-projection error; solving internal and external parameters of the camera by using the weight parameters, radial alignment constraint and a least square method; optimizing the main point of the camera by using the weight parameters; and if the re-projection error or the number of iterations does not meet the requirement, repeatedly calculating the weight parameter of the calibration point and the subsequent steps, otherwise, obtaining a camera calibration result, and obtaining the distortion curved surface of the camera lens.

The invention discloses a GIS in-place robot coordination control method with a posture correction function, and belongs to the technical field of robot coordination control. The method comprises the following steps: establishing a 2D-3D coordinate mapping corresponding relation; fusing the corresponding relation with an operation state predicted by an IMU-based inertial measurement and system dynamics model; performing error calculation on the obtained 6D pose information and target pose information; constructing a nonlinear kinetic equation of the robot, and converting a control target into an optimization problem of various constraints; and converting the obtained control instruction into a torque signal for actually driving the robot to move. According to the method, the influence of mismatching on pose estimation is effectively inhibited by establishing the 2D-3D coordinate mapping relation; through a joint torque PID control algorithm, the torque required by each joint is calculated in real time, and high-precision tracking of an expected trajectory is realized; an MPC optimization control framework with multiple constraints is constructed, and precise, smooth and safe coordination control over the GIS in-place process is achieved.

The invention provides a disordered grabbing device and method for multiple layers of steel balls in a ball milling workshop. The device comprises a steel ball storage pool (1), a mechanical arm (2), a visual module (3) and a control module. The mechanical arm, the visual module and the control module are electrically connected, and the visual module and the mechanical arm are arranged on the two sides of the steel ball storage pool respectively. The steel ball storage pool is used for storing multiple layers of disorderly arranged steel balls; the visual module is used for performing point cloud identification on the positions of the steel balls in the stock bin; the mechanical arm grabs the calibrated steel balls through an end effector electromagnet (12) and places the calibrated steel balls at designated points of the conveying belt. The control module is used for controlling the vision module to obtain the 3D coordinates of the multiple layers of stacked steel balls and controlling the pose, the tail end path and the grabbing rhythm of the mechanical arm.

Examples described herein provide a method that includes capturing data about an environment. The method further includes generating a database of two-dimensional (2D) features and associated three-dimensional (3D) coordinates based at least in part on the data about the environment. The method further includes determining a position (x, y, z) and an orientation (pitch, roll, yaw) of a device within the environment based at least in part on the database of 2D features and associated 3D coordinates. The method further includes causing the device to display, on a display of the device, an augmented reality element at a predetermined location based at least in part on the position and the orientation of the device.

An embodiment includes a method of determining a collision-free space for a robotic welding system. The method includes fixing a location of a part to be welded in a 3D coordinate space of a robotic welding system. An arm of the robotic welding system is moved around the part within the 3D coordinate space. Data corresponding to positions and orientations of the arm in the 3D coordinate space are recorded as the arm is moved within the 3D coordinate space around the part. The data is translated to swept volumes of data within the 3D coordinate space. The swept volumes of data are merged to generate 3D geometry data representing a continuous collision-free space within the 3D coordinate space.

To provide a three-dimensional geological model correction device, method, and program that can quickly and effectively correct a three-dimensional geological model using strata information obtained from face observation.SOLUTION: A three-dimensional geological model correction device comprises: three-dimensional geological model generation means 30 for generating a three-dimensional geological model of a construction target including three-dimensional coordinates and three-dimensional stratum boundary data based on previously acquired stratum information of the construction target; face image acquisition means 21 for acquiring a face image of an already excavated area; face coordinate acquisition means 22 for acquiring three-dimensional coordinates of the face image; two-dimensional stratum boundary data extraction means 23 for extracting two-dimensional stratum boundary data from the face image; identical boundary extraction means 40 for extracting identical boundaries based on proximity of three-dimensional coordinates of boundary lines constituting the three-dimensional stratum boundary data and the boundary lines constituting the two-dimensional stratum boundary data; and three-dimensional stratum boundary data correction means 50 for correcting the three-dimensional stratum boundary data based on the two-dimensional stratum boundary data related to the extracted identical boundaries.SELECTED DRAWING: Figure 1

A method for providing live broadcasting sessions in virtual environments includes creating a virtual environment based on virtual environment creation instructions; connecting the virtual environment to an event that is broadcast through the virtual environment; setting up a live broadcasting session from the virtual environment comprising the event broadcast; sending an invitation to at least one invitee client device; opening up a communication channel enabling access to the at least one invitee to the virtual environment, wherein the invitee client device accesses the virtual environment through a user graphical representation that is positioned at the specified 3D coordinate of the virtual environment; and streaming the live broadcasting session to the at least one invitee client device. Corresponding systems and computer-readable media are also provided.

The invention discloses a raspberry fruit identification and positioning method based on RGB-D image and depth information fusion. The method comprises the following steps: synchronously acquiring an RGB image and a depth image of a raspberry fruit through an RGB-D camera; constructing an RGB-D data set containing a fruit mask and a fruit stem connection point mark; establishing a depth fusion model based on YOLO11, wherein the model comprises a depth feature encoder, a CGAFusionPro fusion module and a 3D positioning head; the training model realizes multi-task learning; inputting an RGB-D image to obtain a 2D detection frame, a fruit surface 3D coordinate, a fruit stem offset and depth information confidence; and performing surface normal vector correction and curvature compensation through a depth perception optimizer to generate accurate 3D coordinates of the stem connection points. The invention provides a raspberry fruit identification and positioning method based on RGB-D image and depth information fusion aiming at the problem of low raspberry fruit identification and positioning accuracy under different scales, through depth information fusion and physical perception optimization, the problems of registration dislocation and positioning deviation in a traditional method are solved, and the accuracy of raspberry fruit identification and positioning is improved. And the raspberry fruit identification and positioning accuracy is effectively improved.

Method for registering medical 2D single-section images in a common 3D coordinate system comprising: acquiring an image sequence of several 2D single-section images of a body part of a patient with an imaging device; for each of the several 2D single-section images: determining a number of 2D contours in the 2D single-section image, determining a sparse point cloud with multiple surface points on each of the 2D contours, and positioning and orienting the point cloud in the 3D coordinate system based on a position and orientation of the 2D single-section image;and registering a first of the point clouds on a second of the point clouds by determining a spatial transformation which transforms the first point cloud into the second point cloud in such a way that a mismatch between the first point cloud transformed according to the transformation and the second point cloud is as small as possible, by minimizing a loss function which specifies a deviation between two 3D volumes densely populated with brightness values ​​obtained by rasterizing the point clouds in question.

A method of measuring a 3D shape of an object comprising a target semi-transparent surface, the method comprising the steps of: with a projector, projecting a first light signal having a predetermined pattern and a predetermined frequency from at least one projector column towards at least one location on a scene; with an image capture device, receiving a second light signal from the at least one location of the scene at an image capture device pixel; measuring an intensity signal received at the image capture device pixel; with the measured intensity signal, generating a correspondence map between the at least one projector column illuminating the at least one location of the scene and the image capture device pixel with coordinates receiving the light signal from the at least one location of the scene; and with the correspondence map, determining 3D coordinates of the at least one location of the scene.

Systems and techniques are described herein for object detection. For example, a computing device can extract, by an encoder of the computing device, a plurality of features from one or more images of an environment of the computing device. The computing device can determine, based on the plurality of features, a first detection of one or more objects and three-dimensional (3D) coordinates for the one or more objects. The computing device can back-project the 3D coordinates of the one or more objects onto the one or more images. The computing device can determine one or more regions of at least one first image of the one or more images based on the back-projection of the 3D coordinates of the one or more objects. computing device determine, based on the one or more regions of the at least one first image, a second detection of the one or more objects.

A system includes a projector that projects onto an object a pattern of light having a temporal modulation that includes a whole number of adjoined modulation cycles. The system further includes a first time-of-flight (ToF) camera having a lens and a two-dimensional (2D) array of pixels that convert light reflected off the object into an electrical signal. At least one processor determines three-dimensional (3D) coordinates of the object point based at least in part on the electrical signal, a length of the baseline, the temporal modulation, and the speed of light in air.

The invention discloses a stereoscopic vision-based model attitude measurement method, belongs to the technical field of attitude measurement, aims to solve the problems of insufficient precision and expensive depth camera of traditional attitude estimation, realizes accurate measurement of model attitude, and has the core of fusing stereoscopic vision geometric characteristics and deep learning advantages. The method specifically comprises the steps of 1, forming a three-dimensional system by using at least two industrial cameras, calibrating internal and external parameters, and calculating a scene depth map through an improved SGBM algorithm to obtain object depth information, and 2, predicting coordinates of pixels in a target 3D coordinate system by using a pre-trained double-branch fusion network in combination with the depth map and left and right eye RGB images, the method comprises the steps of 1, generating a plurality of groups of candidate poses on the basis of association and spatial association, 2, restoring the candidate poses into virtual objects, comparing the virtual objects with real objects through geometric consistency verification to quantify scores, and 3, selecting the candidate pose with the highest score as a final result through a PoseSelection module. And large view field coverage and high-precision positioning are both considered.