294 results about "3d coordinates" patented technology

The invention relates to the field of image processing, and particularly discloses a welded pipe surface defect detection method based on robot visual inspection. The method comprises the steps that a robot carries a binocular camera and an annular LED light source and moves at a constant speed in the axial direction of a welded pipe to collect orthographic and inclined views, and a three-dimensional point cloud is constructed; and establishing a parameterized mapping function based on the point cloud, and converting the 3D coordinate into a 2D expansion surface coordinate. In the convolutional neural network, a first layer is inserted into a spatial transformation network to correct distortion of the expanded image, deformable convolution is adopted to extract edge, local deformation and specific defect response features, and standard convolution is combined to extract global features; and fusing multi-scale features and adding an attention mechanism to improve the weight of a defect region, and outputting a defect category and a bounding box offset after generating a candidate box. The method effectively solves the problems of stretching, deformation and defect distortion of welded pipe curved surface imaging, reduces the imaging difference of the same defect, and remarkably improves the defect positioning precision and recognition accuracy.

Devices, systems, and methods that present a virtual assistant that provides natural assistant interactions in an extended reality (XR) environment. For example, an example process may include presenting a view of a three-dimensional (3D) environment with a virtual assistant. The process may further include receiving data corresponding to first user activity in the 3D coordinate system and identifying a user interaction event associated with the virtual assistant based on the data corresponding to the user activity. The process may further include providing a graphical indication corresponding to one or more attributes associated with the virtual assistant based on identifying the user interaction event. The process may further include generating one or more user interface elements that are positioned at 3D positions based on the 3D coordinate system associated with the 3D environment in accordance with receiving data corresponding to a second user activity.

The invention discloses an iron tower operation safety early warning system and method, and relates to the technical field related to intelligent safety early warning, and the system comprises a data obtaining unit which is used for obtaining Beidou three-dimensional coordinate data and an operation site panoramic image in real time; the operation scene model establishing unit is used for establishing a dynamic three-dimensional operation scene model; the prediction model building unit is used for tracking the motion trail of each operation target based on the coordinate data and building a prediction model fusing kinematics rules and machine learning; and the risk level judgment unit is used for carrying out risk level judgment according to the spatial position and the attribute characteristics of the intersection area and generating operation early warning information. The technical problems of insufficient safety early warning accuracy and timeliness caused by insufficient high-altitude operation target dynamic track monitoring precision and risk early warning lagging in the prior art are solved, and the purposes of accurately identifying the movement track intersection risk in real time, dynamically evaluating the safety level and improving the safety early warning efficiency are achieved. And the early warning accuracy and timeliness of the operation safety of the iron tower are improved.

The invention is notably directed to a computer-implemented method for automatically charging an electric vehicle via an end effector (10) of a robotic arm (40) of an automated vehicle charging robot. The end effector is assumed to be structured so as to be able to connect to a charge port (220) of a vehicle. In addition, the automated vehicle charging robot further includes a camera system (102) having depth sensing capability. The method comprises the following steps. First, a reference position of a reference feature (210) of the vehicle is estimated thanks to the camera system. Next, a pose of the charge port of the vehicle is determined based on the estimated reference position. The robotic arm is subsequently instructed to actuate the end effector, based on the determined pose of the charge port, to connect the end effector to the charge port with a view to charging the vehicle. The reference position is estimated as follows. Both a 2D image and a depth image of a surface portion of the vehicle are obtained. This surface portion includes the reference feature, i.e., the feature of interest. Contour points of the reference feature are then extracted from the 2D image obtained. The 3D coordinates of the extracted contour points are subsequently reconstructed based on the depth image obtained. A geometric object (such a 2D plane) is then matched to the reconstructed 3D coordinates, e.g., by fitting the geometric object to the reconstructed 3D coordinates. Eventually, the reference position of the reference feature is determined based on the matched geometric object. The invention is further directed to related automated vehicle charging robots and computer program products.

The invention relates to the technical field of cable tunnel inspection, in particular to a cable tunnel inspection method and system based on deep learning, a cable image is collected through an inspection robot, automatic defect identification is realized based on a convolutional neural network, and further, temperature, humidity and gas concentration parameters of a defect position are combined, so that the defect position is identified. And splicing with image features to form a fusion feature vector, so that more accurate defect cause judgment is realized. And inputting the fusion feature vector into an inference model embedded with a prior rule, and outputting a cause category and a confidence score in combination with domain knowledge. And reversely positioning the cause source according to the cause label, and controlling the visual angle of the robot to obtain a corresponding image and identify the image to form closed-loop correlation of the defect and the cause source. And finally, in combination with the three-dimensional map and the pose data, the cause source is mapped into a three-dimensional coordinate point, a spatial causal relationship is established, and automatic and closed-loop intelligent inspection work of the cable tunnel is completed.

The invention provides a water leakage point positioning method and device based on resistance gradient calculation, and the method comprises the steps: obtaining a total resistance value and a tail end resistance value of a water leakage induction rope, and reading a preset resistivity parameter per unit length; calculating the distance of a water leakage point based on the total resistance value, the tail end resistance value and the resistivity parameter per unit length; according to the mapping relation between the length of the induction rope and the physical position established in the installation and debugging stage, the distance of the water leakage point is converted into the physical position described by corresponding 3D coordinates or characters; the physical position information is combined with real-time monitoring data to generate alarm prompt information containing a specific water leakage point location, and the alarm prompt information is output to a monitoring interface, so that accurate positioning of the water leakage point can be realized, the fault response time is remarkably shortened, and through a multi-thread data acquisition and dynamic alarm judgment mechanism, the real-time performance and the alarm accuracy of the system are improved; and the false alarm and repeated alarm frequency is reduced.

A computer-implemented method for 3D reconstruction of a target is provided, comprising obtaining an initial global reconstruction of the target in a 3D space, inferred by a global machine learning model; providing, to a user, an initial visualisation of the target based on the reconstruction; receiving, from the user, at least one indication of at least one point of interest in the visualisation; resampling at least one first subsection of the target based on the at least one point of interest to obtain local data, wherein the local data is associated with the subsection based on spatial information that associates the local data with a point in 3D space; inputting the resampled local data and spatial information into a local feature machine learning model to obtain at least one 3D reconstruction of the target, wherein the local feature machine learning model has been trained to output a target reconstruction from local data of resampled subsections, and wherein the 3D coordinate system of the local 3D reconstruction aligns with the global 3D reconstruction; and merging the global 3D reconstruction with the local 3D reconstruction. A corresponding computer system and computer readable medium may also be provided.

The invention provides a three-dimensional calibration method using a speckle image as a calibration object and combining weight parameters. The method comprises the following steps: acquiring a standard grid image and the speckle image through a camera; determining angular point positions in the standard grid image through an angular point detection algorithm; calibrating the stereoscopic vision system by using the obtained angular points to obtain an initial calibration result; calculating 3D coordinates of the camera calibration board in the world by using the disparity map; determining a subset of the reference image corresponding to the checkerboard angular points in the target image through a digital image cross-correlation algorithm; calculating a weight parameter of each calibration point based on the re-projection error; solving internal and external parameters of the camera by using the weight parameters, radial alignment constraint and a least square method; optimizing the main point of the camera by using the weight parameters; and if the re-projection error or the number of iterations does not meet the requirement, repeatedly calculating the weight parameter of the calibration point and the subsequent steps, otherwise, obtaining a camera calibration result, and obtaining the distortion curved surface of the camera lens.

The invention discloses a GIS in-place robot coordination control method with a posture correction function, and belongs to the technical field of robot coordination control. The method comprises the following steps: establishing a 2D-3D coordinate mapping corresponding relation; fusing the corresponding relation with an operation state predicted by an IMU-based inertial measurement and system dynamics model; performing error calculation on the obtained 6D pose information and target pose information; constructing a nonlinear kinetic equation of the robot, and converting a control target into an optimization problem of various constraints; and converting the obtained control instruction into a torque signal for actually driving the robot to move. According to the method, the influence of mismatching on pose estimation is effectively inhibited by establishing the 2D-3D coordinate mapping relation; through a joint torque PID control algorithm, the torque required by each joint is calculated in real time, and high-precision tracking of an expected trajectory is realized; an MPC optimization control framework with multiple constraints is constructed, and precise, smooth and safe coordination control over the GIS in-place process is achieved.

The invention discloses a raspberry fruit identification and positioning method based on RGB-D image and depth information fusion. The method comprises the following steps: synchronously acquiring an RGB image and a depth image of a raspberry fruit through an RGB-D camera; constructing an RGB-D data set containing a fruit mask and a fruit stem connection point mark; establishing a depth fusion model based on YOLO11, wherein the model comprises a depth feature encoder, a CGAFusionPro fusion module and a 3D positioning head; the training model realizes multi-task learning; inputting an RGB-D image to obtain a 2D detection frame, a fruit surface 3D coordinate, a fruit stem offset and depth information confidence; and performing surface normal vector correction and curvature compensation through a depth perception optimizer to generate accurate 3D coordinates of the stem connection points. The invention provides a raspberry fruit identification and positioning method based on RGB-D image and depth information fusion aiming at the problem of low raspberry fruit identification and positioning accuracy under different scales, through depth information fusion and physical perception optimization, the problems of registration dislocation and positioning deviation in a traditional method are solved, and the accuracy of raspberry fruit identification and positioning is improved. And the raspberry fruit identification and positioning accuracy is effectively improved.

A method of measuring a 3D shape of an object comprising a target semi-transparent surface, the method comprising the steps of: with a projector, projecting a first light signal having a predetermined pattern and a predetermined frequency from at least one projector column towards at least one location on a scene; with an image capture device, receiving a second light signal from the at least one location of the scene at an image capture device pixel; measuring an intensity signal received at the image capture device pixel; with the measured intensity signal, generating a correspondence map between the at least one projector column illuminating the at least one location of the scene and the image capture device pixel with coordinates receiving the light signal from the at least one location of the scene; and with the correspondence map, determining 3D coordinates of the at least one location of the scene.

The invention discloses a stereoscopic vision-based model attitude measurement method, belongs to the technical field of attitude measurement, aims to solve the problems of insufficient precision and expensive depth camera of traditional attitude estimation, realizes accurate measurement of model attitude, and has the core of fusing stereoscopic vision geometric characteristics and deep learning advantages. The method specifically comprises the steps of 1, forming a three-dimensional system by using at least two industrial cameras, calibrating internal and external parameters, and calculating a scene depth map through an improved SGBM algorithm to obtain object depth information, and 2, predicting coordinates of pixels in a target 3D coordinate system by using a pre-trained double-branch fusion network in combination with the depth map and left and right eye RGB images, the method comprises the steps of 1, generating a plurality of groups of candidate poses on the basis of association and spatial association, 2, restoring the candidate poses into virtual objects, comparing the virtual objects with real objects through geometric consistency verification to quantify scores, and 3, selecting the candidate pose with the highest score as a final result through a PoseSelection module. And large view field coverage and high-precision positioning are both considered.

A MARKIII cargo containment system cargo hold size measuring method comprises the steps that a cargo hold is divided into 10 faces and 16 angular points in total, laser trackers are arranged at the centers of the bow face and the stern face, a plurality of target balls are arranged in the three faces corresponding to each angular point, the laser trackers are used for detecting the target points in the three faces corresponding to the angular points, and after detection is finished, the cargo hold size is measured through the target balls. According to the method, the planes can be created in polyworks software, after the three planes corresponding to the angular points are created, coordinates of intersection points formed by intersection of the three planes are three-dimensional coordinates of the current angular points, and three-dimensional coordinates of the 16 angular points are obtained through measurement in sequence. And inputting the three-dimensional coordinates of the angular points into EXCEL to directly obtain the lengths of line segments and diagonals of the cargo hold. According to the method disclosed by the invention, the number of times of measurement is reduced from original 64 times to 16 times and is reduced by 75%. And the labor hour is reduced, 20 persons are required to measure for 2 days in the original method, 5 persons are required to measure for 1 day in the method, and the labor hour is reduced by 87.5%. The measurement precision is improved, and calculation errors are avoided.

Aspects of the disclosure provide methods and apparatuses for mesh coding (encoding and / or decoding). In some examples, an apparatus for coding mesh includes processing circuitry. The processing circuitry decodes, three dimensional (3D) coordinates of vertices in a first 3D mesh frame and connectivity information of the vertices from a bitstream that carries the first 3D mesh frame. The first 3D mesh frame represents a surface of an object with polygons. The processing circuitry deduces texture coordinates associated with the vertices, and decodes a texture map for the first 3D mesh frame from the bitstream. The texture map includes first one or more 2D charts with 2D vertices having the texture coordinates. The processing circuitry reconstructs the first 3D mesh frame based on the 3D coordinates of the vertices, the connectivity information of the vertices, the texture map and the texture coordinates.

A computer implemented method for determining a 3D position of a surgical object relative to a region of interest in a patient's anatomy. The method includes receiving an X-ray image and a 3D data set. The X-ray image is a 2D projection image generated with an imaging direction and defining a 3D image coordinate system. The 2D projection image may depict a part of a patient's anatomy and a part of the surgical object. The received 3D data set describes the part of the patient's anatomy and a region of interest in a 3D data coordinate system. In order to determine a spatial relation between the 3D image coordinate system and the 3D data coordinate system, the part of the patient's anatomy is localized based on the X-ray image. Subsequently, the 3D position of the surgical object relative to the region of interest is determined in the 3D coordinate system.

The invention discloses a code-configuration-free digital twin warehouse system and a construction method thereof. The construction method comprises the following steps: initializing a warehouse digital twin platform, importing entity object data, and establishing a basic two-dimensional data visualization framework and a three-dimensional rendering engine; collecting multi-source heterogeneous data in real time and preprocessing the multi-source heterogeneous data to obtain a real-time data source; 2D / 3D coordinate precision mapping and dynamic data coupling are carried out; calling a code-free configuration tool, receiving a real-time data source for a user to drag and use, and assembling a logic flow chart by the user through dragging; analyzing the logic flow chart assembled by the user through middleware to generate a low-level instruction sequence; the low-level instruction sequence is input into an external actuator and a physical simulation engine; virtual verification and situational simulation are executed through a physical simulation engine, and optimization suggestions are output and adjusted. The invention provides a code-free configuration integration mechanism, supports non-technical personnel to autonomously construct data-driven twinning application, and realizes transformation of warehouse digital twinning from static monitoring to dynamic iteration.

The embodiment of the invention discloses an aircraft height measurement method based on binocular vision, and the method comprises the steps: obtaining internal parameters and distortion parameters of a first camera and a second camera, and obtaining a first image and a second image collected by the first camera and the second camera; calibrating the first image and the second image according to the internal reference and the distortion parameter of the first camera and the second camera to obtain a first calibration image and a second calibration image; binocular matching is carried out on the first calibration image and the second calibration image, and a full-image disparity map is obtained based on the first calibration image; performing airport runway semantic segmentation based on the first calibration image to obtain a semantic segmentation map of the airport runway; performing 3D coordinate reduction operation on the 2D pixels according to the full-image disparity map and the semantic segmentation map to obtain a 3D point cloud of the airport runway; fitting a runway plane equation according to the 3D point cloud, converting the runway plane equation to an airframe coordinate system according to the relative position of the first camera and the airframe, and obtaining an aircraft height measurement result according to the runway plane equation under the airframe coordinate system.

The invention discloses a wide-range high-precision multi-structure optical sensor calibration method and system. The method comprises the following steps: completing combined installation of an optical platform; the guide rail is controlled to move away from the device from the initial distance, after movement is completed according to the preset step length, data are sequentially collected through the structured light sensor, the grating ruler position of the calibration block is recorded, and all data are collected until the calibration block moves to the farthest distance; recording an X coordinate corresponding to each circle center on the calibration block, and recording a moving coordinate of the guide rail as a Y coordinate by a grating ruler; all circle centers are recognized on the 2D image of each position, coordinates of the circle centers in a camera coordinate system are obtained, world coordinates on the calibration blocks are combined, world coordinates corresponding to all 3D coordinates output by the camera are calculated through interpolation, and a calibration record table is formed; and combined calibration of multiple sensors is completed. And reliable technical support is provided for multi-sensor fusion measurement.

A monocular high-speed videogrammetry method based on a dynamic platform includes the following steps: collecting a monocular high-frame-rate image sequence of the dynamic platform and performing preprocessing; conducting dynamic image-block data processing on the image sequence based on adaptive brightness compensation and least-squares fitting (LSF) to obtain adjusted images; performing camera-pose calibration on the adjusted images based on sliding-window constraints to obtain calibrated images; and acquiring three-dimensional (3D) coordinates and estimated displacements of tracking points from the calibrated images, and outputting measurement results based on the 3D coordinates and the estimated displacements. Through optimization of the measurement process, the method supports real-time adaptive compensation, dynamic image-block processing, and camera-pose calibration, thereby providing significant application value. Correspondingly, the present disclosure also provides a monocular high-speed videogrammetry system based on a dynamic platform, which includes modules configured to implement the above steps.

The invention relates to the technical field of printed circuit board assembly quality detection, and discloses an IC carrier plate detection method and system based on multi-mode laser imaging and AI fusion. The method comprises the following steps: acquiring three-dimensional point cloud data of a PCBA board to be detected; generating a reference three-dimensional digital twin model according to a standard PCBA design drawing; carrying out spatial registration on the three-dimensional point cloud data and the reference three-dimensional digital twin model, obtaining the three-dimensional point cloud data, carrying out hierarchical processing on the obtained three-dimensional point cloud data, extracting geometric features of a welding spot region, contour features of an element region and surface features of a substrate region, and carrying out fusion to generate a feature vector group; constructing a generative adversarial model based on a preset semi-supervised learning framework and the normal PCBA sample vector group; and inputting the feature vector group into a generative adversarial model, and examining the abnormal vectors, the corresponding three-dimensional coordinates and the abnormal types in the feature vector group by the generative adversarial model to complete the abnormal detection of the PCBA board. The method is suitable for quality control of a high-density and miniaturized PCBA.

The invention relates to the technical field of three-dimensional modeling, in particular to a method for dynamically simulating the restoration effect of a rattan cube slope, which comprises the following steps of: based on a remote sensing image and a point cloud, reconstructing a slope surface grid according to gradient abrupt change and consistent slope direction, extracting three-dimensional coordinates and contact layout of anchoring points, and analyzing stress and density to mark disturbance levels. And screening continuous deformation nodes to form a deformation path, and drawing a tension deviation track and an intersection sequence. According to the method, remote sensing image gray scale and point cloud elevation difference are fused, terrain broken lines are extracted for grid reconstruction according to gradient abrupt change and slope direction consistency, terrain features and engineering layout are matched, three-dimensional coordinates of anchoring points are extracted, and spatial topology layout is generated in a grouping mode according to the layout direction. And constructing a multi-dimensional interference level system according to a main tensile stress and node density discrimination disturbance unit, screening a continuous deformation path, drawing a tension deviation track and indexing an intersection section, and realizing multi-section repair evolution visualization.

The invention discloses a multi-source fusion continuous positioning method in combination with a prior feature map, and aims to solve the problem that an existing vision-inertial odometer easily generates accumulative errors in a GNSS denial environment, the method comprises the following steps: constructing a prior feature map comprising key frame poses, feature point descriptors and 3D coordinates based on an open source algorithm; a track displacement difference alignment strategy is adopted to complete visual-inertial system and GNSS joint initialization, and ESIKF is used to replace EKF to improve fusion precision; robust alignment of a local coordinate system and a map coordinate system is achieved through a single-frame geometric constraint accumulation-batch manifold alignment strategy; based on the reverse PnP, the current 3D point cloud and the historical 2D observation constraint are utilized to resolve the relative pose, the map matching result is used as observation to be fused with the GNSS or replace GNSS observation, the ESIKF loose coupling is utilized to eliminate accumulative errors, and continuous high-precision positioning in the GNSS failure scene is achieved.

According to the measurement positioning system based on cooperation of vision and the total station, the PTZ camera, the total station and the central control module are integrated. The central control module comprises a positioning module which obtains the area where the channel is located through the PTZ camera and carries out coarse positioning on the channel in the obtained image so as to determine the approximate orientation of the target channel. And the cooperation module is used for acquiring and controlling the Hroute and Vroute rotation relative quantity of a total station measurement target point based on the position of the total station laser point in the PTZ camera image, and establishing a high-precision cooperation working mechanism of the total station and the PTZ camera. And the data acquisition module is used for acquiring three-dimensional data of channel edge points when edge scanning is carried out on the channel. And the data processing module is used for fitting the contour of the channel, reconstructing an accurate three-dimensional geometric model of the channel, and calculating accurate three-dimensional coordinates of all preset operation points to realize high-precision positioning measurement of the channel.

The present disclosure provides a method and a system for the dynamic calibration of an in-vehicle camera (102). In particular, the present disclosure provides a system comprising a processor (214) that extracts three-dimensional (3D) coordinates of a plurality of vehicle contours from a pre-stored design model (208) of a vehicle. The processor (214) further detects, using a trained model (210), two-dimensional (2D) pixel coordinates corresponding to a plurality of edges of the plurality of vehicle contours from a plurality of input images. Furthermore, based on the 3D coordinates and the 2D pixel coordinates corresponding to the plurality of edges of the plurality of vehicle contours, the processor (214) calculates one or more camera parameters for the dynamic calibration of the vehicle camera (102).

A system includes a phased array antenna, a signal processing module configured to detect wireless signals received from one or more target devices, a conversion module configured to determine 3D spatial data for the one or more target devices based on the wireless signals, an integration module configured to receive visual data from at least one camera, perform object detection and classification on the visual data to detect and classify one or more objects, and tag the one or more objects with 3D coordinates based on the 3D spatial data for the one or more target devices; and a correlation module configured to synchronize the 3D spatial data from the conversion module with the visual data from the integration module including object detection and classification information for use in a cloud-based augmented reality (AR) content to augment social networking and targeted advertising.

This application provides a 3D reconstruction method, apparatus, device, and readable storage medium. An electronic device acquires multiple first images, each containing the background of a target scene and dynamic objects within that scene, with the sum of these first images capable of displaying a panoramic view of the target scene. The electronic device then performs masking processing on the regions containing dynamic objects in each of the first images to obtain an image set. Based on this image set, the background of the target scene is reconstructed to obtain a background point cloud, which is then used to generate a 3D reconstruction result of the target scene. This approach, by masking the regions containing dynamic objects in the first images of the target scene, eliminates the influence of feature point extraction and matching results on 3D coordinate estimation and camera extrinsic parameter estimation, ensuring successful reconstruction of the background point cloud of the target scene. This improves the success rate of 3D reconstruction and enhances the immersive conferencing experience.

The application discloses a kind of video conference method, device, electronic equipment and storage medium, belong to video conference technical field, the method includes the same fixed position camera, obtains scene image, and the same target in the same scene at least different position image of interval preset distance;According to scene image, establish 3D coordinate system, and determine scene and target parameter;According to the height value of camera and the parameter data of target, and horizon plane, the scene and its corresponding 3D coordinate system are corrected;Video image and the voice signal that is synchronous with video image are collected;Based on 3D coordinate system, and according to video image and voice signal, determine the position of current speaker in video image;According to the position of current speaker in video image and corresponding voice information, the identity of current speaker is marked.Solve the problem that there can be inaccurate position recognition in video conference, improve the overall performance and user experience of system.

The present teaching relates to obtaining three-dimensional (3D) information in a contactless manner using a one-dimensional (1D) distance sensor. The 1D distance sensor is associated with an origin from where a laser beam emitted in an orientation, where the origin / orientation are calibrated with respect to a marker attached thereon and tracked. In a surgery, the calibrated 1D distance sensor is tracked by a camera via the marker attached thereon. When an emitted laser beam hits at a location on a patient, the 3D coordinate thereof in the tracker coordinate system is determined based on tracked marker, the origin / orientation of the 1D distance sensor, and a distance between the origin of the 1D distance sensor and the location.

The invention discloses a multi-camera structured light three-dimensional reconstruction method based on a global phase correspondence table, and the method comprises the steps: building a multi-camera structured light system, and obtaining the parameters of each camera through calibration; a projector projects a structured light pattern sequence, and a plurality of cameras synchronously collect an imaging sequence modulated by the surface of a measured object; calculating two-dimensional absolute phase information based on the imaging sequence; generating a discretized two-dimensional phase key for each effective pixel according to the two-dimensional absolute phase information; constructing a global phase correspondence table, and storing all camera information corresponding to the discretized two-dimensional phase keys; and querying the global phase correspondence table, identifying a multi-camera pixel correspondence relationship sharing the same discretized two-dimensional phase key, and reconstructing a three-dimensional coordinate in combination with parameters calibrated by the cameras. According to the invention, a direct pixel matching relation is established among a plurality of camera views by constructing the global phase correspondence table, and the three-dimensional reconstruction efficiency, precision and consistency of the multi-camera structured light system are improved.