85 results about "Pose tracking" patented technology

A composite pose-estimation algorithm includes a video-object segmentation sub-algorithm (311) configured to determine a mask of a visual object in an image, and an object-pose tracking sub-algorithm (312) configured to track a pose of a visual object over multiple depth-video frames, wherein the pose-estimation algorithm is configured to input a depth video, from which frames are extracted and fed to the video-object segmentation sub-algorithm, which determines respective object masks to be used by the object-pose tracking sub-algorithm alongside the depth video. A method of tracking a pose of a physical object comprises: obtaining a depth video depicting a physical object in a plurality of poses from an input interface (330); forming a storable data item representing the physical object by applying the pose-estimation algorithm to the depth video; and tracking the physical object or a copy thereof using an instance of the pose-estimation algorithm which has been initialized by means of the storable data item.

A method includes obtaining first and second image data of a real-world scene, performing feature extraction to obtain first and second feature maps, and performing pose tracking based on at least one of the first image data, second image data, and pose data to obtain a 6DOF pose of an apparatus. The method also includes generating, based on the 6DOF pose, first feature map, and second feature map, a disparity map between the image data and generating an initial depth map based on the disparity map. The method further includes generating a dense depth map based on the initial depth map and a camera model and generating, based on the dense depth map, a three-dimensional reconstruction of at least pail of the scene. In addition, the method includes rendering an AR or XR display that includes one or more virtual objects positioned to contact one or more surfaces of the reconstruction.

The invention provides a surgical instrument pose analysis method and system, and relates to the field of pose analysis, and the method comprises the steps: selecting a plurality of feature points on a surgical instrument as a plurality of infrared reflective mark points, and carrying out the coding processing of the infrared reflective mark points; carrying out synchronous processing on the infrared camera and the color camera and carrying out space calibration; capturing information of the surgical instrument and generating an infrared image and a color image; and constructing a target prediction network and a loss function based on the infrared image and the color image, and solving the pose by adopting a perspective n-point PnP algorithm based on a random sample consensus RANSAC framework to obtain the refined pose of the surgical instrument. According to the method, a strategy combining global shape information and local mark point features is matched with a perspective n-point PnP algorithm based on a random sample consensus RANSAC framework, the instrument shape mask information is used as additional geometric constraint and verification information to correct errors, and high-precision and high-robustness surgical instrument real-time pose tracking is achieved.

The invention relates to a human body pose tracking method. The method comprises the following steps: establishing a world coordinate system by using an SLAM (Simultaneous Localization and Mapping) system of head-mounted equipment, and acquiring an absolute pose of the head of a user in the world coordinate system; creating an individualized motion model, and obtaining individualized motion model parameters; acquiring real-time pose data of a distributed IMU node, wherein the distributed IMU node comprises a plurality of IMU units distributed on user limbs; and creating a factor graph optimization model, and determining the human body pose of the user by taking the absolute pose of the head as a root node and combining the individualized motion model parameters and the real-time pose data. The method is high in tracking precision, and can avoid the reduction of the tracking precision of the human body pose caused by integral drift. The invention further provides a human body pose tracking device, a robot control method and motion capture gloves.

The invention relates to the technical field of engineering construction measurement and control, and discloses a calibration method for positioning and attitude determination of a heading machine, which comprises the following steps: constructing a data acquisition module, a route making module, a real-time dynamic monitoring module, an execution module, a feedback module and an optimization module; the data acquisition module is used for completing comprehensive acquisition of multi-source data of the heading machine, a route, an environment and an obstacle, the route making module is used for fusing the multi-source data to carry out path planning and error compensation, and the real-time dynamic monitoring module is used for realizing heading machine pose tracking and emergency correction through visual marking points and a laser scanner. The execution module is used for converting an optimal tunneling path into a driving motor control instruction, the feedback module is used for collecting deviation data of an actual operation pose and a planned path in real time and returning the deviation data, the optimization module realizes long-term self-adaptive optimization of the system, and finally, precision, safety and high efficiency of the tunneling operation of the tunneling machine in a complex underground environment are realized.

The invention discloses a target attitude tracking method based on an enhanced Mama attention network, belongs to the technical field of computer vision, and aims to solve the problems that precision and efficiency are difficult to balance, dynamic scene adaptability is poor and semantic and spatial synchronization cannot be considered in an existing target attitude tracking method. Through deep collaboration of feature extraction and preliminary aggregation, time sequence dependence modeling based on Mama, feature optimization of attention enhancement, four-branch multi-scale adaptive attention mask generation, dynamic feature fusion and enhancement, and attitude output and assessment multi-link technical innovation, a multi-link attitude estimation method is provided. The three core problems that precision and efficiency are difficult to balance, dynamic complex scene adaptability is poor and semantic space cannot be synchronized in an existing target attitude tracking method are solved in a targeted mode, and a better solution is provided for attitude tracking requirements of industrial and other actual scenes.

The invention discloses a monocular endoscope three-dimensional pose real-time tracking method and system based on deep learning, and the method comprises the steps: splicing two input images to obtain a spliced image, and outputting a corresponding optical flow graph based on the two input images through an optical flow estimation network; extracting scene features of the two input images based on a feature extractor, and extracting motion features of the optical flow graph; meanwhile, joint features of the spliced images are extracted based on a multi-dimensional feature extractor; after splicing all the extracted features, estimating a relative pose transformation vector of the endoscope at corresponding moments of the two frames of images based on a pose decoder; and iteratively determining the absolute three-dimensional pose of the endoscope at the corresponding moment of each frame of image based on the estimated relative pose transformation vector and the absolute three-dimensional pose of the previous frame, thereby realizing real-time tracking of the endoscope. According to the invention, through deep learning and multi-feature fusion technologies, the precision and real-time performance of three-dimensional pose tracking of the monocular endoscope are significantly improved.

Systems and techniques are provided for determining a pose. A process can include obtaining inertial measurement unit (IMU) data from an IMU associated with a device. The IMU data can be used to determine a propagated state associated with a state estimation engine, wherein the propagated state includes an initial orientation estimate corresponding to a pose of the device. The state estimation engine can comprise an Extended Kalman Filter (EKF). A predicted orientation measurement can be generated using a first machine learning network to process the IMU data and the initial orientation estimate included in the propagated state associated with the state estimation engine. An updated state associated with the state estimation engine can be determined based on using the predicted orientation measurement to update the propagated state. A device pose estimate can be determined based on the updated state associated with the state estimation engine.

An XR system is provided that calibrates display parameters. The XR system captures pose data of the XR system relative to a real-world environment using a pose tracking component. Video data of the real-world environment is captured using one or more cameras of the XR system. One or more reference features in the real-world environment are identified using the video data and pose data. The XR system causes display of one or more virtual objects aligned with the reference features using display parameters. A user interface is displayed to allow a user to adjust the display parameters and XR system receives adjustments to the display parameters from the user via the interface and updates the parameters accordingly. An updated pose of the XR system is captured using the pose tracking component and the virtual objects are then re-displayed using the updated display parameters and updated pose.

Methods provide computer-navigation assisted total hip arthroplasty (THA) procedures. Such procedures include the use of a navigated instrument, e.g., a navigated reamer construct or a navigated inserter construct to prepare the acetabulum and insert an acetabular shell, a camera tracking system adapted to track a pose of the navigated inserter or reamer construct relative to the patient in use, and a computer platform adapted to receive pose tracking information and generate and display navigational guidance for the user.

This invention belongs to the field of on-orbit non-cooperative spacecraft pose tracking technology, specifically involving an online adaptive method to compensate for domain offset in on-orbit non-cooperative spacecraft pose tracking. Step 1: Represent the target spacecraft as 11 three-dimensional key points; extract the key point bounding boxes after projection, and crop to obtain the input RoI; train an asymmetric encoder-decoder network to regress the key point heatmap; Step 2: Design a dynamic memory library M based on a first-in-first-out queue; Step 3: The student model receives complete samples from M, and the teacher model receives masked samples from M; consistency constraints are implemented by minimizing the weighted MSE loss between the student and teacher predicted heatmaps; Step 4: Calculate the average value of the [CLS] token features of all training images as the source domain global class prototype, and align the source domain prototype with the newly calculated class prototypes of the teacher and student models. This invention makes it possible to achieve high-precision, highly robust, and highly practical real-time online tracking of non-cooperative spacecraft pose.

A head-mounted display, pose tracking system, and method are provided. The head-mounted display generates a plurality of first real-time images including a user holding a self-tracking device in a physical space. The head-mounted display receives first self-tracking pose information from the self-tracking device. The head-mounted display generates a hand pose information based on the first real-time images, and the hand pose information is a six-degree-of-freedom information. The head mounted display calculates drift information of the hand pose information and the first self-tracking pose information. The head mounted display calibrates the first self-tracking pose information based on the drift information.

The application relates to the technical field of camera pose tracking, and discloses a low-cost camera six-degree-of-freedom tracking method, which comprises the following steps: acquiring three-dimensional coordinates of a Pivot in a virtual scene global coordinate system, a body coordinate system offset vector of the Pivot to a camera CMOS optical center, and an installation angle deviation matrix of an IMU relative to a camera body; and based on sensing data collected by the IMU, a pose rotation matrix in an IMU coordinate system is solved, and after installation angle deviation compensation and Pan zero point offset compensation are performed, the technical problem that in the prior art, when only relying on a low-cost inertial measurement unit (IMU), position integral error accumulation cannot be avoided, and after discarding visual collection and a high-precision mechanical encoder, it is difficult to balance hardware low cost, environmental adaptability and pose output stability is solved. The application provides a low-cost camera six-degree-of-freedom tracking method.

The invention discloses a fabric-like sewing type cavity self-healing endoscope scene reconstruction method. The method comprises the following steps: carrying out initialization modeling on a two-dimensional Gaussian scene; tracking the pose of the camera; gaussian extension and key frame sampling strategy; a mapping mapping and hole perception completion module; light beam adjustment optimization of fusion smooth constraint is carried out; according to the method, firstly, continuous and dense representation of an endoscope scene is realized through two-dimensional Gaussian scene modeling and a Gaussian extension mechanism; 2, a cavity sensing and complementing mechanism is adopted, so that the mapping process has a fabric-like sewing type self-healing capability; thirdly, light beam adjustment optimization of normal smooth constraint is introduced, and the phenomena of local geometric roughness and Gaussian scene discontinuity are relieved; the overall frame is designed in a modularized mode and can be seamlessly integrated with an existing two-dimensional Gaussian mapping and light beam adjustment system; and 5, the method is suitable for a complex tubular organ scene. According to the method, topological perception and self-healing complementation of the mapping process can be realized in a complex tubular organ scene, and the continuity, stability and geometric quality of an endoscope three-dimensional reconstruction result are improved.

In some aspects, a gesture tracking device may receive availability information from a sensor system including a plurality of sensors based on a current operating condition associated with the plurality of sensors. The attitude tracking device may select a set of sensor modalities associated with the sensor system based on the availability information. The gesture tracking device may select a gesture tracking model based on the selected set of sensor modalities and one or more key performance indicator (KPI) requirements related to a current context associated with a gesture tracking configuration of the client application. The attitude tracking device may estimate an attitude associated with the object using an attitude tracking model based on sensor inputs associated with one or more sensors selected from the plurality of sensors. Numerous other aspects are described.

An XR system is provided that calibrates display parameters. The XR system captures pose data of the XR system relative to a real-world environment using a pose tracking component. Video data of the real-world environment is captured using one or more cameras of the XR system. One or more reference features in the real-world environment are identified using the video data and pose data. The XR system causes display of one or more virtual objects aligned with the reference features using display parameters. A user interface is displayed to allow a user to adjust the display parameters and XR system receives adjustments to the display parameters from the user via the interface and updates the parameters accordingly. An updated pose of the XR system is captured using the pose tracking component and the virtual objects are then re-displayed using the updated display parameters and updated pose.

The invention discloses a control method and system of a microgravity ground test system and a storage medium, relates to the technical field of robot control, and aims to solve the problem that pose tracking errors are accumulated due to the fact that an existing suspension method is difficult to adapt to real-time compensation requirements under high-frequency dynamic motion. Comprising the following steps: step S100, acquiring image information through a binocular camera, correcting radial distortion and tangential distortion of an image acquired by the binocular camera, and detecting target motion attitude and position information by adopting an AprilTag visual reference library; and S200, on the basis of the motion posture and position information obtained through detection, closed-loop control is conducted on horizontal motion of the robot through three-loop control of a current loop, a speed loop and a position loop, a dynamic distortion compensation mechanism is introduced to reduce the influence of motion of the robot or external disturbance, and the motion speed is controlled through an inter-frame difference method. According to the invention, various complex and dynamic tasks of the robot can be stably and reliably completed in a matched manner.

This application relates to the field of pose measurement, and more particularly to a pose tracking measurement method, apparatus, device, and storage medium. This application sets a reflector as a tracking point on the object being measured, establishes a target based on the reflector, and calculates the unit vector of the spatial plane normal vector; obtains a rotation matrix based on the relationship before and after rotation of the target coordinate system, and obtains the target tracking vector based on the rotation matrix and the unit vector; performs relationship matching or fitting on the target coordinate system according to the measurement requirements of the object being measured; and tracks the reflector position based on the target tracking vector and the result of the relationship matching or fitting to perform pose tracking measurement. This solves the practical measurement problems of existing technologies, such as limited pose measurement range, high measurement accuracy requirements, and complex construction of relationships between measured objects.

Techniques and systems are provided for pose prediction. For instance, a process can include combining image features detected from an obtained image with estimated image features to generate combined features; generating temporally encoded features by temporally encoding the combined features; combining detected motion tracking information with estimated motion tracking information to generate combined motion tracking information; generating temporally encoded motion tracking information by temporally encoding the combined motion tracking information; generating spatially encoded multi-modal information by spatially encoding the temporally encoded features and the temporally encoded motion tracking information; and predicting a body pose by regressing the spatially encoded multi-modal information.

The embodiment of the invention provides a high-precision spatial registration method and system based on a total ankle skeleton, and belongs to the technical field of medical image processing, and the method comprises the steps: obtaining tibia and talus three-dimensional data through CT or MRI scanning, and constructing and storing a high-precision skeleton simulation model; positioning trackers containing asymmetrically distributed passive infrared reflection balls are fixed to the tibia and the talus respectively, based on the positioning trackers, double cameras of an optical tracker are used for shooting the reflection balls, three-dimensional coordinates of mark points are obtained through parallax calculation, the real-time space posture of the bone is fitted, a registration matrix is updated, and based on a preoperative model and the tracking posture, the three-dimensional coordinates of the mark points are obtained. A talus coordinate system and a tibia coordinate system are constructed respectively, coarse registration is carried out through coarse registration points based on the coordinate systems, then fine registration points are collected through a probe, fine registration is carried out through a weighted iteration nearest point algorithm, and space registration is completed. According to the invention, through multi-modal data fusion, dynamic pose tracking and a weighted iteration nearest point algorithm, full-ankle bone submillimeter-level high-precision space registration is realized.

The invention relates to the technical field of visual tracking and positioning, in particular to an intelligent positioning system for a precast beam. The feature comparison module is used for constructing a pose tracking vector of each connecting steel bar and judging whether end movement deviation exists in a carrying track section or not, and the deviation analysis module is used for determining the category of the end movement deviation; a track correction module determines that a track correction point is set in a carrying track for pause according to the type of the end movement deviation, or the movement speed of the precast beam is regulated and controlled in a screened correction track section, and therefore the movement deviation is rapidly and accurately determined according to the steel bar projection contour of the night construction scene, and the accuracy of the movement deviation is improved. And targeted measures are taken in time for correction, so that the construction positioning efficiency and the construction positioning precision are improved.

The present invention discloses a method for constructing a 6D pose dataset of general industrial parts. First, a depth camera is fixed to capture an RGB detection image of a pose tracking plate placed in a specified industrial scene; the pose information of the AprilTag on the tracking plate in the detection image is extracted, and the tracking plate pose is obtained from the pose information using a tracking plate pose solving algorithm; according to the tracking plate pose, a virtual-real interaction registration method based on prior pose specification and augmented reality is used to arrange the parts and obtain the placement pose; an RGB-D video containing the full image of the parts and the pose tracking plate is recorded; the tracking plate pose in each frame of the video is calculated using the tracking plate pose solving algorithm, and the calibration pose of the parts is calculated in combination with the part placement pose; finally, the dataset calibration information is generated. The present invention realizes the automated 6D pose annotation of multiple parts in a large number of images in industrial cluttered scenes, greatly improving the annotation efficiency while ensuring the annotation accuracy.

The application discloses a robot operation target 6D pose tracking method based on time sequence point cloud fusion. First, a point cloud video sequence of a robot operation scene is acquired and an operation target is segmented from the point cloud video sequence, then, dense corresponding relations among time sequence observation point clouds are established in a reverse prediction mode, and the dense corresponding relations are taken as a fusion criterion to perform point-by-point feature fusion on the time sequence observation point clouds, on the basis of the point-by-point feature fusion, a transformation pose between adjacent frames is predicted from the time sequence fused features in a confidence degree regression mode, finally, the transformation pose between the adjacent frames is continuously regressed to realize continuous pose tracking of the operation target. The application tracks a 6D pose of a robot operation target from a three-dimensional point cloud video sequence based on a deep learning technology, in view of irregularity and disorder of three-dimensional point cloud data, time sequence correlation information among observation point cloud data is fully utilized and effective feature fusion is performed, accurate and stable tracking is realized, and the application has good engineering practical value.

The invention discloses a virtual-real interaction multi-prop matching tracking method and system based on multiple sensors, and the method comprises the steps: S1, receiving the image data, obtained by a multi-view camera module, of infrared light-emitting points on a plurality of prop bodies, calculating the coordinates of each infrared light-emitting point in a three-dimensional space, and calculating the coordinates of each infrared light-emitting point; obtaining a first movement track set; s2, receiving IMU attitude data sent by an inertial measurement unit on each prop body, and estimating an IMU track of an infrared light-emitting point on each prop body through pre-integration and coordinate transformation to obtain a second motion track set; s3, aligning and normalizing the tracks, and calculating the similarity score of each track through a dynamic time warping algorithm to obtain a similarity score matrix; s4, inputting the similarity score matrix into a Hungary algorithm for optimal matching to obtain an identity binding result; and S5, motion prediction is carried out on the prop body whose identity is successfully bound according to the first motion track, and continuous pose tracking is realized.

The application discloses a pose correction method and device, a chip, equipment and a storage medium, and relates to the technical field of positioning. The method comprises the following steps: determining a pose drift amount based on first sensor data of a first device and second sensor data of a second device, wherein the pose drift amount is used to represent a drift condition of a pose of the first device relative to the second device; and performing pose correction based on the pose drift amount to obtain corrected first sensor data and / or second sensor data. The embodiment scheme of the application improves the accuracy of the device pose tracking result without increasing the cost of additional hardware.

The invention relates to a whole body attitude estimation method and system based on a downward fisheye and an enhanced EgoPoseFormer model, and the method enlarges the human body capture range through the layout of the bottom visual angle of a head-mounted display, compensates geometric nonlinearity in a feature extraction stage through distortion perception convolution, cooperates with an attention mechanism of an embedded pose compensation item, and achieves the estimation of the whole body attitude. And the feature space consistency is maintained when the camera dynamically shakes. Meanwhile, in combination with cross-frame time sequence gating and a multi-modal filtering algorithm, inertial data is utilized to perform kinematics correction on a visual predicted value, and smooth track output is realized in a shielding or rapid motion scene. According to the scheme, the computing power overhead is reduced through model quantification and operator fusion, and low-delay real-time whole body attitude tracking is realized in a mobile terminal chip environment.

The invention discloses a medical bed motion tracking system and method based on trinocular vision. The system adopts a non-equidistant compact three-camera layout, the left and middle base line is about 30cm, the left and right base lines are about 40cm, and all-angle dead-angle-free medical bed pose capture is realized by matching with medical spherical reflective mark points. Aiming at the shielding problem in a clinical environment, a dual anti-shielding strategy is adopted, trinocular redundancy characteristics are physically utilized, and binocular tracking is automatically switched when a monocular is shielded; a marker ball template is introduced in algorithm to match and repair damaged features. In a stereo matching link, an epipolar line and antipolar line dual verification mechanism is provided for a one-to-many mismatching problem, mismatching is eliminated through bidirectional projection consistency, uniqueness and accuracy of a matched triple are ensured, and high-precision three-dimensional reconstruction and pose tracking are further realized.

The invention relates to the technical field of computer vision, and discloses a spacecraft pose tracking method and system based on monocular vision and three-dimensional geometric features. The method specifically comprises the following steps: extracting three-dimensional geometric features including three-dimensional edges and three-dimensional contours from a spacecraft three-dimensional model, and performing approximation of any precision on each three-dimensional geometric feature by using a trigonometric polynomial to obtain a corresponding analytic parameter equation; and sampling the control points based on the analytic parameter equation, and obtaining a maximum likelihood estimation value of the spacecraft pose in a manner of minimizing a ghosting error between the planar projection of the control points and the corresponding image edge points. And correcting the maximum likelihood estimation value of the spacecraft pose by using an extended Kalman filtering model based on second-order autoregression to obtain a final estimation value of the spacecraft pose. According to the method, high-precision and high-efficiency spacecraft tracking can be realized by virtue of very low cost, and the method has very strong robustness for shadow shielding, different backgrounds, distance changes and image noise.

The present invention discloses a method and system for real-time three-dimensional pose tracking of a monocular endoscope based on deep learning, comprising: stitching two input images to obtain a stitched image, using an optical flow estimation network to output a corresponding optical flow map based on the two input images; extracting scene features of the two input images and motion features of the optical flow map using a feature extractor; simultaneously, extracting joint features of the stitched image using a multidimensional feature extractor; after stitching all the extracted features, estimating the relative pose transformation vector of the endoscope at corresponding moments between the two frames of image using a pose decoder; iteratively determining the absolute three-dimensional pose of the endoscope at the corresponding moment of each frame of image based on the estimated relative pose transformation vector and the absolute three-dimensional pose of the previous frame, thereby achieving real-time tracking of the endoscope. Through deep learning and multi-feature fusion technology, the present invention significantly improves the accuracy and real-time performance of three-dimensional pose tracking of a monocular endoscope.

The present invention proposes a method for high-dynamic pose estimation of an intelligent agent based on motion-encoded event plane representation, comprising the following steps: 1. reading the event stream output by an event camera, where each event is represented by a four-dimensional vector; 2. constructing a bidirectional linked list to store the triggering time and sequence of pixels in a local neighborhood, and performing stack updates through an asynchronous event-driven thread; 3. constructing a motion-encoded event plane representation to achieve consistent representation of the environment in high-speed, high-dynamic scenes; 4. constructing a semi-dense scene local map in the form of a 3D point cloud; 5. encoding the spatiotemporal constraints of camera motion based on the event plane representation, and using 3D-2D alignment technology to achieve real-time estimation of the six-degree-of-freedom pose. This method, by leveraging the low latency advantage of the event camera and its natural response to scene edges, combined with semi-dense scene information, can achieve high-precision and robust pose tracking in high-speed, high-dynamic scenes, thereby fully unlocking the potential of event cameras in high-speed unmanned system applications.