Hydraulic pump part tracking and registering method based on model image joint perception
Through the method based on joint perception of model images, the fast point feature histogram algorithm and position estimation calculation method are used to solve the low cost, efficiency and stability problems of hydraulic pump parts tracking and registration, and the real-time and accuracy of hydraulic pump assembly are improved.
Patent Information
- Application Number
- CN202510874146.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-27
- Publication Date
- 2025-07-25
- Estimated Expiration
- 2045-06-27
AI Technical Summary
The prior art is difficult to achieve low-cost, efficient and stable hydraulic pump parts tracking and registration, especially in complex environments, traditional methods have problems such as poor environmental adaptability, high hardware dependence and high computational complexity.
Using a method based on joint perception of model images, a point cloud model is generated by collecting photo streams of hydraulic pump parts, and a two-stage neighborhood hierarchical search mechanism is constructed with a fast point feature histogram algorithm to integrate the three-factor feature weighting to perform 3D-2D feature point matching, and the pose estimation algorithm is used to update the pose information, reducing calculation costs, and improving registration accuracy and robustness.
It has achieved improvements in time performance and robustness, reduced processor computing costs, improved real-time and accuracy of registration, reduced calculation volume by about 20%-50%, enhanced the robustness and expression ability of feature points, and is suitable for hydraulic pump assembly in complex scenarios.
Smart Images

Figure CN120374423A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision image processing, and specifically relates to a method for tracking and registering hydraulic pump parts based on joint perception of model images. Background Art
[0002] The research scope of augmented reality technology is extensive. From the perspective of the technical architecture, it encompasses many key fields such as camera calibration technology, tracking and registration technology, and scene fusion and display technology. Among them, the core essence of an augmented reality system lies in achieving the fusion of virtual information and the real scene. The tracking and registration technology provides spatial position and attitude reference data for virtual-real fusion by solving the pose information of the camera relative to the real scene.
[0003] Due to the similar shapes of some parts of the hydraulic pump, traditional tracking and registration methods based on single modalities such as two-dimensional images or markers have problems such as poor environmental adaptability, high hardware dependence, and high computational complexity, making it difficult to achieve efficient and stable tracking and registration. On the contrary, strategies that can achieve relatively stable tracking and registration, such as high-precision laser scanning, are too costly. Therefore, designing a low-cost, efficient, and stable hydraulic pump part tracking and registration method that also ensures real-time performance and robustness has become an urgent technical problem to be solved, which is of great significance for improving the automation level and quality control of hydraulic pump assembly. Summary of the Invention
[0004] The purpose of the present invention is to provide a method for tracking and registering hydraulic pump parts based on joint perception of model images to solve the technical problems raised in the background art.
[0005] To achieve the above purpose, the present invention provides the following technical solutions: Offline stage: S1. Collect the photo stream of hydraulic pump parts, obtain the point cloud model from the photo stream, denoise and optimize the point cloud density of the point cloud model, and store it in the database as the original point cloud; Online stage: S2. Use the calibrated camera to collect consecutive frame images of the tracking target in the hydraulic pump assembly environment in real time, and construct a two-stage neighborhood hierarchical search mechanism and three-factor feature weighted fusion based on the fast point feature histogram algorithm to extract the feature points of the original point cloud in the database; S3. Perform 3D-2D feature point matching on the feature points extracted from the database and the feature points of the consecutive frame images of the real-time obtained tracking target to achieve dynamic coupling; S4. Update the target pose according to the matching situation, estimate the transformation relationship between the original point cloud in the database and the consecutive frame images of the real-time obtained tracking target, and use the pose estimation algorithm to update the external parameters of the camera rotation and translation information to perform pose estimation of the tracking target object. S5. Introduce corrective measures to optimize the estimated pose, and use TCP to transmit the optimized pose data to the client for visual verification of the hydraulic pump assembly.
[0006] Furthermore, the specific steps of step S2 are as follows: S21. Calculate the Simplified Point Feature Histogram (SPFH) of each point in the original point cloud within its first-stage neighborhood; S22. Perform L1 normalization on the SPFH feature corresponding to each point to obtain the feature vector corresponding to this point ; S23. Further search for K nearest neighbor points in the second-stage neighborhood of each point in the original point cloud, and obtain the Fast Point Feature Histogram (FPFH) feature corresponding to each point by weighted fusion of its SPFH features ; S24. Perform L1 normalization on the FPFH feature corresponding to each point to obtain the feature vector corresponding to this point ;
[0007] Furthermore, the calculation formula of step S21 is: , In the above formula, , represents the first-stage neighborhood corresponding to the i-th point where represents the distance between the i-th point and the j-th point , is a distance threshold; is an angle mapping function, where K represents the number of histogram bins, , respectively represent the polar angle and azimuth angle on the unit vector of the line connecting the i-th point and the j-th point ; is a linear distance weight function, , represents the generation of a K-dimensional SPFH feature for the i-th point.
[0008] Furthermore, the calculation formula of step S22 is: , In the above formula, represents the FPFH feature value of the i-th point , represents the i-th point The corresponding feature vector, and K represents the number of histogram bins.
[0009] Furthermore, the calculation formula in step S23 is: , In the above formula, , represents the second-stage neighborhood corresponding to the m-th point , The function means to find the K points closest to centered on the point ; is the first-stage neighborhood feature weight coefficient; is the Gaussian distance weight function, where is the distance between the neighborhood point and the point , is the smoothing parameter; is the normalized SPFH feature of each neighbor point , represents generating a K-dimensional FPFH feature for the i-th point.
[0010] Furthermore, the calculation formula in step S24 is: , In the above formula, represents the FPFH feature value of the i-th point , represents the corresponding feature vector of the i-th point , and K represents the number of histogram bins.
[0011] Furthermore, the specific steps of step S3 are: S31. Convert the coordinate unit of each 3D point in the extracted feature points and then convert it into homogeneous coordinate form; S32. Project the 3D points in homogeneous coordinate form onto a 2D plane and normalize the pixel coordinates projected onto the 2D plane; S33. Map the normalized projected point coordinates to the pixel coordinate system of the actual image and perform coordinate transformation according to the focal length and image center coordinates in the camera internal parameters; S34. Calculate the Euclidean distance between the projected points mapped to the pixel coordinate system of the actual image and the feature points extracted from consecutive frame images of the tracking target, and determine whether the calculation result is less than a preset threshold. If so, the corresponding feature points are matched to achieve 3D-2D feature point dynamic coupling.
[0012] Further, in step S3, the ORB algorithm is used to obtain the feature points of the consecutive frame images of the tracking target in real time. In step S4, the E-PnP algorithm is used for pose estimation, and in step S5, the Levenberg-Marquardt algorithm is used for the correction measure.
[0013] Beneficial effects: 1. The present invention shows certain advantages in terms of real-time performance and robustness. In terms of time performance, it can reduce the computational cost of the processor and achieve real-time processing. In terms of registration accuracy, through the effective fusion of point cloud features and visual features, a deep improvement is proposed based on the Fast Point Feature Histogram (FPFH) algorithm. A two-stage neighborhood search strategy and a three-factor feature weighted fusion method are adopted, and combined with the nested loop mechanism of the pose estimation algorithm, the overall operation efficiency of the algorithm is improved, thereby reducing the computational cost and enhancing the accuracy and robustness of the overall matching. Generally speaking, the present invention shows good effects in multi-angle performance evaluations, verifying its practical application potential and technological advancement in complex scenarios.
[0014] 2. The present invention proposes a deep improvement based on the Fast Point Feature Histogram (FPFH) algorithm, constructs a two-stage neighborhood hierarchical search mechanism, and realizes a feature learning process from coarse to fine through a hierarchical local region feature extraction and global feature fusion strategy. Combined with feature weighted fusion technology, it effectively suppresses the interference of redundant information, enhances the expression ability of key features, reduces the computational amount by about 20% - 50% while ensuring the accuracy of feature extraction, and improves the robustness and expression ability of the extracted feature points to the local features of the three-dimensional point cloud. Description of the Drawings
[0015] In order to more clearly illustrate the technical solutions implemented by the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0016] Figure 1 It is a flowchart of the hydraulic pump part tracking and registration method based on model-image joint perception of the present invention; Figure 2 It is a framework diagram of the hydraulic pump part tracking and registration method based on model-image joint perception of the present invention; Figure 3 It is a schematic diagram of constructing a hydraulic pump part point cloud model and extracting feature points by using the structure from motion technology of the present invention; Figure 4 It is a system diagram of the pose data transmission of the hydraulic pump parts of the present invention. Detailed Embodiments
[0017] Next, in combination with the accompanying drawings in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0018] As Figures 1-4 shown, a method for tracking and registering hydraulic pump parts based on model image joint perception provided by the present invention is specifically as follows: S1. In the offline stage, collect a photo stream of hydraulic pump parts, obtain a point cloud model from the photo stream, process and optimize the point cloud model, and store it in the database as the original point cloud. In this embodiment, the Structure from Motion (SFM) technology is used to generate the point cloud model of the hydraulic pump parts, and at the same time, the data is transmitted to the data layer through the network layer to prepare for subsequent tracking and registration.
[0019] In this embodiment, to further illustrate step S1, the specific steps are as follows: S11. Use a multi-camera array to synchronously collect a photo stream of hydraulic pump parts, denoise the collected images, and use software such as Reality Capture to stitch an initial sparse point cloud model through the SFM technology; S12. With the help of professional 3D data processing software such as Meshlab, accurately identify and remove various types of noise points existing in the initial sparse point cloud model; S13. Optimize the point cloud density through the ICP algorithm, and store the optimized point cloud model in the database.
[0020] S2. In the online stage, use a calibrated camera to collect consecutive frame images of the tracking target in the hydraulic pump assembly environment, and construct a two-stage neighborhood hierarchical search mechanism and a three-factor feature weighted fusion based on the Fast Point Feature Histogram (FPFH) algorithm to extract the feature points of the original point cloud in the database; these feature points can be used in many key research links such as part identification, matching, 3D reconstruction, and performance analysis. At the hardware requirement level, only an ordinary RGB camera is required to complete the relevant tasks, reducing the benefit cost in the implementation process.
[0021] To effectively ensure the accuracy of 3D registration and accurately obtain the camera internal parameters and distortion parameters, first use the Zhang Zhengyou calibration method for calibration. It is a camera calibration method based on a planar checkerboard. Usually, only one planar calibration object (usually a checkerboard) needs to be used to take several images at different positions and poses for calibration.
[0022] In this embodiment, to further illustrate step S2, the specific steps are as follows: Most current point cloud feature extraction techniques rely on a single-stage adaptive radius search framework, resulting in a computational efficiency bottleneck. This embodiment proposes a deep improvement based on the Fast Point Feature Histogram (FPFH) algorithm, constructing a two-stage neighborhood hierarchical search mechanism: through a hierarchical local region feature extraction and global feature fusion strategy, an end-to-end feature learning process is realized; combined with feature weighted fusion technology, redundant information interference is effectively suppressed, and the key feature expression ability is enhanced. While ensuring the accuracy of feature extraction, the computational amount is reduced by about 20%-50%, improving the robustness and expression ability of the extracted feature points to the local features of the three-dimensional point cloud.
[0023] S21. Calculate the Simplified Point Feature Histogram (SPFH) of each point in the original point cloud within its first-stage neighborhood. The calculation formula is: , In the above formula, , represents the i-th point corresponding to the first-stage neighborhood, where represents the distance between the i-th point relative to the j-th point , is a distance threshold used to define which points belong to the neighborhood range; is an angle mapping function, where K represents the number of histogram bins, , are respectively the polar angle and azimuth angle on the unit vector of the line connecting the i-th point and the j-th point ; is a linear distance weight function, , and finally is taken. Each point in the original point cloud generates a 64-dimensional SPFH feature, that is, .
[0024] S22. Perform L1 normalization on the SPFH feature corresponding to each point to obtain the feature vector corresponding to this point. The calculation formula is: , In the above formula, represents the SPFH feature of the i-th point , represents the feature vector corresponding to the i-th point , and K represents the number of histogram bins.
[0025] S23. Further search for K nearest neighbor points for each point in the original point cloud within its second-stage neighborhood, and obtain the Fast Point Feature Histogram (FPFH) feature corresponding to each point through weighted fusion of its SPFH features , and the calculation formula is as follows: , In the above formula, , represents the m-th point corresponding to the second-stage neighborhood, The function represents finding the K nearest points with as the center; is the weight coefficient of the first-stage neighborhood feature; is the Gaussian distance weight function, where is the distance between the neighborhood point and the point , is the smoothing parameter; is the normalized SPFH feature of each neighbor point , and finally take , and each point generates a 64-dimensional FPFH feature, that is .
[0026] S24. Perform L1 normalization on the FPFH feature corresponding to each point to obtain the feature vector corresponding to this point, and the calculation formula is: , In the above formula, represents the FPFH feature value of the i-th point , represents the feature vector corresponding to the i-th point , and K represents the number of histogram bins.
[0027] S3. Perform 3D-2D feature point matching on the feature points extracted from the database and the feature points of the consecutive frame images of the tracking target obtained in real time using the ORB algorithm to achieve dynamic coupling; In this embodiment, to further illustrate step S3, the specific steps are as follows: S31. Convert the coordinate unit of each 3D point in the extracted feature points to ensure the accuracy and consistency of the calculation; assuming that the original point cloud coordinate unit is centimeters, and the subsequent calculation needs to use meters as the unit, then the unit conversion is achieved through simple numerical transformation operations. As shown below, let the original 3D point coordinate be , and the converted coordinate be , then the conversion formula is: , To perform the projection calculation of 3D points, the 3D point coordinates need to be converted into homogeneous coordinate form first; using the pre-initialized projection matrix, perform matrix multiplication with the 3D points in homogeneous coordinate form. Let the 3D point coordinates be , converted to homogeneous coordinates , and the formula for constructing homogeneous coordinates is: , where P is a coordinate matrix, is a homogeneous coordinate matrix.
[0028] S32. Project the 3D points in homogeneous coordinate form onto the 2D plane. Let the projection matrix be , and the projected coordinates be , then the projection operation formula is: , where is a projection matrix, is a coordinate matrix.
[0029] Since the third dimension of the projected coordinates contains depth information, when converting them into pixel coordinates of the 2D image plane, it is necessary to normalize the pixel coordinates projected onto the 2D plane. Let the projected coordinates be (n points, with shape (3,n)), and the normalized coordinates be (with shape (2,n)), and the normalization formula is: , S33. Map the normalized projected point coordinates to the pixel coordinate system of the actual image; perform coordinate transformation based on the focal length and image center coordinates in the camera intrinsics for subsequent matching and visualization processing with the feature points in the image. Let the focal length in the camera intrinsics be , , the image center coordinates be , , the image size be , convert the normalized coordinates to the coordinates in the pixel coordinate system, and the calculation formula is as follows: , S34. Calculate the Euclidean distance between the projection points mapped to the pixel coordinate system of the actual image and the feature points extracted from the consecutive frame images of the tracking target, and determine whether the calculation result is less than a preset threshold. If so, the corresponding feature points are matched to achieve 3D-2D feature point dynamic coupling. Let the coordinate of a certain projection point projected onto the 2D image plane be , and the coordinate of the ORB feature point extracted from the image be . The formula for calculating the Euclidean distance d between them is: , When , these two points are considered to be matched, where
[0030] represents the preset threshold.
[0031] In the present invention, nested loops are used to traverse the projected 2D feature points and the ORB feature points in the image, calculate the Euclidean distance between them. When the distance is less than the set threshold, these two points are considered a pair of matching points, and their index combinations are added to the list. S4. Update the target pose according to the matching situation, estimate the transformation relationship between the original point cloud in the database and the consecutive frame images of the tracking target obtained in real time, use the E-PnP pose estimation algorithm to update the external parameters (rotation and translation information) of the camera, and perform pose estimation of the tracking target object to achieve the effect of real-time tracking of the target object and updating its projected position in the image.
[0032] In the present invention, according to the result of feature matching in the previous text, the target pose is updated. The external parameters of the object relative to the camera are estimated through the PnP algorithm, and then the projection matrix is updated to reflect the real-time pose change of the object in the camera's field of view. A distance threshold for feature matching is set to determine whether the projected 3D points match the ORB feature points extracted from the 2D image. E-PnP is based on knowing the coordinates of multiple 3D space points in the world coordinate system and their corresponding projection coordinates on the 2D image plane (usually at least 5 pairs of matching points are required for effective pose estimation). When the number of matching point pairs is greater than or equal to 5, the corresponding 3D point coordinates and 2D image point coordinates are extracted from the matching results. When a sufficient number of matching point pairs are collected and the camera internal parameters are known, the pose (rotation and translation) of the object relative to the camera is solved.
[0033] In this embodiment, a series of equations are established based on the perspective projection model to solve the pose parameters: Assume a point in space, and its coordinate in the camera coordinate system is . The coordinate transformation between them is performed through the rotation matrix R and the translation vector t, as follows: , where R is a rotation matrix and t is a translation vector.
[0034] The camera intrinsic matrix K obtained by the above calibration projects the points in the camera coordinate system onto the 2D image plane to obtain pixel coordinates , and the projection relationship is as follows: , where S is a scale factor.
[0035] For each 3D-2D point pair , expanding the above projection relationship gives: , Substituting yields a system of linear equations , and the rotation matrix R and translation vector t are obtained using singular value decomposition (SVD).
[0036] S5. In case the tracking target is lost, corrective measures are introduced to optimize the estimated pose. The optimized pose data is transmitted to the Unity client using TCP (Transmission Control Protocol) for visual verification of hydraulic pump assembly. In the client interface, virtual-real fusion is performed based on the pose data for visual verification of hydraulic pump assembly, avoiding the accumulation of errors as the tracking registration time increases.
[0037] The specific corrective measures are as follows: The Levenberg-Marquardt algorithm is used to perform non-linear iterative optimization on the estimated pose, and finally a more accurate rotation matrix R and translation vector t are obtained. The reprojection error is defined by the following formula: , In the above formula, represents the process of projecting a 3D point onto the 2D image plane, and represent the abscissa and ordinate of the pixel obtained by projecting the point in the camera coordinate system onto the 2D image plane when estimating the pose parameters, respectively.
[0038] The new rotation matrix R and translation vector t reflect the latest pose of the object relative to the camera in the current frame image. Subsequently, it can continue to be used to project the points of the 3D model onto the next frame image plane for a new round of feature matching and pose update loop. The present invention transmits the optimized pose data to the Unity client, and virtual-real fusion is performed in the Unity Game interface based on the pose data for visual verification of hydraulic pump assembly.
[0039] The present invention designs a two-way communication system using the classic C / S architecture model. In this architecture, the role division is clear: the server acts as the data producer, responsible for data generation and transmission; the client acts as the receiver, responsible for the update processing of pose data. The server constructs an asynchronous I / O multiplexing mechanism based on the select module, which improves the concurrent processing ability of a hydraulic pump part tracking and registration method based on model image joint perception provided by the present invention, can manage multiple concurrent connections, ensure the fast and orderly transmission of data, and is enhanced by multimodal data, thereby enhancing the system's ability to cope with complex and changing environments.
[0040] In terms of protocol layer design, the present invention uses a fixed-length packet header (4 bytes) to identify the payload length, followed by the application data serialized by JSON. This design can, on the one hand, accurately ensure the boundary recognition of data frames and avoid confusion and misalignment during data transmission; on the other hand, the JSON-serialized data format also supports the flexible expansion of structured data, leaving sufficient space for future function expansion and data structure optimization.
[0041] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present invention. Any reference signs in the claims should not be regarded as limiting the claims involved.
[0042] In addition, it should be understood that although this specification is described according to embodiments, not every embodiment only contains an independent technical solution. This narrative way of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.
Claims
1. A method for tracking and registering hydraulic pump parts based on combined perception of model and image, characterized in that It includes the following steps: Offline stage: S1. Collect the photo stream of hydraulic pump parts, obtain the point cloud model from the photo stream, denoise the point cloud model and optimize the point cloud density, and then store it in the database as the original point cloud; Online stage: S2. Use the calibrated camera to collect the continuous frame images of the tracking target in the hydraulic pump assembly environment in real time, and construct a two-stage neighborhood hierarchical search mechanism and three-factor feature weighted fusion based on the fast point feature histogram algorithm to extract the feature points of the original point cloud in the database; S3. Perform 3D-2D feature point matching on the feature points extracted from the database and the feature points of the continuous frame images of the tracking target obtained in real time to achieve dynamic coupling; S4. Update the target pose according to the matching situation, estimate the transformation relationship between the original point cloud in the database and the continuous frame images of the tracking target obtained in real time, and use the pose estimation algorithm to update the external parameters of the camera rotation and translation information to perform the pose estimation of the tracking target object; S5. Introduce corrective measures to optimize the estimated pose, and use TCP to transmit the optimized pose data to the client for visual verification of hydraulic pump assembly.
2. The hydraulic pump part tracking and registration method according to claim 1, wherein, The specific steps of step S2 are as follows: S21. Calculate the simplified point feature histogram (SPFH) of each point in the original point cloud within its first-stage neighborhood; S22. For the SPFH feature corresponding to each point perform L1 normalization to obtain the feature vector corresponding to this point ; S23. Further search for K nearest neighbor points for each point in the original point cloud within its second-stage neighborhood, and obtain the fast point feature histogram (FPFH) feature corresponding to each point by weighted fusion of its SPFH features , and obtain the fast point feature histogram (FPFH) feature corresponding to each point; S24. For the FPFH feature corresponding to each point perform L1 normalization to obtain the feature vector corresponding to this point .
3. The hydraulic pump part tracking and registration method according to claim 2, characterized in that: The calculation formula of step S21 is: , In the above formula, , represents the first-stage neighborhood corresponding to the i-th point, where represents the distance of the i-th point relative to the j-th point , and is a distance threshold; is an angle mapping function, where K represents the number of histogram bins, , and are respectively the polar angle and azimuth angle on the unit vector of the line connecting the i-th point and the j-th point ; is a linear distance weight function, , , and indicates that the i-th point generates a K-dimensional SPFH feature.
4. The hydraulic pump part tracking and registration method according to claim 2, characterized in that: The calculation formula of step S22 is: , In the above formula, represents the SPFH feature of the i-th point , represents the feature vector corresponding to the i-th point , and K represents the number of histogram bins.
5. The method for tracking and registering hydraulic pump parts according to claim 2, wherein: The calculation formula of step S23 is: , In the above formula, , represents the second-stage neighborhood corresponding to the m-th point , The function means to find the K points closest to the point centered at ; is the weight coefficient of the first-stage neighborhood feature; is the Gaussian distance weight function, where is the distance between the neighborhood point and the point , is the smoothing parameter; is the normalized SPFH feature of each neighbor point , represents the i-th point generating the K-dimensional FPFH feature.
6. The hydraulic pump part tracking and registration method according to claim 2, characterized in that: The calculation formula of step S24 is: , In the above formula, represents the FPFH eigenvalue of the i-th point , represents the eigenvector corresponding to the i-th point , and K represents the number of histogram bins.
7. The method for tracking and registering hydraulic pump parts according to claim 1, characterized in that: The specific steps of step S3 are as follows: S31. Convert the coordinate unit of each 3D point in the extracted feature points and then convert it into the homogeneous coordinate form; S32. Project the 3D points in the homogeneous coordinate form onto the 2D plane, and normalize the pixel coordinates projected onto the 2D plane; S33. Map the normalized projected point coordinates to the pixel coordinate system of the actual image, and perform coordinate transformation according to the focal length and image center coordinates in the camera internal parameters; S34. Calculate the Euclidean distance between the projected points mapped to the pixel coordinate system of the actual image and the feature points extracted from the continuous frame images of the tracking target, and judge whether the calculation result is less than the preset threshold. If so, the corresponding feature points are matched to achieve 3D-2D feature point dynamic coupling.
8. The hydraulic pump part tracking and registration method according to claim 1, characterized in that: In step S3, the feature points of the continuous frame images of the tracking target obtained in real time are obtained by using the ORB algorithm. The pose estimation algorithm in step S4 uses E-PnP, and the corrective measure in step S5 uses the Levenberg-Marquardt algorithm.
Citation Information
Patent Citations
Virtual-real registration method based on point cloud and visual feature fusion
CN109960402A
Camera real-time tracking registration method
CN110059651A
Oil filter target recognition and positioning method based on machine version
CN110378376A
Photovoltaic power station operation and maintenance method and system combining three-dimensional surveying and mapping and digital twinning
CN118736444A