An object pose estimation method and device based on a three-dimensional point cloud and a storage medium

Abstract

The application relates to the field of computer vision, and in particular to a three-dimensional point cloud-based object pose estimation method, device and storage medium. The method comprises the following steps: obtaining a plurality of first points in a template point cloud, and collecting a plurality of second points in a scanned point cloud of an object; calculating a feature descriptor of each first point and second point; pairing the first points and the second points based on the similarity and difference filtering threshold between the feature descriptors, to obtain a matching point pair set; building a displacement space graph according to the position information of the sum of any two matching point pairs, and extracting a plurality of feature description groups from the graph based on a maximum group search algorithm; and selecting a target feature description group from the plurality of feature description groups to perform registration, so as to determine the corresponding object pose. While improving the data quality, the power and time requirements of the subsequent stage are reduced, so that the speed and accuracy of the registration are taken into account.

Description

Technical Field

[0001] This application relates to the field of computer vision, and in particular to a method, device and storage medium for object pose estimation based on 3D point clouds. Background Technology

[0002] 3D point cloud registration and pose estimation are fundamental problems in many cutting-edge technology fields such as computer vision, robot perception, and industrial automation. The core objective is to align 3D point clouds acquired from different times, different viewpoints, or different sensors into the same unified coordinate system through rigid body transformations (i.e., rotation and translation), and to accurately estimate the relative pose between sensors or objects in the process.

[0003] With the rapid development and widespread application of depth cameras, LiDAR, and structured light scanning equipment, acquiring high-quality 3D point clouds has become increasingly convenient, placing higher demands on the accuracy, robustness, and efficiency of point cloud registration algorithms.

[0004] For example, the invention patent application with publication number CN118628542A discloses a maximal clique 3D point cloud registration method that introduces prior knowledge of overlapping regions. First, the input point cloud is preprocessed and local features are extracted, and an initial correspondence is formed through feature matching. Then, the geometric compatibility relationship between the initial matches is described using graph space. Next, the maximal clique is searched in the compatibility graph to generate a six-degree-of-freedom pose transformation hypothesis. Finally, all the generated hypotheses are verified and the optimal one is selected for registration.

[0005] For example, patent application CN118729988A discloses a method for precision measurement of workpieces based on binocular structured light. By mounting the imaging device on the production line stage, interference-free and stable imaging of the workpiece can be achieved, greatly improving inspection efficiency and accuracy. This automated inspection method not only reduces manual operation but also enables 24-hour uninterrupted operation, significantly improving production efficiency and product quality.

[0006] For example, patent application CN119151999A discloses a method based on the maximal clique "hypothesis". The robust 3D point cloud registration method of "verification" first preprocesses the input point cloud and extracts local features, and forms an initial correspondence through feature matching; then, the initial matching is modeled as a compatibility map and the maximal clique in the map is searched; next, a maximal clique pool is constructed using seed matching to generate six-DOF pose hypotheses; finally, all generated hypotheses are verified in a stepwise manner and the optimal one is selected for registration.

[0007] However, current point cloud registration methods struggle to balance speed and accuracy. High-precision algorithms typically have high computational complexity and overhead, making it difficult to meet real-time requirements. On the other hand, fast registration, which prioritizes efficiency, often leads to registration errors due to neglecting geometric details, resulting in low practicality. Summary of the Invention

[0008] The main objective of this application is to provide a method, device, and storage medium for object pose estimation based on 3D point clouds. To address the aforementioned technical problems, this application specifically adopts the following technical solution:

[0009] A first aspect of this application is to provide a method for estimating the pose of an object based on a 3D point cloud, the method comprising:

[0010] S1, acquire template point cloud and collect scanned point cloud of object, wherein the template point cloud includes several Class I points and the scanned point cloud includes several Class II points;

[0011] S2, calculate the feature descriptor for each of the first-class points and the second-class points; wherein, the feature descriptor is used to represent the vector or histogram of local geometric features or information of the point cloud;

[0012] S3, based on the similarity between the feature descriptors, pair the first-class points and the second-class points to obtain a set of matching point pairs. ,in, and These are the first-class and second-class points that were successfully matched, respectively.

[0013] S4, based on any two matching point pairs and The location information is used to calculate the first type of positional relationship between two matching point pairs in the template point cloud and the second type of positional relationship in the scanned point cloud; if the first type of positional relationship and the second type of positional relationship meet the difference filtering threshold, then the two matching point pairs are associated.

[0014] S5. Construct a displacement space graph based on all established relationships and corresponding matching point pairs, and extract several feature description cliques from the graph based on the maximal clique search algorithm; wherein, the displacement space graph is an undirected graph structure; the maximal clique search algorithm is used to find subgraphs in the undirected graph structure where any two points are connected as feature description cliques;

[0015] S6. Select a target feature description group from a plurality of the aforementioned feature description groups, and perform registration based on the first-class points and second-class points in the target feature description group to determine the corresponding object pose.

[0016] In some embodiments, the method further includes: during the process of acquiring the scanning point cloud of an object, performing real-time analysis of the current scanning scene using a preset target recognition algorithm to determine the target area where the object is located in three-dimensional space; and using the point cloud within the target area as the scanning point cloud.

[0017] In some embodiments, before step S2, the method further includes: downsampling the scanned point cloud based on a preset voxel size.

[0018] In some embodiments, S3 further includes: determining each binary point based on a nearest neighbor search algorithm. The closest type of points in the template point cloud and each Class I point The s closest target binary points in the scanned point cloud Where s is an integer greater than or equal to 1; the two types of points With the corresponding target type points Pairing is performed to obtain an initial set of matching point pairs: The initial set of matching point pairs As a set of matching point pairs Alternatively, for the initial set of matching point pairs... Filtering is performed to obtain a set of matching point pairs. This includes: if the initial matching point pair contains two types of points For target class one points Target Type II points If so, then the initial matching point pair is retained.

[0019] In some embodiments, S4 includes: obtaining any two matching point pairs. and According to the matching point pair and Location information, calculation and A type of positional relationship between According to the matching point pair and Location information, calculation and The two types of positional relationships between ;when and satisfy At that time, and Add an edge between ;in, This is the threshold for difference filtering.

[0020] In some embodiments, after step S5, the method further includes: randomly selecting at least three verification matching point pairs from the feature description group; for any two verification matching point pairs... and Perform the following steps respectively: based on the matching point pair and Location information calculation and A type of side length between According to the matching point pair and Location information calculation and The second type of side length between Calculate the first-class edge length between any two pairs of verification matching points. and the second type of side length The ratio of the side lengths between When the ratio of the side lengths between the verified matching point pairs all satisfy... At the same time, the verification matching point pairs are retained to update the set of matching point pairs in the feature description clique; wherein, The threshold value is the side length.

[0021] In some embodiments, S6 further includes: S61, selecting the feature descriptor group with the largest number of current point clouds that has not participated in registration as the target feature descriptor group; S62, counting the number of point clouds that have been successfully registered globally under the object pose; if the number of point clouds reaches the number filtering threshold, then outputting the object pose as the target object pose; S63, if the number of point clouds does not reach the number filtering threshold, or if the duration of a single registration is greater than a first duration threshold, then reselecting the target feature descriptor group.

[0022] In some embodiments, the reselection of the target feature description group includes: returning to the execution step S61; or, performing geometric fitting on several point clouds in each feature description group to obtain the corresponding axis; obtaining the target axis of the current target feature description group; and selecting, from the feature description groups whose point cloud quantity is greater than or equal to the lower limit threshold, a feature description group whose deviation between the axis and the target axis is greater than a preset deviation threshold as the target feature description group.

[0023] A second aspect of this application is to provide a computer device, the device comprising:

[0024] Memory, used to store computer programs;

[0025] A processor is configured to execute the computer program and, in executing the computer program, implement the steps of the object pose estimation method based on three-dimensional point clouds as provided in any embodiment of this application.

[0026] A third aspect of this application is that a computer-readable storage medium is also provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the object pose estimation method based on three-dimensional point clouds provided in any embodiment of this application.

[0027] Beneficial effects:

[0028] This application provides a method, device, and storage medium for object pose estimation based on 3D point clouds. Through a multi-stage point cloud screening and verification mechanism, it achieves progressive data simplification, improving data quality while reducing the computing power and time requirements of subsequent stages, thus balancing the speed and accuracy of registration. It is particularly suitable for industrial scenarios with large point cloud scales or high robustness requirements.

[0029] During the data input stage, target region cropping and voxel downsampling are introduced to significantly reduce the amount of data for subsequent processing, providing lightweight and information-complete point cloud input for subsequent processes. Object recognition algorithms locate the region where objects are located, retaining only relevant point clouds and eliminating interference from irrelevant background point clouds. This achieves a significant reduction in data volume at the source, making it particularly suitable for object localization in complex scenes. Simultaneously, a preset voxel size is used to downsample the scanned point cloud, reducing point cloud density with controllable loss of detail.

[0030] In the initial matching phase, after preliminary pairing based on feature descriptors, a bidirectional consistency verification mechanism is introduced. This means that a match is considered valid only if the points of class I and class II in a matched pair are within each other's nearest neighbors. This filters out incorrectly matched point pairs caused by feature ambiguity due to downsampling, improving the quality of the matched point pair set. Furthermore, the nearest neighbor domain can retain a moderate margin of error, ensuring that subsequent steps have sufficient and reliable candidate data.

[0031] During the feature clique generation stage, a standard for establishing associations between matching point pairs in the displacement space graph is set based on a difference filtering threshold, thereby adjusting the candidate range for maximal clique search. Matching point pairs that have not established any associations are directly eliminated and no longer participate in subsequent calculations. Thanks to data simplification in the early stages (pruning, downsampling, and bidirectional consistency verification have significantly reduced the number of nodes), the number of graph nodes is significantly reduced, making maximal clique search, which originally had combinatorial explosion properties, efficient and feasible in practice.

[0032] To further improve the quality within the feature description clique, a three-point consistency verification mechanism is introduced. When the point cloud scale is large and the downsampling degree is high, the accuracy of the feature descriptors is limited, requiring a relaxation of the difference filtering threshold to provide sufficient data for subsequent steps. This can also lead to incorrect matching point pairs being mixed into the clique. In this case, by checking whether the side length ratio of any three point pairs falls within a reasonable range, such abnormal point pairs can be effectively removed. Since the verification object is only specific point pairs within the clique, the computational cost is controllable; and in high-precision, small-scale point cloud scenarios, this step can be skipped, exhibiting good adaptability.

[0033] During the pose determination phase, registration is performed using only a single feature description clique, resulting in a significant reduction in computational burden. The validity of the result is assessed by combining the number of successfully matched point logs. If invalid, a new clique can be selected based on a dual criterion of the number of nodes within the clique and geometric differences, considering both the size and structural differences of the clique to avoid getting trapped in local optima. Because the amount of data for a single registration is relatively small, the speed is greatly improved, allowing for efficient attempts at multiple cliques and increasing the overall success rate. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of this application; for those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0035] Figure 1 This is a schematic flowchart of an object pose estimation method based on three-dimensional point clouds provided in an embodiment of this application;

[0036] Figure 2 This is a schematic diagram of a feature description group provided in an embodiment of this application;

[0037] Figure 3 This is a schematic diagram of a target area where an object is located, provided in an embodiment of this application;

[0038] Figure 4 This is a schematic diagram illustrating a coordinate system relationship provided in an embodiment of this application;

[0039] Figure 5 This is a schematic diagram of the overall process of three-dimensional point cloud pose estimation provided in an embodiment of this application;

[0040] Figure 6 This is a schematic block diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0042] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0043] In this document, suffixes such as “module,” “part,” or “unit” used to denote elements are used only for illustrative purposes and have no specific meaning in themselves. Therefore, “module,” “part,” or “unit” may be used interchangeably.

[0044] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0045] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," and "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0046] In this document, the term “and / or” includes any and all combinations of one or more of the listed related items.

[0047] In this article, the term "multiple" means two or more, that is, it includes two, three, four, five, etc.

[0048] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0049] This application provides a method, device, and storage medium for object pose estimation based on 3D point clouds. Through a multi-stage point cloud screening and verification mechanism, it achieves progressive data simplification, improving data quality while reducing the computing power and time requirements of subsequent stages, thus balancing the speed and accuracy of registration. It is particularly suitable for industrial scenarios with large point cloud scales or high robustness requirements.

[0050] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0051] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating an object pose estimation method based on 3D point clouds provided in an embodiment of this application, such as... Figure 1 As shown in the figure, this application provides a method for estimating object pose based on three-dimensional point clouds.

[0052] S1, acquire template point cloud and collect scanned point cloud of object, wherein the template point cloud includes several Class I points and the scanned point cloud includes several Class II points.

[0053] The template point cloud is a complete 3D point cloud of the target object in the standard coordinate system that has been acquired and stored in advance, and is used to provide a reference benchmark in pose estimation.

[0054] Among them, the scanned point cloud is a real-time 3D point cloud that is actually collected, including the target object, reflecting the state of the object in its current unknown and random pose and environment.

[0055] S2, calculate the feature descriptor for each of the first-class points and the second-class points.

[0056] Among them, the feature descriptors of points in the point cloud are vectors or histograms used to represent the local geometric features or information of the point cloud. For specific calculation methods, please refer to relevant technologies, such as SIFT (Scale Invariant Feature Transform), SURF (Accelerated Robust Feature Transform), FPFH (Fast Point Feature Histogram), SHOT (Signature Histogram), etc.

[0057] Specifically, for each Class I and Class II point, geometric information (such as normal vectors, curvature, relative positions, etc.) of surrounding points is collected within a neighborhood of a preset radius centered on the point itself, and encoded into a numerical vector of fixed dimension. The feature descriptors for Class I points can be pre-generated and stored.

[0058] S3, based on the similarity between the feature descriptors, pair the first-class points and the second-class points to obtain a set of matching point pairs. ,in, and These are the first-class and second-class points that were successfully matched, respectively.

[0059] Specifically, for each binary point in the scanned point cloud, its feature descriptor is compared with the feature descriptors of all binary points, and the similarity is evaluated using either Euclidean distance or cosine distance. The binary point with the highest similarity is selected to form the initial matching point pair. ,in, From the template point cloud, these are points of a successfully matched class. , From the scanned point cloud, these are successfully matched Class II points. A set of matching point pairs (also known as point pairs or point groups) consists of multiple matching point pairs. .

[0060] S4, based on any two matching point pairs and The location information is used to calculate the first type of positional relationship between two matching point pairs in the template point cloud and the second type of positional relationship in the scanned point cloud; if the first type of positional relationship and the second type of positional relationship meet the difference filtering threshold, then the two matching point pairs are associated.

[0061] Specifically, for the set of matching point pairs Multiple matching point pairs are subjected to bidirectional consistency verification. For any two matching point pairs, a first-class positional relationship and a second-class positional relationship are calculated. The first-class positional relationship refers to the geometric relationship between two first-class points in the template point cloud; correspondingly, the second-class positional relationship refers to the geometric relationship between two second-class points in the scanned point cloud. For example, the positional relationship can be represented by a relative displacement vector or Euclidean distance.

[0062] Compare the positional differences between the first-class and second-class positional relationships. If the positional difference is less than or equal to the preset difference filtering threshold, the two matching point pairs are determined to be consistent in geometric structure, and an association is established between them; otherwise, they are determined not to be consistent, and no association is established.

[0063] The difference filtering threshold is the maximum allowed positional difference between the Class I and Class II positional relationships of two matched point pairs. For example, it can be set to 2–5 mm in millimeter-level accuracy scenarios. It can also be adaptively adjusted according to the point cloud density and the voxel size used in downsampling to effectively suppress noise interference while preserving sufficiently correct matches.

[0064] It should be understood that since rigid body transformation preserves the distance and relative orientation between points, ideally the two types of positional relationships should be completely consistent (or differ by only one global rotation). Therefore, the distance between any two matching point pairs in the template point cloud should be consistent with or within a certain range as the distance between the corresponding points in the scanned point cloud.

[0065] S5. Based on all established relationships and corresponding matching point pairs, construct a displacement space graph and extract several feature description cliques from the graph using a maximal clique search algorithm.

[0066] The displacement space graph is an undirected graph structure, consisting of multiple nodes and edges connecting them. Each node corresponds to a pair of matching points, and each node can be a pair of matching points. It can also be used for matching point pairs. or Correspondingly, each edge can connect two pairs of matched points that have an association relationship, or it can connect one of the matched points in a pair. or The length of the edge can be the positional difference between the first-class positional relationship and the second-class positional relationship.

[0067] The maximal clique search algorithm is an existing algorithm used to find the largest complete subgraph (i.e., a subgraph in which any two points are connected) in an undirected graph structure. The maximal clique search algorithm can identify the set of mutually consistent maximum matching point pairs in a displacement space graph. Each maximal clique constitutes a feature description clique, representing a set of highly consistent matching point pairs that support the same rigid body transformation.

[0068] Depending on the node definition, feature descriptor clusters can be formed independently in template point clouds and / or scanned point clouds. (See [link to relevant documentation]). Figure 2 , Figure 2 This is a schematic diagram of a feature descriptor group provided in an embodiment of this application, such as... Figure 2 As shown, the elliptical region M on the left represents a template point cloud consisting of several points. The partial feature description cluster is formed, and the elliptical region N on the right represents a group of points in the scanned point cloud. The feature description groups are partially formed, and the feature description groups on both sides are consistent. There is a one-to-one correspondence between the feature description groups in the template point cloud and the scanned point cloud, as shown by the dashed line connection. Feature description group D1 corresponds to feature description group D1′, and feature description group D2 corresponds to feature description group D2′, so as to be used for subsequent cross-point cloud structure alignment and pose estimation.

[0069] S6. Select a target feature description group from a plurality of the aforementioned feature description groups, and perform registration based on the first-class points and second-class points in the target feature description group to determine the corresponding object pose.

[0070] Specifically, after extracting several feature descriptor groups, one of them is selected as the target feature descriptor group for pose solving. After selecting the target feature descriptor group, all matching point pairs within it are used as input. A preset registration algorithm is used to align the first type of points in the template point cloud to the corresponding second type of points in the scanned point cloud using the required rotation and translation parameters, thereby obtaining the currently estimated object pose.

[0071] For example, the clique containing the largest number of matching point clouds is preferentially selected, as it typically has stronger geometric support and higher statistical reliability. For example, the clique with the most uniform distribution of point pairs and the widest coverage of the object surface is preferentially selected. For example, the clique with the smallest sum of geometric consistency errors within the clique is selected. For example, a non-empty clique is randomly selected to enable parallel attempts of multiple hypotheses.

[0072] In this embodiment, a first round of data simplification is performed during the data input stage. By cropping the target region and downsampling voxels, the amount of data for subsequent processing is greatly reduced, providing a lightweight and complete point cloud input for subsequent processes.

[0073] In some embodiments, to improve processing efficiency and reduce interference from irrelevant data, the method further includes: during the process of collecting the scanning point cloud of an object, performing real-time analysis of the current scanning scene using a preset target recognition algorithm to determine the target area where the object is located in three-dimensional space; and using the point cloud within the target area as the scanning point cloud.

[0074] Among them, the object recognition algorithm is used to automatically detect and locate objects of a specific category from a 3D scene point cloud. The output can include the spatial range of the object's location, such as a 3D bounding box (also known as a detection box). Specifically, it can be a deep learning-based 3D object detection model (such as VoteNet), or a traditional template matching or geometric rule method.

[0075] Specifically, while acquiring and scanning point clouds, a pre-defined target recognition algorithm is integrated to perform real-time analysis of the current scene. This allows for the rapid localization of the approximate spatial range of the object under test from the complete scene point cloud. Only the point cloud within this 3D bounding box (or semantic segmentation mask) is extracted for subsequent processing, while point clouds from the background, other objects, or empty areas are directly removed. It should be understood that this cropping operation significantly reduces the total amount of point cloud data, accelerates subsequent processing, and reduces noise interference. This is particularly suitable for applications with complex backgrounds but clearly defined targets, such as industrial production lines and warehousing logistics.

[0076] For example, the target recognition algorithm is a deep learning recognition algorithm. It obtains the coordinates of the detection box and, based on the camera intrinsic parameters K and distortion parameters D, maps the detection box region from pixel space to actual phase space, achieving accurate extraction of the object region. Phase matching is performed only within the target object region, thereby reducing the matching of irrelevant regions. It should be understood that by locating the region where the object is located through the target recognition algorithm, only relevant point clouds are retained, and the interference of irrelevant background point clouds is eliminated, achieving a significant reduction in data volume at the source, which is particularly suitable for object localization in complex scenes.

[0077] In some embodiments, a projector projects a series of fringe lights and Gray code fringe lights with phase shift relationships onto the surface of a target object. The fringe lights are ideal projection fringes generated by the six-step phase shift method using a general N-step phase shift formula. Multiple phase-shifted fringe images are obtained by capturing images of each fringe light illumination using a binocular camera. It should be understood that due to differences in height and reflectivity, deformed phase-shifted fringes modulated by the object's surface information can be captured by the camera. The phase-shifted fringe images are first epipolarized and distortion-removed using binocular positioning parameters to simplify subsequent stereo matching, allowing the search to be performed only in the horizontal direction. The wrapping phase of the phase-shifted fringe images is calculated based on the six-step phase shift method. The monotonically increasing absolute phase is obtained through a time-domain phase unpacking algorithm to achieve phase unpacking and obtain a phase unfolded diagram.

[0078] Please see Figure 3 , Figure 3 This is a schematic diagram of a target area where an object is located, provided in an embodiment of this application. Figure 3 As shown, while the camera is capturing images, a pre-set target recognition algorithm runs in parallel, taking the full white light image e under full white light projection as input. The pre-set target recognition algorithm then obtains the target detection bounding box. The location width data is output, where x and y are the horizontal and vertical coordinates of the target detection box in the image pixel coordinate system before correction, respectively, and w and h are the horizontal length and vertical height of the target detection box before correction, respectively. These four variables can describe a regular quadrilateral detection box. Since the image will be deformed after correction, the detection box also needs to undergo epipolar correction and distortion correction processes to update the target detection box, and only the image within the target detection box on the absolute phase unfolded map is retained for subsequent calculations.

[0079] Based on the coordinates of the edge points of the detection box, the phase-decapsulated image (e.g.) Figure 3 The first phase unfolded image (f) is cropped to obtain unpacked images of the target object regions from the left and right cameras that contain only absolute phase information, as shown below. Figure 3 The second phase unfolded diagram g is shown. Under epipolar constraints, pixels with the same absolute phase value between the left and right cameras are searched in the same row as corresponding points. Corresponding points are matched according to the absolute phase value within the same row. After calculating the normalized coordinates, the depth information of corresponding points is calculated using the camera intrinsic and extrinsic parameters based on the principle of binocular ranging. The x and y coordinates of the pixel coordinate system points in the camera coordinate system are calculated according to the camera model. The original point cloud data of the target object region (i.e., the scanned point cloud) is obtained by combining the depth information and the camera extrinsic parameters with the world coordinate system as the reference.

[0080] In some embodiments, a template point cloud of the target object is constructed. The hardware includes a scanner and a motorized turntable, whose relative positions are known and remain stationary during the scanning process. The target object is placed on the turntable, and the angle rotated by the turntable is recorded during each frame of scanning. After a week of scanning, the first frame image and the angle of the turntable are used. For reference, the turntable angle corresponding to each subsequent frame of data is used as... Prior data is used to adjust the point cloud pose. Assuming the turntable is stationary, the camera coordinate system is rotated in the opposite direction by the corresponding turntable angle. Therefore, a good initial pose can be obtained without coarse registration, and fine registration can be performed directly, thus ensuring the success rate of the multi-frame point cloud registration process and obtaining the complete point cloud of the object.

[0081] Move the complete point cloud to the origin of the world coordinate system, and set the reference pose by rotation to obtain a template point cloud with the world coordinate system as the origin and a custom reference pose.

[0082] Please see Figure 4 , Figure 4 This is a schematic diagram illustrating a coordinate system relationship provided in an embodiment of this application, as shown below. Figure 4As shown, the spatial relationships between the world coordinate system, the left camera coordinate system, the right camera coordinate system, and the pixel coordinate system are illustrated. Specifically, the spatial relationship diagram u between the world coordinate system, the left camera coordinate system, and the right camera coordinate system, and the spatial relationship diagram v between the left camera coordinate system and the pixel coordinate system are shown. The coordinate system of the template point cloud is the world coordinate system, and the coordinate system of the reconstructed scan point cloud is the left camera coordinate system. In order to obtain the correct pose transformation relationship, it is necessary to unify them to the world coordinate system through coordinate transformation to achieve alignment with the template point cloud.

[0083] In some embodiments, before step S2, the method further includes: downsampling the scanned point cloud based on a preset voxel size.

[0084] In this context, a voxel refers to a cubic mesh in three-dimensional space, while the voxel size is the side length of the mesh and a key parameter controlling the granularity of downsampling. A larger voxel size results in a higher degree of downsampling, smaller data volume, and faster speed, but also greater loss of detail.

[0085] Specifically, a regular cubic mesh of preset voxel size is used to traverse all point clouds. For multiple points falling within the same mesh, only one representative point (such as the first point, the center point, or the average point) is retained, and the rest are discarded, achieving downsampling. Thus, the original dense point cloud is simplified into a sparser point cloud with a more sparse distribution but with its geometric structure largely preserved, effectively improving the computational efficiency of subsequent processes. It should be understood that downsampling the scanned point cloud using a preset voxel size reduces the point cloud density while controlling the loss of detail.

[0086] In some embodiments, the raw point cloud data is clustered to filter out small-scale irrelevant point clouds and noise within the focal length range.

[0087] In some embodiments, the range of clustering and downsampling parameter selection is defined by the original point cloud size in order to achieve a balance between efficiency and information preservation.

[0088] It should be understood that due to the high precision of structured light scanning, the resulting surface point cloud is enormous. However, the pose estimation required for registration relies solely on preserving basic geometric features and details sufficient for accurate calculation of surface normals. Directly using the raw point cloud would introduce a large amount of redundant data, significantly reducing efficiency. Therefore, during the data input stage, preprocessing of the point cloud, such as target region cropping, voxel downsampling, and cluster filtering, significantly reduces the amount of data required for subsequent processing, providing a lightweight and information-complete point cloud input for later workflows.

[0089] In this embodiment, a second round of data simplification is performed during the initial matching stage. Through a bidirectional consistency verification mechanism, point pairs that are incorrectly matched due to feature ambiguity caused by downsampling are filtered out, thereby improving data quality.

[0090] In some embodiments, S3 further includes: determining each binary point based on a nearest neighbor search algorithm. The closest type of points in the template point cloud and each Class I point The s closest target binary points in the scanned point cloud Where s is an integer greater than or equal to 1; the two types of points With the corresponding target type points Pairing is performed to obtain an initial set of matching point pairs: .

[0091] Specifically, a k-dimensional tree is built for the feature descriptors and a nearest neighbor search is performed to obtain the set of point pairs with the highest similarity between the template point cloud and the scanned point cloud. In some embodiments, the initial set of matching point pairs is... As a set of matching point pairs .

[0092] In some embodiments, S3 further includes: processing the initial set of matching point pairs. Filtering is performed to obtain a set of matching point pairs. This includes: if the initial matching point pair contains two types of points For target class one points Target Type II points If so, then the initial matching point pair is retained.

[0093] Specifically, a bidirectional consistency check is performed on the point pairs. When a certain type of point in template point cloud A is in the point pair set... The nearest neighbor is a class II point in the scanned point cloud B. And the first class of points in template point cloud A Type II points in scanned point cloud B When the first s nearest neighbors are found, the matching point pair is retained, that is, the pair that satisfies the condition is retained. and The combination of N point pairs yields .

[0094] KD-trees are binary space partitioning trees used to organize point data in k-dimensional space. They support efficient nearest neighbor search operations and are widely used in point cloud processing.

[0095] It should be understood that in the bidirectional consistency verification mechanism, a match is considered valid only if the points of type I and type II in a matched point pair are within each other's nearest neighbor region. Point clouds corresponding to matched point pairs that fail the bidirectional consistency verification are then filtered out. Furthermore, the nearest neighbor region can retain a moderate margin of error, ensuring that subsequent steps have sufficient and reliable candidate data.

[0096] In this embodiment, a third round of data simplification is performed during the feature clique generation stage. A standard for establishing associations between matching point pairs in the displacement space graph is set based on a difference filtering threshold. Matching point pairs without any association are directly eliminated and no longer participate in subsequent calculations, effectively eliminating noisy matches and incorrect pairings, thus improving the speed and accuracy of subsequent pose estimation. Furthermore, thanks to the data simplification in the earlier stages (point cloud pruning, downsampling, and bidirectional consistency verification significantly reduce the number of nodes), the number of graph nodes is significantly reduced, making the maximal clique search, which originally exhibited combinatorial explosion characteristics, efficient and feasible in practice.

[0097] In some embodiments, S4 includes: obtaining any two matching point pairs. and According to the matching point pair and Location information, calculation and A type of positional relationship between According to the matching point pair and Location information, calculation and The two types of positional relationships between ;when and satisfy At that time, and Add an edge between ;in, This is the threshold for difference filtering.

[0098] Specifically, for Given N sets of matching point pairs, obtain any two distinct matching point pairs. and ,in, and It is a point in the point pair that belongs to template point cloud A. and Given points belonging to the scanned point cloud B in a point pair, calculate the displacement vector between Class I and Class II points. ,and Not equal to ,when and satisfy At the time, in the matching point pair and Establish a relationship between them, correspondingly, For matching point pairs The recorded relationships are used to construct the displacement space map. Furthermore, when each node of the displacement space graph is a matching point pair... (like , )hour, This can be transformed into edges connecting nodes in the graph. The maximal clique search algorithm can be based on the edges Determine whether any two Class I points in an undirected graph structure are connected, and then extract the feature descriptor clique.

[0099] In some embodiments, the differential filtering threshold is affected by voxel size and can be set to a value corresponding to 2 to 3 times the voxel size.

[0100] In some embodiments, a maximal clique search algorithm is used to extract a set of local consistency from the displacement space graph G. That is, the feature description group. For example, It can be , It can be .

[0101] It should be understood that step S4 can iterate through all possible combinations of matching point pairs, establish associations between matching point pairs and proceed to subsequent steps, and filter out invalid matching point pairs that fail to establish associations with any other matching point pairs.

[0102] In this embodiment, a fourth round of data simplification is performed after the feature cluster is generated. A three-point consistency verification mechanism is introduced within the feature description cluster to filter out point pairs that fail the verification. Since the verification object is only a specific pair of points within the cluster, the computational cost is controllable. In high-precision, small-scale point cloud scenarios, this step can be skipped, demonstrating good adaptability.

[0103] In some embodiments, a three-point consistency verification is performed on each feature descriptor group. After S5, the method further includes: randomly selecting at least three verification matching point pairs from the feature descriptor group; for any two verification matching point pairs... and Perform the following steps respectively: based on the matching point pair and Location information calculation and A type of side length between According to the matching point pair and Location information calculation and The second type of side length between Calculate the first-class edge length between any two pairs of verification matching points. and the second type of side length The ratio of the side lengths between When the ratio of the side lengths between the verified matching point pairs all satisfy... At the same time, the verification matching point pairs are retained to update the set of matching point pairs in the feature description clique; wherein, The threshold value is the side length.

[0104] For example, for feature description clusters containing no fewer than three point clouds, at least three point groups are randomly selected. Calculate the side lengths of the triangles containing the three point groups in the scanned point cloud and the template point cloud, respectively. and For any two distinct pairs of points, the side length between each pair can be calculated separately, for example, for , for ;or for , for ;or, for , for Specifically, this includes: calculating the lengths of multiple sides of the same type. : , , ; Calculate the lengths of multiple binary sides : , , ;in, For a class of points or Location information Type II points or Location information. If point group and The ratio of the side lengths satisfies , point group and The ratio of the side lengths satisfies , point group and The ratio of the side lengths satisfies If so, retain 3 matching point pairs to update the set of matching point pairs in the feature description group.

[0105] For example, when randomly selecting 4 point groups At that time, calculate the side lengths of the four point groups in the scanned point cloud and the template point cloud respectively, and and For any two distinct groups of points, the side length between each group can be calculated separately, including: calculating a class of side lengths: , , , , , Second-class side length: , , , , , ;in, For a class of points or Location information Type II points or Location information. If the ratio of the side lengths of any two verified matching point pairs satisfies... If so, retain 4 matching point pairs to update the set of matching point pairs in the feature description group.

[0106] The side length filtering threshold can be set to a value greater than or equal to 0.5.

[0107] It should be understood that the size of the point cloud and the degree of downsampling directly affect the quality and quantity of feature descriptors. To ensure processing speed, when the original point cloud is large, a large voxel size is required for strong downsampling, which carries the risk of leading to point cloud sparsity, loss of geometric details, and a reduction in the number of usable high-discrimination feature descriptors. If strict screening conditions are still applied at this time, it is very easy to fail to generate effective feature descriptor cliques due to insufficient matching point pairs. Therefore, it is necessary to relax the difference filtering threshold in the construction of maximal cliques (e.g., setting...). (), to retain enough candidate matching point pairs.

[0108] However, this lenient strategy may introduce scale-inconsistent mismatches, such as the distance between two points in the template point cloud. The distance of the corresponding point in the scanned point cloud The difference They happen to meet the condition of being less than the difference filtering threshold and are included in the same feature descriptor group, but their side length is less than... This does not conform to the scale invariance of rigid body transformation. In this case, the three-point consistency verification passes the dimensionless side length ratio constraint (filtered by the side length threshold). (Control) can effectively identify and eliminate such distorted matches.

[0109] Conversely, when the point cloud is small, downsampling is weak, and density is high, the feature descriptors are rich and accurate, and strict thresholding can be applied during the maximal clique stage (e.g., ...). This method can eliminate the vast majority of inconsistent matching pairs. Intra-cluster pairs are already highly reliable, and three-point consistency is almost unnecessary to trigger. Therefore, three-point consistency is essentially a compensation mechanism for the noise introduced by loose cluster screening in large-scale, low-density scenarios, maintaining the geometric rationality of the final registration while ensuring recall.

[0110] In some embodiments, when the point cloud density in the scanned point cloud is greater than a preset point cloud density and / or the number of points is greater than an upper limit threshold, a three-point consistency verification is performed on each feature descriptor group.

[0111] Point cloud density refers to the number of points collected per unit area (e.g., per square meter), while the number of points in a 3D point cloud refers to the total number of discrete points contained in the 3D point cloud. Preset point cloud density and upper limit thresholds can be configured in advance according to the accuracy requirements of the application scenario.

[0112] In this embodiment, during the pose determination stage, only a single feature description clique is used for registration, resulting in a significant reduction in computational burden. The validity of the result is determined by combining the number of successfully matched point logs. If invalid, a new clique can be selected based on both the number of nodes within the clique and its geometric differences, considering both the size and structural differences of the clique to avoid getting trapped in local optima. Because the amount of data for a single registration is small, the speed is greatly improved, allowing for efficient attempts at multiple cliques and increasing the overall success rate.

[0113] In some embodiments, S6 further includes: S61, selecting the feature descriptor group with the largest number of current point clouds that has not participated in registration as the target feature descriptor group.

[0114] Specifically, the number of point clouds in all feature descriptor cliques is counted, and the largest feature descriptor clique that currently has the most points and has not yet been used for registration is selected as the target feature descriptor clique. For example, during the initial registration, a clique like the one with the most points is selected. Figure 2 The pose is solved using the feature descriptor clique D1 or D1′ with the most points in the point cloud. It should be understood that the largest feature descriptor clique contains more filtered pairs of similar features, and in practical applications, the probability of successful registration is usually higher.

[0115] In some embodiments, S6 further includes: S62, counting the number of point clouds that have been successfully registered globally under the object pose; if the number of point clouds reaches the number filtering threshold, then outputting the object pose as the target object pose.

[0116] Specifically, step S62 corresponds to the registration and verification process. Based on the current object pose, the scanned point cloud and the template point cloud are unified to the same coordinate system. In the transformed results, successfully registered point pairs in the scanned point cloud and the template point cloud are found. The number of all successfully registered point clouds is counted. When the number of point clouds reaches or exceeds the number filtering threshold, it indicates that the current pose has sufficient global consistency support. The object pose is determined as the target object pose and output, thus completing this pose estimation task.

[0117] For example, based on the current object pose, the entire template point cloud is transformed into the coordinate system of the scanned point cloud to obtain the registration point cloud, which includes several third-class points transformed from first-class points. The registration effect is evaluated by the overlap between the third-class points and the second-class points.

[0118] In some embodiments, S6 further includes: S63, if the number of point clouds does not reach the number filtering threshold, or if the single registration duration is greater than the first duration threshold, reselect the target feature descriptor group.

[0119] Specifically, if the number of successfully registered point clouds does not reach the preset number filtering threshold, or the time taken for a single registration exceeds the first duration threshold, the current registration attempt is deemed a failure. A new target feature description clique is selected from the remaining unused cliques, and pose solving and verification are performed again, until the number of point clouds in a registration reaches the number filtering threshold and the process terminates successfully, or the cumulative time exceeds a higher-level time limit (such as the second duration threshold), forcibly exiting and triggering an adaptive parameter tuning mechanism. The number filtering threshold is the minimum number of points required to determine if a pose is valid. It can be dynamically set as a fixed proportion of the total number of points in the source point cloud, such as 85% to ensure registration accuracy. If the scanned part structure has some similar regions, the similarity requirement needs to be adaptively increased, such as to 95%. The first duration threshold limits the maximum allowed time for a single registration operation, allowing for timely interruption of inefficient calculations and ensuring the real-time performance of the overall process.

[0120] It should be understood that in some special application scenarios, such as when an object has multiple highly similar local structures (such as symmetrical parts or repetitive textures), the largest feature descriptor clique is prone to mismatch, and the complex clique structure may also cause computational lag. Therefore, the switching of feature descriptor cliques is triggered by the number filtering threshold and the first duration threshold to balance the speed and accuracy of registration, which is especially suitable for industrial scenarios with large point cloud scales or high robustness requirements.

[0121] In some embodiments, the reselection of the target feature descriptor group includes: returning to execution step S61;

[0122] For example, such as Figure 2As shown, in the first round of registration, the largest clique is selected as the target feature description clique, i.e., feature description clique D1. If the verification fails, D1 is marked as "participated in registration". When returning to S61 in the next round, feature description clique D1 will be skipped and the second largest clique will be selected as the new target feature description clique, i.e., feature description clique D2, and the subsequent registration and verification process will be repeated.

[0123] In some embodiments, the number of point clouds contained in all feature description groups is counted. The feature description groups can be sorted from most to least number of point clouds to obtain a candidate queue sorted by number. The usage status of the groups is updated in real time according to the registration usage records (for example, after the first round of registration, feature description group D1 is "participated in registration" and feature description group D2 is "not participated in registration"). Based on the candidate queue and the usage status of the groups, multiple geometric hypotheses can be explored in an orderly manner, effectively addressing the mismatch problem in complex application scenarios.

[0124] In some embodiments, the method further includes: performing geometric fitting on several point clouds in each feature descriptor group to obtain corresponding axes; obtaining the target axis of the current target feature descriptor group; and selecting, from the feature descriptor groups whose point cloud quantity is greater than or equal to a lower limit threshold, feature descriptor groups whose deviation between the axis and the target axis is greater than a preset deviation threshold as target feature descriptor groups.

[0125] Geometric fitting refers to the process of mathematically modeling a subset of point clouds to extract its main geometric features. Common techniques include principal component analysis, RANSAC line / plane fitting, and least squares fitting. Correspondingly, the axis is the direction vector describing the orientation of the clique space, obtained through geometric fitting.

[0126] Specifically, geometric fitting is performed on a subset of point clouds within each feature description clique, and its dominant direction is extracted as the axis of that feature description clique. After obtaining the axis of the current target clique, a new clique can be selected by combining the number of nodes within the clique and the geometric differences, taking into account both the size of the clique and its structural uniqueness. This avoids repeatedly trying erroneous assumptions about similar directions in highly similar local structures, instead prioritizing the exploration of regions with significantly different geometric orientations, thus improving the ability to escape local optima. This is particularly suitable for application scenarios where feature description cliques selected based on the number of point clouds (such as the largest clique) repeatedly fail to register.

[0127] The lower limit threshold is used to identify feature descriptor clusters with a large number of point clouds, prioritizing the selection of such feature descriptor clusters. For example, it can be set to 0.1% of the total number of matched point pairs, or to a point cloud count of 1. The preset deviation threshold is used to identify feature descriptor clusters with large angles between axes, prioritizing the selection of such feature descriptor clusters. The specific value can be determined based on the average angle between the axes of multiple feature descriptor clusters and the target axis, ensuring that the feature descriptor clusters registered in the two rounds have significant differences in geometric orientation, thus enhancing the diversity of attitude assumptions.

[0128] In some embodiments, when there is no feature description group whose number of point clouds is greater than or equal to the lower limit threshold and whose deviation between the axis and the target axis is greater than a preset deviation threshold, the feature description group with the largest number of current point clouds that has not participated in registration is selected as the target feature description group; or, the feature description group with the largest deviation between the current axis and the target axis that has not participated in registration is selected as the target feature description group.

[0129] In some embodiments, the preset registration algorithm can refer to relevant prior art. Exemplary examples include the Umeyama algorithm and the Iterative Nearest Point (ICP) algorithm, where the ICP algorithm can be an ICP point-to-plane registration algorithm. For example, a single-step SVD decomposition is performed using the Umeyama algorithm to solve for the rotation matrix R. Further, the translation vector t is estimated based on the centroid of the feature descriptor clique. The rotation matrix R and the translation vector t are used as prior estimated poses to obtain an initial transformation matrix. Based on the initial transformation matrix, the ICP point-to-plane registration algorithm is used to optimize the pose estimation for fine registration, resulting in a finely registered pose transformation matrix. Further, the transformation matrix of this registration is used to perform coordinate system transformation on the scanned point cloud and the template point cloud to verify the registration effect.

[0130] In some embodiments, the method further includes: monitoring the cumulative time consumed in the current registration process; when the cumulative time consumes more than a second time threshold, determining and adjusting the value of a parameter according to the performance preference of the application scenario and a preset progressive strategy corresponding to the performance preference of the application scenario, until the pose of the target object is determined within the second time threshold; the parameter includes at least one of the following: difference filtering threshold, number filtering threshold, voxel size, and side length filtering threshold.

[0131] Specifically, during the registration process, the cumulative time from the start to the present is continuously monitored. If the pose cannot be successfully output after multiple attempts with different feature descriptors, and the total time exceeds the preset second time threshold, an adaptive parameter tuning mechanism is triggered. At this time, based on the preset performance preferences (such as "speed priority" or "accuracy priority") in the application scenario, the parameters to be adjusted are determined according to the preset progressive strategy. Only one parameter is adjusted at a time, and the process is re-executed from the earliest processing step affected by that parameter (for example, adjusting the voxel size requires starting from downsampling processing, and adjusting the difference filtering threshold requires starting from step S4).

[0132] The second duration threshold is the maximum allowed cumulative running time for the entire pose estimation process, used to trigger the adaptive parameter tuning mechanism. It can be set according to the real-time requirements of the actual application, such as 3s or 6s in industrial grasping scenarios. Furthermore, the first duration threshold is the single registration timeout threshold, and the second duration threshold is greater than the first duration threshold.

[0133] Therefore, when convergence is difficult in complex scenarios, configuration can be proactively optimized, ensuring new parameters take effect while avoiding unnecessary recalculations, thus improving registration success rate and system robustness in complex scenarios. Furthermore, embodiments of this application provide differentiated parameter adjustment sequences for different application scenarios, achieving precise adaptation between speed and accuracy requirements.

[0134] In some embodiments, when the performance preference is speed-first mode, the parameter adjustment priority in the preset progressive strategy is: a. increase the difference filtering threshold to relax the generation criteria in the displacement space map; b. increase the side length filtering threshold; c. reduce the voxel size and restore the difference filtering threshold and the side length filtering threshold to their original settings.

[0135] In some embodiments, when the performance preference is a precision-first mode, the parameter adjustment priority in the preset progressive strategy is: d. reduce the voxel size; e. increase the side length filtering threshold; f. increase the difference filtering threshold.

[0136] Specifically, increasing the difference filtering threshold can relax the criterion for judging the geometric consistency between matching point pairs in the displacement space graph, enabling more point pairs to establish associations, thereby expanding the candidate range of maximal clique search and avoiding the loss of effective cliques due to overly strict thresholds. Increasing the side length filtering threshold can reduce the scale ratio requirement in the three-point consistency verification, allowing reasonable matching to be retained under slight deformation or noise interference, and preventing over-filtering and rejection of correct point pairs. On the other hand, reducing the voxel size can improve the point cloud density and geometric details from the data source, enhance the discriminative ability of feature descriptors, and provide richer and more reliable input for subsequent matching.

[0137] Based on the above mechanism, in speed-priority mode, lightweight and low-overhead parameters are adjusted first. First, the difference filtering threshold is increased to quickly expand the available matching point pairs. Even with the introduction of a small amount of noise, the subsequent three-point consistency verification mechanism can perform secondary fine filtering on point pairs within the clique, eliminating scale-distorted erroneous matches. Therefore, basic accuracy is maintained while ensuring efficiency. If adjusting the difference filtering threshold still fails to output the target object pose within the second time threshold, the side length filtering threshold is increased to further relax the scale constraint. These two adjustments only need to be re-executed from the S4 stage (i.e., the feature clique generation stage), without redoing downsampling or feature extraction, significantly saving time. Only when the above strategies fail to output a valid pose within the second time threshold is the time-cost-intensive voxel reduction method used. Although this requires rerunning the entire process from downsampling, it can reconstruct high-quality features and, together with restoring the original difference filtering threshold and side length filtering threshold, increase the number of point clouds filtered during the process, thereby reducing the amount of subsequent environmental processing required and ensuring processing speed to a certain extent.

[0138] In accuracy-first mode, the goal is to more robustly and efficiently approximate the true pose. Therefore, voxel size is prioritized to ensure the input point cloud possesses sufficient geometric information. If registration still fails on this high-fidelity data, the problem may not lie in data density, but rather in overly stringent subsequent data filtering. In this case, the side length filtering threshold and difference filtering threshold are gradually relaxed to reduce the stringency of the filtering.

[0139] In some embodiments, the time consumption of two key stages in the registration process is monitored respectively: the first is the time consumption of the preprocessing and mapping stage (including steps S2 to S5, including feature descriptor calculation, matching point pair generation, displacement space map construction, etc.); the second is the time consumption of the registration stage (including steps S6, S61–S63, etc., involving feature cluster selection, pose solving and verification).

[0140] If the first time taken is greater than or equal to the preset third time threshold, it indicates that the current voxel size is too small, resulting in an excessively large point cloud size after downsampling, leading to an excessive computational burden on feature extraction and graph construction. In this case, the voxel size should be increased to further reduce the point cloud density and improve processing efficiency. If the second time taken is greater than or equal to the fourth time threshold, it indicates that the voxel size is too large, causing the point cloud to be too sparse. This not only results in the loss of key geometric details but may also lead to inaccurate normal estimation, which in turn reduces the discriminative power of feature descriptors, lowers matching quality, and causes multiple registration failures and retries. In this case, the voxel size should be reduced to restore necessary point cloud details and feature reliability. The third time threshold (e.g., 4s) and the fourth time threshold (e.g., 1s, 2s, or 3s) can be determined based on empirical factors such as the object scale, computing power of the computing platform, and real-time requirements in the actual application scenario.

[0141] It should be understood that, in the face of extreme situations (such as the inability to match an accurate pose for an extended period), this application embodiment also provides a scene-adaptive dynamic parameter tuning mechanism. The accumulated time is monitored, and upon timeout, progressive parameter adjustments are initiated based on a performance preference prioritizing speed or accuracy. For example, in speed-priority mode, the difference filtering threshold is first relaxed, with subsequent relaxations used as compensation. Only when necessary are parameters such as voxel size, which can trigger process recalculation and have long processing times, considered for adjustment. In accuracy-priority mode, voxel size is preferentially reduced to improve point cloud quality and feature quantity, thereby improving data quality.

[0142] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating the overall process of 3D point cloud pose estimation provided in an embodiment of this application. The process starts from a control panel and supports operations such as model loading, scanning, movement, and rotation. Raw data is acquired by a binocular camera and then enters the point cloud segmentation module. Coordinate transformation is completed through calibration of intrinsic and extrinsic parameters. Simultaneously, the target recognition module uses a YOLO model to perform real-time detection of the camera video stream, outputting the target bounding box and its two-dimensional coordinates. This helps determine the approximate position of the target object in 3D space. Finally, the point cloud containing only the target object (i.e., the scanned point cloud) is extracted.

[0143] Subsequently, the point cloud of the target object undergoes preprocessing steps such as downsampling and filtering to remove noise and reduce data volume. The preprocessed point cloud is then input into the registration module, where coarse registration is performed to obtain the initial pose, followed by fine registration to optimize the result, ultimately outputting the matching result between the template point cloud and the scanned point cloud. Furthermore, the entire process can simultaneously display the target detection result and the registration result through a display interface, achieving closed-loop processing from image recognition to 3D pose estimation, suitable for scenarios such as industrial automation and robot grasping.

[0144] Therefore, the embodiments of this application reduce the computational load and improve the point cloud reconstruction speed by cropping the computational area. Multiple verification of feature descriptors reduces the false matching rate, and the use of graph-based feature descriptor clusters, through multiple iterations, effectively improves the registration failure problem caused by getting trapped in local optima in traditional iterative algorithms, thus increasing the registration success rate. The object pose estimation method based on 3D point clouds and its specific implementation steps provided in this application can be widely applied to various point cloud scenarios, such as reverse engineering, surface reconstruction, and environmental point cloud reconstruction.

[0145] The methods and apparatus of this application can be used in a wide variety of general-purpose or special-purpose computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer terminal devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0146] Please see Figure 6 , Figure 6 This is a schematic block diagram illustrating the structure of a computer device according to an embodiment of this application. The computer device may be a terminal device or a server.

[0147] For example, the above method can be implemented as a computer program, which can be used in, for example... Figure 6 It runs on the computer device shown.

[0148] like Figure 6 As shown, the computer device includes a processor, memory, and network interface connected via a system bus, wherein the memory may include non-volatile storage media and internal memory.

[0149] Non-volatile storage media can store operating systems and computer programs. These computer programs include program instructions that, when executed, cause the processor to perform any object pose estimation method based on 3D point clouds and its specific implementation steps.

[0150] The processor provides computing and control capabilities, supporting the operation of the entire computer device.

[0151] The internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any object pose estimation method based on 3D point clouds and the specific implementation steps of the method.

[0152] This network interface is used for network communication, such as sending assigned tasks.

[0153] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0154] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps:

[0155] S1, acquire template point cloud and collect scanned point cloud of object, wherein the template point cloud includes several Class I points and the scanned point cloud includes several Class II points;

[0156] S2, calculate the feature descriptor for each of the first-class points and the second-class points; wherein, the feature descriptor is used to represent the vector or histogram of local geometric features or information of the point cloud;

[0157] S3, based on the similarity between the feature descriptors, pair the first-class points and the second-class points to obtain a set of matching point pairs. ,in, and These are the first-class and second-class points that were successfully matched, respectively.

[0158] S4, based on any two matching point pairs and The location information is used to calculate the first type of positional relationship between two matching point pairs in the template point cloud and the second type of positional relationship in the scanned point cloud; if the first type of positional relationship and the second type of positional relationship meet the difference filtering threshold, then the two matching point pairs are associated.

[0159] S5. Construct a displacement space graph based on all established relationships and corresponding matching point pairs, and extract several feature description cliques from the graph based on the maximal clique search algorithm; wherein, the displacement space graph is an undirected graph structure; the maximal clique search algorithm is used to find subgraphs in the undirected graph structure where any two points are connected as feature description cliques;

[0160] S6. Select a target feature description group from a plurality of the aforementioned feature description groups, and perform registration based on the first-class points and second-class points in the target feature description group to determine the corresponding object pose.

[0161] For example, the processor is used to run a computer program stored in the memory, and is also used to implement the steps and specific implementation steps of the object pose estimation method based on three-dimensional point cloud provided in any embodiment of this application, which will not be repeated here.

[0162] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement the steps and specific implementation steps of the object pose estimation method based on three-dimensional point clouds provided in any of the embodiments of this application.

[0163] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiments, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device.

[0164] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for object pose estimation based on 3D point clouds, characterized in that, The method includes: S1, acquire template point cloud and collect scanned point cloud of object, wherein the template point cloud includes several Class I points and the scanned point cloud includes several Class II points; S2, calculate the feature descriptor for each of the first-class points and the second-class points; wherein, the feature descriptor is a vector or histogram used to represent the local geometric features or information of the point cloud; S3, based on the similarity between the feature descriptors, pair the first-class points and the second-class points to obtain a set of matching point pairs. ,in, and These are the first-class and second-class points that were successfully matched, respectively. S3 further includes: Based on the nearest neighbor search algorithm, each binary point is determined. The closest type of points in the template point cloud and each Class I point The s closest target binary points in the scanned point cloud Where s is an integer greater than or equal to 1; The two types of points With the corresponding target type points Pairing is performed to obtain an initial set of matching point pairs: ; The initial set of matching point pairs As a set of matching point pairs Alternatively, for the initial set of matching point pairs... Filtering is performed to obtain a set of matching point pairs. This includes: if the initial matching point pair contains two types of points For target class one points Target Type II points If so, then the initial matching point pair is retained; S4, based on any two matching point pairs and The location information is used to calculate the first type of positional relationship between two matching point pairs in the template point cloud and the second type of positional relationship in the scanned point cloud; if the first type of positional relationship and the second type of positional relationship meet the difference filtering threshold, then the two matching point pairs are associated. S5. Construct a displacement space graph based on all established relationships and corresponding matching point pairs, and extract several feature description cliques from the graph based on the maximal clique search algorithm; wherein, the displacement space graph is an undirected graph structure; the maximal clique search algorithm is used to find subgraphs in the undirected graph structure where any two points are connected as feature description cliques; S6, selecting a target feature description clique from a plurality of the aforementioned feature description cliques, and performing registration based on Class I and Class II points in the target feature description clique to determine the corresponding object pose; after S5, the method further includes: At least three verification matching point pairs are randomly selected from the feature description group; For any two pairs of verification matching points and Perform the following steps respectively: based on the matching point pair and Location information calculation and A type of side length between According to the matching point pair and Location information calculation and The second type of side length between ; Calculate the first-class edge length between any two pairs of verified matching points. and the second type of side length The ratio of the side lengths between ; When the ratio of the side lengths between the verified matching point pairs all satisfy At the same time, the verification matching point pairs are retained to update the set of matching point pairs in the feature description clique; wherein, The threshold value is the side length. The cumulative time consumed in the current registration process is measured; when the cumulative time exceeds the second time threshold, a parameter value is determined and adjusted according to the preset progressive strategy corresponding to the performance preference in the application scenario, until the target object pose is determined within the second time threshold; the parameter includes at least one of the following: difference filtering threshold, number filtering threshold, voxel size, and side length filtering threshold.

2. The method according to claim 1, characterized in that, The method further includes: During the process of collecting the scanning point cloud of an object, the current scanning scene is analyzed in real time through a preset target recognition algorithm to determine the target area where the object is located in three-dimensional space. The point cloud within the target area is used as the scanned point cloud.

3. The method according to claim 1, characterized in that, Before step S2, the method further includes: downsampling the scanned point cloud based on a preset voxel size.

4. The method according to claim 1, characterized in that, S4 includes: Get any two matching point pairs and ; According to the matching point pair and Location information, calculation and A type of positional relationship between ; According to the matching point pair and Location information, calculation and The two types of positional relationships between ; when and satisfy At that time, and Add an edge between ;in, This is the threshold for difference filtering.

5. The method according to claim 1, characterized in that, S6 further includes: S61, Select the feature description cluster with the largest number of current point clouds that has not participated in registration as the target feature description cluster; S62, count the number of point clouds that have been successfully registered globally under the object pose; if the number of point clouds reaches the number filtering threshold, then output the object pose as the target object pose. S63, if the number of point clouds does not reach the number filtering threshold, or if the registration duration of a single registration exceeds the first duration threshold, the target feature descriptor group is reselected.

6. The method according to claim 5, characterized in that, The reselection of the target feature description group includes: Return to step S61; or, Geometric fitting is performed on several point clouds in each feature description cluster to obtain the corresponding axes; Obtain the target axis of the current target feature description cluster; From the feature description groups whose point cloud quantity is greater than or equal to the lower limit threshold, select feature description groups whose deviation between the axis and the target axis is greater than a preset deviation threshold, and use them as target feature description groups.

7. A computer device, characterized in that, The device includes: Memory, used to store computer programs; A processor is configured to execute the computer program and, in executing the computer program, implement the object pose estimation method based on three-dimensional point clouds as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the object pose estimation method based on three-dimensional point clouds as described in any one of claims 1 to 6.

Images