Point cloud matching method, robot arm control method, electronic device, medium and product

Abstract

The present disclosure relates to a point cloud matching method, a robot arm control method, an electronic device, a medium and a product. The point cloud matching method comprises: obtaining a template point cloud and a first feature descriptor; obtaining a plurality of preprocessed scene point clouds, the plurality of preprocessed scene point clouds having different scales for feature extraction; performing, for each preprocessed scene point cloud, a first operation: determining a second feature descriptor based on the scale of the preprocessed scene point cloud, determining a first evaluation index and a second evaluation index of the preprocessed scene point cloud based on the first feature descriptor, the second feature descriptor and a distance threshold, and determining a comprehensive evaluation index of the preprocessed scene point cloud based on the first evaluation index, the second evaluation index and a weight coefficient; and selecting, according to the comprehensive evaluation index, a preprocessed scene point cloud from the plurality of preprocessed scene point clouds to perform point cloud matching with the template point cloud. The present disclosure can improve the matching accuracy and adaptability of point cloud matching.

Description

Technical Field

[0001] This disclosure relates to the field of machine vision technology, and more specifically, to a point cloud matching method, a robotic arm control method based on point cloud matching, electronic devices, computer-readable storage media, and computer program products. Background Technology

[0002] Machine vision-based visual guidance methods for industrial robots are a key research direction in the field of intelligent manufacturing. Point cloud matching is a widely used visual guidance method for industrial robots. It can align point clouds of a target acquired from multiple different perspectives to the same coordinate system, thereby matching the target into a complete point cloud model and guiding the industrial robot to operate on the target. Summary of the Invention

[0003] A brief overview of this disclosure is given below to provide a basic understanding of some aspects of it. However, it should be understood that this overview is not an exhaustive summary of this disclosure. It is not intended to identify key or essential parts of this disclosure, nor is it intended to limit the scope of this disclosure. Its purpose is merely to present certain concepts of this disclosure in a simplified form as a prelude to the more detailed description that follows.

[0004] Template point clouds and scene point clouds are key components of point cloud matching. Template point clouds are typically clean and complete point clouds pre-generated for the target using offline scanning or computer-aided design (CAD). They describe the target's geometry at a standard scale and serve as a reference standard for actual matching. Scene point clouds are point clouds formed based on relevant data about the target in a real-world working environment, acquired in real-time by cameras (e.g., binocular structured light cameras) and sensors. Scene point clouds usually also include the background (such as other objects, noise, etc.). During actual matching, the point cloud corresponding to the template point cloud needs to be found within the scene point cloud to locate the target.

[0005] The point cloud matching methods of related technologies only construct scene point clouds at a single scale, and therefore can only match targets at a specified viewpoint (including distance, angle, etc.). If the actual viewpoint of the target is different from the specified viewpoint, the matching accuracy of the point cloud matching methods of related technologies will decrease significantly. Therefore, the point cloud matching methods of related technologies have poor adaptability and cannot be applied to targets at various viewpoints.

[0006] To overcome the aforementioned problems in the related technologies, the present disclosure proposes the following solutions to achieve point cloud matching more reliably.

[0007] According to a first aspect of this disclosure, a point cloud matching method is provided, comprising: acquiring a template point cloud of a target and a first feature descriptor of the template point cloud; acquiring a plurality of preprocessed scene point clouds of the target, the plurality of preprocessed scene point clouds having different scales for feature extraction; performing a first operation for each preprocessed scene point cloud, the first operation comprising: determining a second feature descriptor of the preprocessed scene point cloud based on the scale of the preprocessed scene point cloud; determining a first evaluation index and a second evaluation index of the preprocessed scene point cloud based on the first feature descriptor, the second feature descriptor, and a distance threshold, wherein the first evaluation index indicates the feature distance between the template point cloud and the preprocessed scene point cloud, the second evaluation index indicates the three-dimensional geometric error between the template point cloud and the preprocessed scene point cloud; and determining a comprehensive evaluation index of the preprocessed scene point cloud based on the first evaluation index, the second evaluation index, and a weighting coefficient for the second evaluation index; and selecting a preprocessed scene point cloud from the plurality of preprocessed scene point clouds and performing point cloud matching with the template point cloud according to the comprehensive evaluation index.

[0008] In some embodiments, the first feature descriptor is a first Fast Point Feature Histogram (FPFH) descriptor, and the second feature descriptor is a second Fast Point Feature Histogram descriptor.

[0009] In some embodiments, determining a first evaluation index and a second evaluation index of the preprocessed scene point cloud based on a first feature descriptor, a second feature descriptor, and a distance threshold includes: determining an interior point set and a transformation matrix based on the first feature descriptor, the second feature descriptor, and a distance threshold, wherein the interior point set includes multiple interior point pairs, each interior point pair including a template point and a corresponding scene point; determining the first evaluation index based on the interior point set; and determining the second evaluation index based on the interior point set and the transformation matrix.

[0010] In some embodiments, the method satisfies at least one of the following: a first evaluation metric is determined based on the feature distance between the first feature descriptor of the template point in each inlier pair and the second feature descriptor of the corresponding scene point; or a second evaluation metric is determined based on the spatial distance between the three-dimensional coordinates of the template point in each inlier pair in the inlier set after transformation by the transformation matrix and the three-dimensional coordinates of the corresponding scene point.

[0011] In some embodiments, each preprocessed scene point cloud is obtained by preprocessing an initial scene point cloud based on a second operation, the second operation including: determining a parameter strategy for the initial scene point cloud based on the environmental quality score of the initial scene point cloud, the parameter strategy including preprocessing parameters for the initial scene point cloud, the preprocessing parameters including at least one of statistical filter standard deviation factor, voxel downsampling size and point cloud quality expectation; and preprocessing the initial scene point cloud based on the preprocessing parameters.

[0012] In some embodiments, the second operation further includes: determining the height range of the target and the height of the target's center point based on the initial scene point cloud; updating the height range of the target based on the target's height range, the target's center point height, and a filtering coefficient; and filtering the initial scene point cloud based on the updated target height range.

[0013] In some embodiments, the parameter strategy also includes a scale and a distance threshold.

[0014] In some embodiments, the method satisfies at least one of the following: determining a first parameter strategy in response to an environmental quality score not being less than a first environmental quality score; determining a second parameter strategy in response to an environmental quality score not being less than a second environmental quality score and less than the first environmental quality score; and determining a third parameter strategy in response to an environmental quality score less than the second environmental quality score, wherein the first environmental quality score is greater than the second environmental quality score, the scale in the first parameter strategy is smaller than the scale in the second parameter strategy, and the scale in the second parameter strategy is smaller than the scale in the third parameter strategy, the distance threshold in the first parameter strategy is smaller than the distance threshold in the second parameter strategy, and the distance threshold in the second parameter strategy is smaller than the distance threshold in the third parameter strategy.

[0015] In some embodiments, the plurality of preprocessed scene point clouds include a first preprocessed scene point cloud, a second preprocessed scene point cloud, and a third preprocessed scene point cloud, wherein the scale of the first preprocessed scene point cloud is the scale in the parameter strategy, the scale of the second preprocessed scene point cloud is smaller than the scale in the parameter strategy, and the scale of the third preprocessed scene point cloud is larger than the scale in the parameter strategy.

[0016] In some embodiments, the environmental quality score is determined based on at least one of the target's light intensity, vibration intensity, and surface reflectivity.

[0017] According to a second aspect of this disclosure, a robotic arm control method based on point cloud matching is provided, comprising: acquiring a template point cloud of a target and a first feature descriptor of the template point cloud; acquiring multiple preprocessed scene point clouds of the target, the multiple preprocessed scene point clouds having different scales for feature extraction; and performing a first operation for each pair of preprocessed scene point clouds, the first operation comprising: determining a second feature descriptor of the preprocessed scene point cloud based on the scale of the preprocessed scene point cloud, and determining a first evaluation index of the preprocessed scene point cloud based on the first feature descriptor, the second feature descriptor, and a distance threshold. The system includes a first evaluation index and a second evaluation index, wherein the first evaluation index indicates the feature distance between the template point cloud and the preprocessed scene point cloud, the second evaluation index indicates the three-dimensional geometric error between the template point cloud and the preprocessed scene point cloud, and a comprehensive evaluation index is determined based on the first evaluation index, the second evaluation index, and the weighting coefficients used for the second evaluation index to determine the preprocessed scene point cloud; and based on the comprehensive evaluation index, the system selects a preprocessed scene point cloud from multiple preprocessed scene point clouds and performs point cloud matching with the template point cloud to obtain the transformed three-dimensional coordinates of the target; and controls the robotic arm to complete the operation related to the target based on the transformed three-dimensional coordinates of the target.

[0018] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory storing computer-executable instructions, which, when executed by the processor, cause the processor to perform a point cloud matching method according to any embodiment of the first aspect of this disclosure or a point cloud matching-based robotic arm control method according to the second aspect of this disclosure.

[0019] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having computer-executable instructions stored thereon, which, when executed by a computer, cause the computer to perform a point cloud matching method according to any embodiment of the first aspect of this disclosure or a point cloud matching-based robotic arm control method according to the second aspect of this disclosure.

[0020] According to a fifth aspect of this disclosure, a computer program product is provided, the computer program product including instructions that, when executed by a processor, implement the point cloud matching method according to any embodiment of the first aspect of this disclosure or the point cloud matching-based robotic arm control method according to the second aspect of this disclosure.

[0021] According to the point cloud matching method provided in the embodiments of this disclosure, the method constructs multiple preprocessed scene point clouds of different scales and selects the preprocessed scene point cloud that best matches the template point cloud from among the multiple preprocessed scene point clouds of different scales as the final target scene point cloud for actual matching with the template point cloud. Therefore, the point cloud matching method of this disclosure can improve the adaptability of the point cloud matching method to targets under different viewpoints and is beneficial to improving the matching accuracy for targets under different viewpoints. Attached Figure Description

[0022] The foregoing and other features and advantages of this disclosure will become clear from the following description of embodiments illustrated in conjunction with the accompanying drawings. The drawings, incorporated herein and forming a part of the specification, are further used to explain the principles of this disclosure and to enable those skilled in the art to make and use it. Wherein:

[0023] Figure 1 This is a flowchart illustrating a point cloud matching method according to some embodiments of the present disclosure;

[0024] Figure 2 This is a flowchart illustrating a point cloud matching-based robotic arm control method according to some embodiments of the present disclosure;

[0025] Figure 3 This is a schematic diagram illustrating exemplary working scenarios according to some embodiments of the present disclosure;

[0026] Figure 4 This is a schematic block diagram illustrating an electronic device according to some embodiments of the present disclosure;

[0027] Figure 5 This is a schematic block diagram illustrating a computer system on which embodiments of the present disclosure may be implemented.

[0028] Note that in the embodiments described below, the same reference numerals are sometimes used across different figures to denote the same parts or parts with the same function, and repeated descriptions are omitted. In some cases, similar reference numerals and letters are used to denote similar items, so once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0029] For ease of understanding, the positions, dimensions, and extents of the structures shown in the accompanying drawings and other materials may not represent actual positions, dimensions, and extents. Therefore, this disclosure is not limited to the positions, dimensions, and extents disclosed in the accompanying drawings and other materials. Detailed Implementation

[0030] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0031] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the scope of this disclosure or its application or use. That is, the structures and methods herein are shown in an exemplary manner to illustrate different embodiments of the structures and methods in this disclosure. However, those skilled in the art will understand that they merely illustrate exemplary ways that can be used to implement this disclosure, and not exhaustive ways. Furthermore, the drawings are not necessarily drawn to scale, and some features may be enlarged to show details of specific components.

[0032] In addition, techniques, methods and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods and equipment should be considered part of the specification.

[0033] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0034] Figure 1 This is a flowchart illustrating a point cloud matching method 100 according to some embodiments of the present disclosure. For example... Figure 1 As shown, the point cloud matching method 100 may include steps S102 to S108.

[0035] In step S102, a template point cloud of the target and a first feature descriptor of the template point cloud are obtained.

[0036] In step S104, multiple preprocessed scene point clouds about the target are obtained, and the multiple preprocessed scene point clouds have different scales for feature extraction.

[0037] As a non-limiting example, multiple preprocessed scene point clouds may include a first preprocessed scene point cloud, a second preprocessed scene point cloud, and a third preprocessed scene point cloud. The first preprocessed scene point cloud has a first scale, and the first scale indicates a normal vector computation radius of 5 mm. The second preprocessed scene point cloud has a second scale, and the second scale indicates a normal vector computation radius of 3 mm. The third preprocessed scene point cloud has a third scale, and the third scale indicates a normal vector computation radius of 8 mm. Compared to the first scale, the second scale indicates a smaller normal vector computation radius to further improve the discriminative power and accuracy of point cloud matching, while the third scale indicates a larger normal vector computation radius to further improve the stability and robustness of point cloud matching. Therefore, multiple preprocessed scene point clouds can cover different scales, thereby meeting different feature extraction needs.

[0038] In step S106, a first operation is performed for each preprocessed scene point cloud. The first operation may include: determining a second feature descriptor for the preprocessed scene point cloud based on its scale; determining a first evaluation index and a second evaluation index for the preprocessed scene point cloud based on the first feature descriptor, the second feature descriptor, and a distance threshold; and determining a comprehensive evaluation index for the preprocessed scene point cloud based on the first evaluation index, the second evaluation index, and a weighting coefficient for the second evaluation index. Here, the first evaluation index indicates the feature distance between the template point cloud and the preprocessed scene point cloud, and the second evaluation index indicates the three-dimensional geometric error between the template point cloud and the preprocessed scene point cloud.

[0039] In some embodiments, the first feature descriptor may be a first FPFH descriptor, and the second feature descriptor may be a second FPFH descriptor.

[0040] In some examples, the first feature descriptor of the template point cloud can be the first feature descriptor of each template point in the template point cloud, and the second feature descriptor of the preprocessed scene point cloud can be the second feature descriptor of each scene point in the preprocessed scene point cloud. In other examples, the first feature descriptor of the template point cloud can also be the first feature descriptor of each template keypoint in the template point cloud, and the second feature descriptor of the preprocessed scene point cloud can also be the second feature descriptor of each scene keypoint in the preprocessed scene point cloud.

[0041] Taking the first preprocessed scene point cloud in the aforementioned example as an example, the second FPFH descriptor of each point in the first preprocessed scene point cloud can be determined through the following non-restrictive steps: Calculate the normal vector of each point in the first preprocessed scene point cloud based on a normal vector radius of 5 mm and the first preprocessed scene point cloud; Select a set of scene key points from the first preprocessed scene point cloud; determine the second FPFH descriptor of each scene key point in the scene key point set based on the normal vector of each scene key point in the scene key point set, the FPFH calculation radius, and the first preprocessed scene point cloud, where the FPFH calculation radius can be 5 times the normal vector calculation radius (i.e., 25 mm). Of course, it is possible to determine the second FPFH descriptor directly for each scene point without selecting scene key points; however, selecting scene key points can reduce the computational load, which is beneficial for improving computational efficiency and reducing resource consumption.

[0042] The process of determining the first FPFH descriptor is similar to that of determining the second FPFH descriptor, and will not be repeated here.

[0043] In some embodiments, determining a first evaluation index and a second evaluation index of the preprocessed scene point cloud based on a first feature descriptor, a second feature descriptor, and a distance threshold includes: determining an interior point set and a transformation matrix based on the first feature descriptor, the second feature descriptor, and a distance threshold, wherein the interior point set includes multiple interior point pairs, each interior point pair including a template point and a corresponding scene point; determining the first evaluation index based on the interior point set; and determining the second evaluation index based on the interior point set and the transformation matrix.

[0044] In some embodiments, determining the interior point set and transformation matrix based on the first feature descriptor, the second feature descriptor, and a distance threshold may include: determining a corresponding point set based on the first feature descriptor and the second feature descriptor, the corresponding point set including multiple corresponding point pairs, each corresponding point pair including a template point and a scene point with the smallest feature distance to the template point; performing multiple third operations, the third operations may include: constructing a subset with a specified number of corresponding point pairs based on the corresponding point set, determining an initial transformation matrix based on the subset, and determining an interior point set from the corresponding point set based on the initial transformation matrix, wherein the spatial distance between the 3D coordinates of the template point in each interior point pair after transformation by the initial transformation matrix and the 3D coordinates of the corresponding scene point is less than a distance threshold; selecting the initial transformation matrix with the largest number of interior point pairs in its related interior point set as the transformation matrix.

[0045] As a non-limiting example, firstly, a template keypoint set I can be selected from the template point cloud, and a scene keypoint set J can be selected from the first preprocessed scene point cloud. For each template keypoint, each scene keypoint is iterated to find the scene keypoint j with the smallest feature distance to the template keypoint i. Thus, the template keypoint i and the scene keypoint j form a corresponding point pair (i, j), thereby obtaining the corresponding point set M. The first feature descriptor of the template keypoint is f. i The second feature descriptor for the key points of the scene is f. j The feature distance between template keypoints and scene keypoints is the Euclidean distance between the first and second feature descriptors, i.e., feature distance d = ||f i -f j ||2, where ||·||2 is the L2 norm. For ease of description, in sections not involving keypoint filtering, "keypoint" will no longer be emphasized but will be directly described as scene points or template points. Scene points or template points can be filtered scene keypoints or template keypoints, or they can be unfiltered scene points or template points.

[0046] Secondly, after obtaining the corresponding point set, the interior point set and transformation matrix can be obtained through iteration using the Random Sample Consensus (RANSAC) algorithm. Specifically, in each iteration: three pairs of corresponding points can be randomly selected from the corresponding point set, and the initial transformation matrix T can be calculated based on the three-dimensional coordinates of these three pairs of corresponding points; for each pair of corresponding points (i, j) in the corresponding point set, the three-dimensional coordinates k of the template point i in the corresponding point pair are calculated. i The transformed three-dimensional coordinates k are obtained by performing a transformation based on the initial transformation matrix T. i T And determine k i T The three-dimensional coordinates k of the corresponding scene point j j Spatial distance dist=||k i T -k j||2; Compare the spatial distance of the corresponding point pair with a distance threshold. If the spatial distance does not exceed the distance threshold, the corresponding point pair is determined to be an interior point pair, thereby determining the interior point set P associated with the initial transformation matrix T. After multiple iterations, the initial transformation matrix with the largest number of interior point pairs in its associated interior point set is selected as the final transformation matrix, and its associated interior point set is selected as the final interior point set. Thus, the interior point set obtained after feature distance matching and spatial distance verification satisfies geometric consistency and can be used for subsequent precise transformations. In some embodiments, since the dimensions of the feature distance indicated by the first evaluation index and the spatial distance indicated by the second evaluation index are not consistent, it is necessary to align the dimensions of the first evaluation index with the dimensions of the second evaluation index to calculate the comprehensive evaluation index. In some examples, the first evaluation index and the second evaluation index can be normalized separately before subsequent calculations.

[0047] In some examples, the Sample Consensus Initial Alignment (SAC-IA) algorithm can be used to filter the interior point set and obtain a preliminary transformation matrix, which can then be combined with the Iterative Closest Point (ICP) algorithm to determine the precise transformation matrix.

[0048] In this document, the same or similar characters may be used to represent the same or similar variables. Therefore, once a variable is defined in one embodiment, it does not need to be described again in subsequent embodiments.

[0049] In some embodiments, the first evaluation metric is determined based on the feature distance between the first feature descriptor of the template point in each intrapoint pair and the second feature descriptor of the corresponding scene point. That is, the first evaluation metric can be determined based on the feature distance between the template point and the corresponding scene point in each intrapoint pair.

[0050] In some examples, the first evaluation metric can be the root mean square of the feature distances between the template point and the corresponding scene point in all interior point pairs. Continuing with the aforementioned non-restrictive example, assuming the interior point set P includes N interior point pairs, the first evaluation metric... ,in, Let be the feature distance between the p-th inner point pairs. Therefore, by aggregating the feature distances of all inner point pairs, we can obtain the feature distance between point clouds, and thus evaluate the consistency of features between the template point cloud and the preprocessed scene point cloud based on a first evaluation metric.

[0051] In some embodiments, the second evaluation metric is determined based on the spatial distance between the 3D coordinates of the template point in each pair of interior points in the interior point set after transformation by the transformation matrix and the 3D coordinates of the corresponding scene point. Therefore, the 3D geometric error indicated by the second evaluation metric can represent the actual distance deviation in 3D space between the template point cloud and the scene point cloud after alignment by the transformation matrix.

[0052] In some examples, the second evaluation metric can be the root mean square of the spatial distance between the transformed 3D coordinates of the template point and the corresponding 3D coordinates of the scene point in all interior point pairs. Continuing with the aforementioned non-restrictive example, assuming the interior point set P includes N interior point pairs, the second evaluation metric... ,in, Let be the spatial distance between the p-th pair of interior points. Therefore, by aggregating the spatial distances of all interior point pairs, the spatial distance between point clouds can be obtained. This allows us to evaluate the 3D geometric consistency, or 3D geometric error, between the template point cloud and the preprocessed scene point cloud based on a second evaluation metric.

[0053] After obtaining the first evaluation index and the second evaluation index, the comprehensive evaluation index of the preprocessed scene point cloud can be determined based on the first evaluation index, the second evaluation index, and the weighting coefficient used for the second evaluation index (which can be simply referred to as the "weighting coefficient").

[0054] In some embodiments, the comprehensive evaluation index can be the product of the second evaluation index and the weighting coefficient, plus the sum of the first evaluation index. Continuing with the foregoing non-limiting example, the comprehensive evaluation index of the first preprocessed scene cloud... ,in, , where is the weighting coefficient. The process for determining the comprehensive evaluation index of each preprocessed scene point cloud is similar to the process given in the non-restrictive examples of the first preprocessed scene point cloud, and will not be repeated here.

[0055] In some embodiments, since the dimensions of the feature distance indicated by the first evaluation index and the spatial distance indicated by the second evaluation index are not consistent, it is necessary to align the dimensions of the first evaluation index with those of the second evaluation index to calculate the comprehensive evaluation index. In some examples, the first and second evaluation indices can be normalized separately before subsequent calculations.

[0056] As a non-restrictive example, the statistics for the first and second evaluation metrics of all candidate point clouds can be calculated on a representative validation set. These statistics can be the median, mean, or standard deviation. Then, the first evaluation metric of the scene point clouds in the actual matching process can be divided by the statistics of the first evaluation metric, and the second evaluation metric of the scene point clouds in the actual matching process can be divided by the statistics of the second evaluation metric, thereby normalizing the first and second evaluation metrics.

[0057] In some examples, the weight coefficients can be directly set to 1 after normalization. In other examples, the weight coefficients can be determined through empirical calibration or validation set search. For example, on a representative dataset, a grid search can be performed on the weight coefficients on a logarithmic scale with matching accuracy as the evaluation metric, and the weight coefficient with the highest matching accuracy can be selected as the final weight coefficient.

[0058] After obtaining a comprehensive evaluation index by matching multiple preprocessed scene point clouds with template point clouds, the smaller the value of the comprehensive evaluation index, the better the match between the corresponding preprocessed scene point cloud and the template point cloud.

[0059] In step S108, based on a comprehensive evaluation index, a preprocessed scene point cloud (i.e., the target scene point cloud) is selected from multiple preprocessed scene point clouds and matched with a template point cloud. In some examples, the preprocessed scene point cloud with the smallest comprehensive evaluation index value can be selected as the target scene point cloud.

[0060] Therefore, the point cloud matching method disclosed herein is applicable to target matching from multiple perspectives and helps improve the matching accuracy of targets from different perspectives. Compared with single-scale point cloud matching methods, in the point cloud matching method disclosed herein, even if matching fails at a certain scale (i.e., the preprocessed scene point cloud and the template point cloud do not match at that scale), other scales can still provide effective matching. When performing point cloud matching, the spatial correspondence between the template point cloud and the preprocessed scene point cloud can be determined, and the coordinates of the target in the coordinate system of the preprocessed scene point cloud (i.e., the three-dimensional coordinates after transformation by the transformation matrix) can be obtained. This provides key pose input for the industrial robot that performs subsequent operations on the target, achieving more accurate geometric positioning and registration, and thus guiding the industrial robot to better complete the relevant tasks.

[0061] Furthermore, this disclosure does not aggregate feature descriptors from multiple preprocessed scene point clouds to form a comprehensive feature descriptor. This is because there may be scene point clouds among the multiple preprocessed scene point clouds that cannot match the template point cloud well. If the feature descriptors of scene point clouds that cannot match the template point cloud well are aggregated with the feature descriptors of other scene point clouds, it will affect the matching between the comprehensive feature descriptor and the template point cloud, and will instead degrade the accuracy of point cloud matching.

[0062] In some embodiments, each preprocessed scene point cloud may be obtained by preprocessing an initial scene point cloud based on a second operation. In some examples, the initial scene point cloud may be constructed based on relevant data of targets in a real working environment acquired by a binocular structured light camera.

[0063] In this embodiment, the second operation may include: determining a parameter strategy for the initial scene point cloud based on the environmental quality score of the initial scene point cloud, the parameter strategy including preprocessing parameters for the initial scene point cloud, the preprocessing parameters including at least one of statistical filter standard deviation factor, voxel downsampling size and expected point cloud quality; and preprocessing the initial scene point cloud based on the preprocessing parameters.

[0064] In some embodiments, the parameter strategy may further include a scale and a distance threshold. Here, the scale is used to indicate the radius of the normal vector calculation during feature extraction. For example, when the scale is 2 mm, the radius of the normal vector calculation does not exceed 2 mm. Typically, the radius of the normal vector calculation is taken as the scale. The distance threshold can be used as a criterion for filtering the set of interior points.

[0065] In some examples, the scale and distance thresholds included in the parameter strategy are scale and distance thresholds that are determined by expert experience under the environmental conditions corresponding to the parameter strategy, and that can guarantee the accuracy and stability of point cloud matching.

[0066] In some embodiments, the multiple preprocessed scene point clouds may include a first preprocessed scene point cloud, a second preprocessed scene point cloud, and a third preprocessed scene point cloud. The scale of the first preprocessed scene point cloud is the scale specified in the parameter strategy, the scale of the second preprocessed scene point cloud is smaller than the scale specified in the parameter strategy, and the scale of the third preprocessed scene point cloud is larger than the scale specified in the parameter strategy. Therefore, by scaling down and up based on the scale specified in the parameter strategy to form multiple scales, the scales of the multiple preprocessed scene point clouds can cover different needs for feature extraction.

[0067] In some embodiments, the environmental quality score is determined based on at least one of the target's light intensity, vibration intensity, and surface reflectivity. In some examples, the target's light intensity can be acquired using a light intensity sensor, the target's vibration intensity can be acquired using a vibration sensor, and the target's surface reflectivity can be acquired using a material sensing device or a device for detecting the reflectivity of an object's surface.

[0068] In some examples, the environmental quality score can be the sum of the product of the illumination score indicating the light intensity of the target and the illumination weighting coefficient, the product of the vibration score indicating the vibration intensity of the target and the vibration weighting coefficient, and the product of the reflectance score indicating the surface reflectance of the target and the reflectance weighting coefficient. Of course, the environmental quality score can also be determined based on many other environmental factors, such as visibility, occlusion, etc.

[0069] As a non-restrictive example, environmental quality score ,in, For illumination fraction, For vibration fractions, The reflectance fraction, The illumination weighting coefficient and , The vibration weighting coefficient and , The reflectivity weighting coefficient and Here, the illumination fraction, vibration fraction, and reflectance fraction are all normalized fractions.

[0070] Where L is the illumination intensity in lux, and 500-2000 lux is the optimal range where structured light projection has the best visibility; more than 10000 lux is the high illumination range where the projection is flooded by ambient light and the point cloud quality decreases by more than 60%; less than 100 lux is the low illumination range where the projection is not visible and the point cloud density decreases by more than 80%.

[0071] Where A is the vibration intensity in m / s² 2 (meters per square second), of which, not exceeding 2 m / s² is the low vibration range, with point cloud position error less than 0.1 mm; 2-5 m / s² is the medium vibration range, with point cloud position error of 0.1-0.3 mm; and exceeding 10 m / s² is the high vibration range, with point cloud position error exceeding 0.5 mm.

[0072] Where R is the surface reflectance, and 20%-60% is the optimal range, such as the surface reflectance of typical metals and plastics; below 5% is the low reflectance range, with a point cloud missing rate of more than 50% and a depth measurement error of more than 2 mm; above 90% is the high reflectance range, where specular reflection causes point cloud distortion and the feature extraction failure rate exceeds 40%.

[0073] In some embodiments, the point cloud matching method 100 further satisfies at least one of the following: determining a first parameter strategy in response to an environmental quality score not being less than a first environmental quality score; determining a second parameter strategy in response to an environmental quality score not being less than a second environmental quality score and less than the first environmental quality score; and determining a third parameter strategy in response to an environmental quality score less than the second environmental quality score. Here, the first environmental quality score is greater than the second environmental quality score, the scale in the first parameter strategy is smaller than the scale in the second parameter strategy, and the scale in the second parameter strategy is smaller than the scale in the third parameter strategy, the distance threshold in the first parameter strategy is smaller than the distance threshold in the second parameter strategy, and the distance threshold in the second parameter strategy is smaller than the distance threshold in the third parameter strategy.

[0074] As a non-limiting example, the first environmental quality score can be 0.8, and the second environmental quality score can be 0.5. The first parameter strategy may include: distance threshold: 5 mm; normal vector calculation radius: 5 mm; statistical filter standard deviation factor: 1; voxel downsampling size: 1.5 mm; expected point cloud quality: noise less than 0.1 mm, integrity greater than 95%. The second parameter strategy may include: distance threshold: 8 mm; statistical filter standard deviation factor: 1.5; voxel downsampling size: 2 mm; normal vector calculation radius: 8 mm; expected point cloud quality: noise 0.1-0.3 mm, integrity 85%-95%. The third parameter strategy may include: distance threshold: 12 mm; statistical filter standard deviation factor: 2; voxel downsampling size: 3 mm; normal vector calculation radius: 12 mm; expected point cloud quality: noise 0.3-0.5 mm, integrity 70%-85%.

[0075] It can be observed that the lower the environmental quality score, the larger the distance threshold to tolerate greater geometric deformation, and the larger the normal vector calculation radius to improve the robustness and stability of point cloud matching; the larger the standard deviation factor of the statistical filter used for preprocessing to filter out more outlier noise; and the larger the voxel downsampling size used for preprocessing to perform coarser sampling. Therefore, by assessing the quality of the target's real working environment and adopting corresponding parameter strategies, the impact of environmental quality on point cloud construction and subsequent point cloud matching can be effectively reduced, thereby improving the accuracy of point cloud matching.

[0076] In some embodiments, when the reflectance score is below a reflectance score threshold, the structured light parameters of the binocular structured light camera used to construct the template point cloud and scene point cloud can be adjusted. For example, when the surface reflectance is determined to be in the low reflectance range, the projection intensity can be increased, the exposure time can be extended, and the point cloud density threshold can be decreased. When the surface reflectance is determined to be in the high reflectance range, the projection intensity can be reduced, the exposure time can be shortened, polarization filtering can be applied to reduce specular reflection interference, and data can be acquired from more angles to improve point cloud integrity through multi-angle fusion.

[0077] In some embodiments, the collected data on environmental factors (such as light intensity, vibration intensity, surface reflectivity, etc.) can also be validated. For example, regarding light intensity, it can be verified whether the collected light intensity data is within the range of the light sensor (e.g., 0-100,000 lux); regarding vibration intensity, it can be verified whether the collected vibration intensity data is within the range of the vibration sensor (e.g., 0-50 m / s). 2 Within a certain range, the surface reflectance can be verified to ensure it falls within a reasonable range (0-100%). If the collected data is within the corresponding range, it is considered valid. Otherwise, it is considered invalid.

[0078] In some embodiments, if the collected data on an environmental factor is invalid, the previously collected valid data for that environmental factor can be used. If the data for that environmental factor is invalid multiple times consecutively, the parameter strategy can be determined to be a third parameter strategy, and the relevant information can be recorded for subsequent analysis and optimization.

[0079] As a non-limiting illustrative example, taking a screw as the target, in practical applications, when a camera takes a picture, the scene may also include other objects such as tooling frames and material boxes, and the screw is placed in the material box. Therefore, the high point in the scene is not necessarily the screw cap. However, screws often come in various sizes, and different sizes of screws require different filtering parameters. Currently, point cloud matching methods in related technologies require manual adjustment of filtering parameters, lacking adaptability.

[0080] Therefore, in some embodiments, the second operation may further include: determining the height range of the target and the height of the target's center point based on the initial scene point cloud; updating the height range of the target based on the target's height range, the target's center point height, and a filtering coefficient; and filtering the initial scene point cloud based on the updated target height range.

[0081] In some examples, the Z-axis value range of the screw can be determined based on the initial scene point cloud (in 3D coordinates, the Z-axis is usually used to indicate the height of an object), that is, the maximum Z-axis value of the screw. max and the minimum Z-axis value Zmin And determine the Z-axis value of the screw's center point. Then, the filtration coefficient can be combined. and To update the Z-axis value range of the screw. The updated minimum Z-axis value of the screw. The updated maximum Z-axis value of the screw The updated Z-axis value of the screw's center point Finally, use the updated Z-axis value range. This is used to filter the initial scene point cloud. The filtering coefficient can be adjusted within the range of 0.3-0.6 according to the density and noise level of the initial scene point cloud. For example, and You can take 0.4 for all of them.

[0082] Of course, it's understandable that if the center point of the target is not the center point of the scene, the density peak value along the Z-axis or a known height offset can be used as a substitute. In addition, the height range can be truncated to the percentile to obtain an updated height range.

[0083] Therefore, the filtering parameters can be adaptively adjusted according to different target specifications, thereby achieving adaptive filtering of the initial scene point cloud, improving the automation level of the point cloud matching method, and eliminating the need for manual adjustment of the filtering parameters.

[0084] Preprocessing the initial scene point cloud (including parameter strategy selection and Z-axis filtering) can not only reduce the interference of the background on point cloud matching, but also adaptively adjust the preprocessing parameters, scale and distance threshold according to various environmental factors of the target's real working environment. This reduces the difference between the scene corresponding to the template point cloud and the real working environment, and improves the stability and accuracy of point cloud matching.

[0085] In another aspect, this disclosure also provides a robotic arm control method based on point cloud matching. Figure 2 This is a flowchart illustrating a point cloud matching-based robotic arm control method 200 according to some embodiments of the present disclosure. Figure 2 As shown, the robotic arm control method 200 based on point cloud matching may include steps S202 to S210.

[0086] In step S202, a template point cloud of the target and a first feature descriptor of the template point cloud are obtained.

[0087] In step S204, multiple preprocessed scene point clouds about the target are obtained, and the multiple preprocessed scene point clouds have different scales for feature extraction.

[0088] In step S206, a first operation is performed for each preprocessed scene point cloud. The first operation may include: determining a second feature descriptor for the preprocessed scene point cloud based on its scale; determining a first evaluation index and a second evaluation index for the preprocessed scene point cloud based on the first feature descriptor, the second feature descriptor, and a distance threshold; and determining a comprehensive evaluation index for the preprocessed scene point cloud based on the first evaluation index, the second evaluation index, and a weighting coefficient for the second evaluation index. Here, the first evaluation index indicates the feature distance between the template point cloud and the preprocessed scene point cloud, and the second evaluation index indicates the three-dimensional geometric error between the template point cloud and the preprocessed scene point cloud.

[0089] In step S208, based on the comprehensive evaluation index, the preprocessed scene point cloud is selected from multiple preprocessed scene point clouds and the template point cloud is matched to obtain the transformed three-dimensional coordinates of the target.

[0090] In step S210, the robotic arm is controlled to complete the operation related to the target based on the transformed three-dimensional coordinates of the target.

[0091] The point cloud matching steps (steps S202 to S208) in the point cloud matching-based robotic arm control method 200 can refer to any of the aforementioned embodiments of the point cloud matching method 100, and will not be repeated here.

[0092] In some examples, the preprocessed scene point cloud with the lowest comprehensive evaluation index value can be selected as the target scene point cloud, and its transformation matrix can be used as the target transformation matrix. The 3D coordinates of the target in the template point cloud coordinate system are transformed into 3D coordinates in the scene point cloud coordinate system (or, in other words, the camera coordinate system) using the target transformation matrix. Furthermore, the 3D coordinates of the target in the camera coordinate system are transformed into the robotic arm coordinate system using a fixed transformation matrix obtained from hand-eye calibration. This yields the target's pose in the robotic arm's coordinate system, allowing the robotic arm's motion trajectory to be determined based on the target's pose. Finally, the robotic arm is controlled to complete the task related to the target based on the determined motion trajectory.

[0093] Taking excavator production and assembly as an example, the tightening of track roller bolts is a crucial step in chassis assembly. A single excavator production line needs to assemble six different excavator models, requiring six different sizes of track roller bolts. For a robotic arm using point cloud matching methods as its visual guidance, this method would need to be executed six times. However, for a robotic arm applying the point cloud matching method of this disclosure, for different track roller bolts, the robotic arm selects the most suitable pre-processed scene point cloud from multiple pre-processed scene point clouds of different scales for each type of track roller bolt, and then performs point cloud matching with a template point cloud. This guides the robotic arm to complete the tightening operation for the six different sizes of track roller bolts. Therefore, the computational load on the robotic arm's control system can be reduced, improving control efficiency.

[0094] For the purpose of non-restrictive explanation, combined with Figure 3 The complete example described below is provided.

[0095] Task: In the exemplary work scenario 300, the robotic arm 302 picks up a screw 308 from the material box 310 based on the binocular structured light camera 306.

[0096] Environmental factor data collection: Sensor module 304 (which can be equipped with light sensors, vibration sensors, material sensing devices, etc.) collects data at a light intensity of 1200 lux and a vibration intensity of 3.5 m / s². 2 The surface reflectivity is 45%, and the environmental quality score is determined to be 0.91. Based on this, the first parameter strategy is selected, and the scale of the first preprocessed scene point cloud is determined to be 5 mm, the scale of the second preprocessed scene point cloud is 3 mm, and the scale of the third preprocessed scene point cloud is 8 mm.

[0097] Input parameters: Template point cloud: 678 points; each preprocessed scene point cloud: 710 points.

[0098] Processing results: Multi-scale matching, the preprocessed scene point cloud with a scale of 5 mm (i.e., the normal vector calculation radius is 5 mm) was selected as the target scene point cloud, matching score: 183.85, interior point set: 678 interior point pairs.

[0099] Coordinate output: Camera coordinate system: [335.4, -52.7, 330.3] mm; Attitude: Roll = -20°, Pitch = 15°, Yaw = 96°.

[0100] Robotic arm guidance information: Target position: X=335.4 mm, Y=-52.7 mm, Z=330.3 mm; Target attitude: Roll=-20°, Pitch=15°, Yaw=96°; Tightening torque: 120 N·m; Tightening angle: 180°; Positioning accuracy: ±0.5 mm; Attitude accuracy: ±1°.

[0101] Therefore, the point cloud matching-based robotic arm control method disclosed herein can accurately complete tasks related to the target.

[0102] This disclosure also provides an electronic device. (See reference...) Figure 4 This is a schematic block diagram illustrating an electronic device 400 according to some embodiments of the present disclosure. Figure 4 As shown, electronic device 400 includes a processor 402 and a memory 404 storing computer-executable instructions that, when executed by processor 402, cause processor 402 to perform the methods described according to any of the foregoing embodiments (e.g., point cloud matching method 100 or point cloud matching-based robotic arm control method 200). Processor 402 may be, for example, a central processing unit (CPU) of electronic device 400. Processor 402 may be any type of general-purpose processor or may be a processor specifically designed for point cloud matching or point cloud matching-based robotic arm control, such as an application-specific integrated circuit (“ASIC”). Memory 404 may be coupled to processor 402 and may include various computer-readable media accessible by processor 402. In various embodiments, memory 404 described herein may include volatile and non-volatile media, removable and non-removable media. For example, memory 404 may include any combination of random access memory (“RAM”), dynamic RAM (“DRAM”), static RAM (“SRAM”), read-only memory (“ROM”), flash memory, cache memory, and / or any other type of non-transient computer-readable media. The memory 404 may store instructions that, when executed by the processor 402, cause the processor 402 to execute the method described according to any of the foregoing embodiments of this disclosure.

[0103] The electronic device 400 is configured to perform the method described in any of the foregoing embodiments, and therefore reference can be made to the preceding description of various embodiments of the method, which will not be repeated here.

[0104] This disclosure also provides a computer-readable storage medium having computer-executable instructions stored thereon, which, when executed by a processor, cause the processor to perform a point cloud matching method or a point cloud matching-based robotic arm control method according to any of the foregoing embodiments of this disclosure.

[0105] This disclosure also provides a computer program product that may include instructions that, when executed by a processor, can implement the point cloud matching method or the point cloud matching-based robotic arm control method according to any of the foregoing embodiments of this disclosure. The instructions may be any set of instructions that can be directly executed by one or more processors, such as machine code, or any set of instructions that can be indirectly executed, such as a script. The instructions may be stored in an object code format for direct processing by one or more processors, or stored in any other computer language, including scripts or sets of independent source code modules that are interpreted on demand or compiled in advance.

[0106] Figure 5This is a schematic block diagram illustrating a computer system 500 on which embodiments of the present disclosure may be implemented. The computer system 500 includes a bus 502 or other communication mechanism for transmitting information, and a processing means 504 coupled to the bus 502 for processing information. The computer system 500 also includes a memory coupled to the bus 502 for storing instructions to be executed by the processing means 504; the memory may be random access memory 506 or other dynamic storage device. The memory may also be used to store temporary variables or other intermediate information during the execution of instructions to be executed by the processing means 504. The computer system 500 also includes a read-only memory 508 or other static storage device coupled to the bus 502 for storing static information and instructions for the processing means 504. A storage device 510, such as a magnetic disk or optical disk, is provided and coupled to the bus 502 for storing information and instructions. The computer system 500 may be coupled via the bus 502 to an output device 512 for providing output to a user, such as, but not limited to, a display (such as a cathode ray tube (CRT) or liquid crystal display (LCD)), a speaker, etc. Input devices 514, such as a keyboard, mouse, and microphone, are coupled to bus 502 for transmitting information and command selections to processing device 504. Computer system 500 can execute embodiments of this disclosure. Consistent with certain implementations of this disclosure, computer system 500 provides results in response to processing device 504 executing one or more sequences of one or more instructions contained in memory. Such instructions may be read into memory from another computer-readable medium, such as storage device 510. Execution of the sequence of instructions contained in memory causes processing device 504 to perform the methods described herein. Alternatively, hard-wired circuitry may be used in place of or in combination with software instructions to implement the teachings. Therefore, implementations of this disclosure are not limited to any particular combination of hardware circuitry and software. In various embodiments, computer system 500 may be connected across a network via network interface 516 to one or more other computer systems, such as computer system 500, to form a networked system. This network may include a private network or a public network such as the Internet. In a networked system, one or more computer systems may store data and supply data to other computer systems. As used herein, the term "computer-readable medium" refers to any medium that participates in providing instructions to the processing device 504 for execution. Such media can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical discs or magnetic disks such as storage device 510. Volatile media include dynamic memory such as memory. Transmission media include coaxial cables, copper wires, and optical fibers, including wiring that includes bus 502.Common forms of computer-readable media or computer program products include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, or any other magnetic media, CD-ROMs, digital video discs (DVDs), Blu-ray discs, any other optical media, thumb drives, memory cards, RAM, PROMs and EPROMs, fast EPROMs, any other memory chips or cartridges, or any other tangible media from which a computer can read. Various forms of computer-readable media may be involved when carrying one or more sequences of one or more instructions to processing device 504 for execution. For example, instructions may initially be carried on a disk of a remote computer. The remote computer may load the instructions into its dynamic memory and transmit the instructions over a telephone line using a modem. A modem local to computer system 500 may receive data over a telephone line and convert the data into an infrared signal using an infrared transmitter. An infrared detector coupled to bus 502 may receive the data carried in the infrared signal and place the data on bus 502. Bus 502 carries the data to memory, and processing device 504 retrieves the instructions from memory and executes the instructions. Optionally, instructions received from the memory may be stored on the storage device 510 before or after execution by the processing device 504.

[0107] According to various embodiments, instructions configured to be executed by a processing device to perform a method are stored on a computer-readable medium. The computer-readable medium may be a device for storing digital information. For example, a computer-readable medium includes a compact disc read-only memory (CD-ROM) as known in the art for storing software. The computer-readable medium is accessed by a processor adapted to execute the instructions configured to be executed.

[0108] The foregoing has described one or more exemplary embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0109] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this disclosure does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0110] While one or more embodiments of this disclosure provide the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or terminal product execution, the methods shown in the embodiments or drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment).

[0111] The terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitation, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first" or "second" to denote names does not indicate any particular order.

[0112] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more embodiments of this disclosure, the functions of each module can be implemented in one or more software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0113] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0116] Those skilled in the art will understand that one or more embodiments of this disclosure may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0117] One or more embodiments of this disclosure can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can reside in local and remote computer storage media, including storage devices.

[0118] The same or similar parts between the various embodiments of this disclosure can be referred to mutually, and each embodiment focuses on describing the differences from other embodiments. In particular, for the apparatus embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to the description of the method embodiments. In the description of this disclosure, the descriptions of terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., mean that the specific feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of this disclosure. In this disclosure, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this disclosure and the features of the different embodiments or examples.

[0119] Additionally, when used in this disclosure, the terms “here,” “above,” “below,” “below,” “in the following,” “overall,” and similar terms should refer to the entirety of this disclosure and not any particular part thereof. Furthermore, unless expressly stated otherwise or otherwise understood in the context in which they are used, conditional language used herein, such as “may,” “possibly,” “for example,” “like,” etc., is generally intended to express that certain embodiments include, while other embodiments do not, certain features, elements, and / or states. Therefore, such conditional language is not generally intended to imply that one or more embodiments require features, elements, and / or states in any way, or whether such features, elements, and / or states are included or performed in any particular embodiment.

[0120] The above description is merely an embodiment of one or more embodiments of this disclosure and is not intended to limit the scope of the one or more embodiments of this disclosure. Various modifications and variations can be made to the one or more embodiments of this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims.

Claims

1. A point cloud matching method, characterized in that, include: Obtain a template point cloud of the target and a first feature descriptor of the template point cloud; Obtain multiple preprocessed scene point clouds for the target, the multiple preprocessed scene point clouds having different scales for feature extraction; For each preprocessed scene point cloud, a first operation is performed, the first operation including: Based on the scale of the preprocessed scene point cloud, a second feature descriptor for the preprocessed scene point cloud is determined. Based on the first feature descriptor, the second feature descriptor, and a distance threshold, a first evaluation index and a second evaluation index are determined for the preprocessed scene point cloud. The first evaluation index indicates the feature distance between the template point cloud and the preprocessed scene point cloud, and the second evaluation index indicates the three-dimensional geometric error between the template point cloud and the preprocessed scene point cloud. A comprehensive evaluation index for the preprocessed scene point cloud is determined based on the first evaluation index, the second evaluation index, and the weighting coefficients used for the second evaluation index; and Based on comprehensive evaluation indicators, a preprocessed scene point cloud is selected from the multiple preprocessed scene point clouds and matched with the template point cloud. The determination of the first and second evaluation metrics of the preprocessed scene point cloud based on the first feature descriptor, the second feature descriptor, and the distance threshold includes: The interior point set and transformation matrix are determined based on the first feature descriptor, the second feature descriptor and the distance threshold. The interior point set includes multiple interior point pairs, and each interior point pair includes a template point and a corresponding scene point. The first evaluation index is determined based on the set of interior points; and The second evaluation index is determined based on the interior point set and the transformation matrix. Wherein, the first evaluation index is determined based on the feature distance between the first feature descriptor of the template point in each inner point pair and the second feature descriptor of the corresponding scene point; and / or the second evaluation index is determined based on the spatial distance between the three-dimensional coordinates of the template point in each inner point pair in the inner point set after transformation by the transformation matrix and the three-dimensional coordinates of the corresponding scene point.

2. The point cloud matching method according to claim 1, characterized in that, The first feature descriptor is a first fast point feature histogram descriptor, and the second feature descriptor is a second fast point feature histogram descriptor.

3. The point cloud matching method according to claim 1, characterized in that, Each preprocessed scene point cloud is obtained by preprocessing the initial scene point cloud based on a second operation, which includes: Based on the environmental quality score of the initial scene point cloud, a parameter strategy for the initial scene point cloud is determined. This parameter strategy includes preprocessing parameters for the initial scene point cloud, which include at least one of statistical filter standard deviation factor, voxel downsampling size, and expected point cloud quality. The initial scene point cloud is preprocessed based on the preprocessing parameters.

4. The point cloud matching method according to claim 3, characterized in that, The second operation also includes: The height range of the target and the height of the target's center point are determined based on the initial scene point cloud. The target's height range is updated based on the target's height range, the target's center point height, and the filtering coefficient; and The initial scene point cloud is filtered based on the updated height range of the target.

5. The point cloud matching method according to claim 3, characterized in that, The parameter strategy also includes scale and distance thresholds.

6. The point cloud matching method according to claim 5, characterized in that, At least one of the following must be met: In response to the environmental quality score being not less than the first environmental quality score, the parameter strategy is determined to be the first parameter strategy; In response to the environmental quality score being not less than the second environmental quality score and less than the first environmental quality score, the parameter strategy is determined to be the second parameter strategy; In response to the environmental quality score being less than the second environmental quality score, the parameter strategy is determined to be a third parameter strategy. in, The first environmental quality score is greater than the second environmental quality score. The scale in the first parameter strategy is smaller than the scale in the second parameter strategy, and the scale in the second parameter strategy is smaller than the scale in the third parameter strategy. The distance threshold in the first parameter strategy is less than the distance threshold in the second parameter strategy, and the distance threshold in the second parameter strategy is less than the distance threshold in the third parameter strategy.

7. The point cloud matching method according to claim 5, characterized in that, The plurality of preprocessed scene point clouds include a first preprocessed scene point cloud, a second preprocessed scene point cloud, and a third preprocessed scene point cloud, wherein the scale of the first preprocessed scene point cloud is the scale in the parameter strategy, the scale of the second preprocessed scene point cloud is smaller than the scale in the parameter strategy, and the scale of the third preprocessed scene point cloud is larger than the scale in the parameter strategy.

8. The point cloud matching method according to claim 3, characterized in that, The environmental quality score is determined based on at least one of the target's light intensity, vibration intensity, and surface reflectivity.

9. A robotic arm control method based on point cloud matching, characterized in that, include: Obtain a template point cloud of the target and a first feature descriptor of the template point cloud; Obtain multiple preprocessed scene point clouds for the target, the multiple preprocessed scene point clouds having different scales for feature extraction; For each preprocessed scene point cloud pair, a first operation is performed, the first operation including: Based on the scale of the preprocessed scene point cloud, a second feature descriptor for the preprocessed scene point cloud is determined. Based on the first feature descriptor, the second feature descriptor, and a distance threshold, a first evaluation index and a second evaluation index are determined for the preprocessed scene point cloud. The first evaluation index indicates the feature distance between the template point cloud and the preprocessed scene point cloud, and the second evaluation index indicates the three-dimensional geometric error between the template point cloud and the preprocessed scene point cloud. A comprehensive evaluation index for the preprocessed scene point cloud is determined based on the first evaluation index, the second evaluation index, and the weighting coefficients used for the second evaluation index; and Based on the comprehensive evaluation index, a preprocessed scene point cloud is selected from the multiple preprocessed scene point clouds and a template point cloud is matched to obtain the transformed three-dimensional coordinates of the target. Based on the transformed three-dimensional coordinates of the target, the robotic arm is controlled to complete the task related to the target. The determination of the first and second evaluation metrics of the preprocessed scene point cloud based on the first feature descriptor, the second feature descriptor, and the distance threshold includes: The interior point set and transformation matrix are determined based on the first feature descriptor, the second feature descriptor and the distance threshold. The interior point set includes multiple interior point pairs, and each interior point pair includes a template point and a corresponding scene point. The first evaluation index is determined based on the set of interior points; and The second evaluation index is determined based on the interior point set and the transformation matrix. Wherein, the first evaluation index is determined based on the feature distance between the first feature descriptor of the template point in each inner point pair and the second feature descriptor of the corresponding scene point; and / or the second evaluation index is determined based on the spatial distance between the three-dimensional coordinates of the template point in each inner point pair in the inner point set after transformation by the transformation matrix and the three-dimensional coordinates of the corresponding scene point.

10. An electronic device, characterized in that, include: processor; as well as A memory storing computer-executable instructions, which, when executed by the processor, cause the processor to perform the point cloud matching method according to any one of claims 1 to 8 or the point cloud matching-based robotic arm control method according to claim 9.

11. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that, When executed by a computer, the computer-executable instructions cause the computer to perform the point cloud matching method according to any one of claims 1 to 8 or the robotic arm control method based on point cloud matching according to claim 9.

12. A computer program product, characterized in that, The computer program product includes instructions that, when executed by a processor, implement the point cloud matching method according to any one of claims 1 to 8 or the robotic arm control method based on point cloud matching according to claim 9.

Images