A viewpoint planning method for part feature contours based on clustering segmentation

Abstract

This invention discloses a viewpoint planning method for part feature contours based on clustering and segmentation, belonging to the field of 3D measurement. This method obtains feature contour clusters through clustering and segmentation, formulates more targeted viewpoint generation strategies, avoids measuring irrelevant areas, and reduces measurement time. Simultaneously, the clustering process effectively integrates feature contours, reduces the number of viewpoints generated, and improves the utilization rate of each viewpoint, thereby increasing overall measurement efficiency. When calculating the measurable area of ​​each viewpoint, the overall measurability of the feature contour is initially judged based on the measurability constraints of the binocular camera and projector, reducing subsequent computational load. Then, the feature contour is translated along the opposite direction of the normal, and each point of the translated contour is accurately calculated based on the measurability constraints of the binocular camera and projector, eliminating the problem of misjudgment of measurability caused by self-occlusion such as wall thickness, ensuring good measurement results when measuring different features.

Description

Technical Field

[0001] This invention belongs to the field of three-dimensional measurement, and more specifically, relates to a viewpoint planning method for part feature contours based on clustering segmentation. Background Technology

[0002] Features such as circular holes, openings, and outer contours play a crucial role in ensuring assembly accuracy and structural integrity, and are essential for guaranteeing the overall quality of parts. Precise three-dimensional measurement of these features not only ensures the functional reliability of parts but also provides valuable data for optimizing manufacturing processes, minimizing material waste, and reducing production costs.

[0003] Structured light technology boasts numerous advantages, including non-contact measurement, high precision, rapid processing, and high resolution, making it an ideal choice for automated 3D measurement of industrial parts. It can quickly and accurately acquire 3D information of object surfaces and is widely used in automotive manufacturing, aerospace, and machining. However, when measuring feature contours, different contours may require different viewpoints to obtain their complete information. This means that measurements need to be taken in multiple directions to ensure the integrity of the reconstruction, easily leading to an increase in the number of viewpoints and a decrease in overall measurement efficiency. For example, suppose a part has 300 contour features such as circular holes and threaded holes. Each contour requires information from 3 different angles to be fully acquired; manual planning might require as many as 300 viewpoints.

[0004] To address the aforementioned issues, existing researchers have attempted to automatically generate viewpoints and paths based on part models to improve measurement efficiency, employing various automatic viewpoint generation techniques such as point clouds, triangular meshes, uniform rational B-splines, and voxels. These techniques allow for the automatic analysis of the part model to generate viewpoints and paths, reducing manual intervention and improving measurement accuracy and efficiency. However, while these methods improve viewpoint planning efficiency to some extent, the following problems still exist:

[0005] (1) Existing viewpoint planning methods typically aim at the overall coverage of the part, lacking refined viewpoint planning for feature contours. Therefore, when inspecting key features in industrial settings, targeted planning is difficult, limiting measurement efficiency and accuracy. Especially when feature contours are complex or have minute geometric details, existing methods fail to fully consider the specific requirements of these features, often failing to accurately plan the optimal viewpoint, thus requiring additional viewpoints to supplement measurements or wasting significant time in irrelevant areas. This not only leads to inefficiency in the measurement process but also affects the accuracy and reliability of the inspection.

[0006] (2) Existing viewpoint planning methods typically treat the contour and freeform surface as a whole to calculate the measurable area, neglecting the self-occlusion of the feature contour surface and the special characteristics of feature extraction. This can easily lead to inaccurate judgment of the measurability of the feature contour, resulting in unstable viewpoint planning results. For example, in thin-walled parts, the wall thickness is often ignored. When using binocular structured light for measurement, some areas may only be able to measure the upper surface and not the wall thickness area, resulting in inaccurate data acquisition. For deep holes in thick-walled parts, conventional methods require the use of a large number of triangular facets to fit the wall thickness area to ensure the capture of minute features. However, due to the influence of factors such as baseline, complete measurement is not possible, making it difficult to judge the measurability of features.

[0007] Therefore, there is an urgent need for a fast and automatic viewpoint planning method for feature contours, which can accurately calculate the measurable area of ​​the feature contour and improve the efficiency and stability of viewpoint planning. Summary of the Invention

[0008] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides an automatic viewpoint planning method for part feature contours based on clustering segmentation. This solves the technical problems of existing viewpoint planning methods, which mostly plan for the overall three-dimensional structure and coverage of the part, resulting in low efficiency in feature contour viewpoint planning, inaccurate judgment of feature contour measurability, and poor stability of viewpoint planning results.

[0009] To achieve the above objectives, according to a first aspect of the present invention, a viewpoint planning method for part feature contours based on clustering segmentation is provided, applied to a surface structured light 3D measurement system including left and right cameras and a projector, the method comprising:

[0010] S1, calculate the geometric centroid of each feature contour to be planned for the part, and take the feature contour to be planned corresponding to the geometric centroid closest to the center point of the three-dimensional model of the part as the current cluster;

[0011] S2, determine whether there is a feature contour to be planned that meets the current clustering conditions. If so, cluster it into the current cluster and update the current cluster and the current clustering conditions. Otherwise, proceed to S3.

[0012] The current clustering conditions include: the distance between the geometric centroid of the feature contour and the geometric centroid of the current cluster is less than a distance threshold; the angle between the normals of the feature contour and the current cluster is less than an angle threshold; and after the feature contour is clustered into the current cluster, the length, width, and height of the minimum bounding cube of the cluster are less than the length, width, and front and rear depth distances in the optimal depth range of the left or right camera, respectively.

[0013] S3, take the feature profile to be planned that is farthest from the geometric centroid of the current cluster as the new cluster, update the current cluster, return to S2, until all feature profiles to be planned are assigned to a cluster.

[0014] S4, each cluster is treated as a target cluster and processed to obtain q candidate viewpoint poses; the processing includes: uniformly generating q candidate viewpoint directions around the normal direction of the target cluster at a certain angle, and generating the poses of the corresponding q candidate viewpoints, determining their reachability based on the poses of each candidate viewpoint, and obtaining a set of reachable viewpoints;

[0015] S5: Calculate the measurable points of each reachable viewpoint as the target viewpoint, and obtain the optimal viewpoint set by characterizing the measurement coverage of each reachable viewpoint and planning the optimal measurement path.

[0016] The calculation includes: determining the visibility of each feature contour in the cluster to which the target viewpoint belongs under the target viewpoint; taking the visible feature contour as the original feature contour; translating each sampling point on the original feature contour along the opposite direction of the normal of the original feature contour to form a new sampling point; and taking the new sampling point visible under the target viewpoint as the measurable point of the target viewpoint.

[0017] According to a second aspect of the present invention, an electronic device is provided, comprising: a computer-readable storage medium and a processor;

[0018] The computer-readable storage medium is used to store executable instructions;

[0019] The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect.

[0020] According to a third aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to perform the method as described in the first aspect.

[0021] According to a fourth aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in the first aspect.

[0022] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0023] The method provided by this invention obtains feature contour clusters based on clustering segmentation. By analyzing the spatial correlation of each feature contour, the method achieves feature contour segmentation, thereby formulating a more targeted viewpoint generation strategy, avoiding the measurement of irrelevant areas, and reducing measurement time. At the same time, the clustering process effectively integrates feature contours, reduces the number of viewpoints generated, and improves the utilization rate of each viewpoint, thus improving the overall measurement efficiency. When calculating the measurable area of ​​each viewpoint, the visibility of the original feature contour is first determined by visibility constraints, and then the original feature contour is projected along the reverse direction of the normal, reducing the impact of excessively thin walls on the measurable area. Considering the influence of feature occlusion, the visible areas of each camera and projector are calculated, and the occlusion situation of the three is comprehensively judged to obtain a more accurate measurable area, further improving the measurement accuracy and the precision of feature reconstruction, ensuring good measurement results when measuring different features. Attached Figure Description

[0024] Figure 1 This is one of the flowcharts for a viewpoint planning method for part feature contours based on clustering segmentation provided in an embodiment of the present invention;

[0025] Figure 2 The second flowchart of the viewpoint planning method for part feature contours based on clustering segmentation provided in this embodiment of the invention;

[0026] Figure 3 This is a schematic diagram illustrating clustering and segmenting of feature contours according to an embodiment of the present invention;

[0027] Figure 4 This is a schematic diagram of candidate viewpoint poses generated based on the geometric centroid and normal of a cluster, provided in an embodiment of the present invention.

[0028] Figure 5 This is a schematic diagram of the visible area after calculating and excluding self-occlusion for a certain visible feature contour, provided by an embodiment of the present invention.

[0029] Figure 6 A flowchart of a random genetic algorithm provided in an embodiment of the present invention;

[0030] Figure 7 A schematic diagram of a weighted and directed node design provided for an embodiment of the present invention;

[0031] Figure 8 The optimal viewpoint and optimal motion path obtained by using a random genetic algorithm and a greedy algorithm are provided in the embodiments of the present invention.

[0032] Figure 9 The simulation test results are shown in the figure provided for the embodiments of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0034] This invention provides a viewpoint planning method for part feature contours based on clustering segmentation, applied to a surface structured light 3D measurement system. The surface structured light 3D measurement system includes left and right cameras and a projector. Typically, for ease of calculation, the parameters of the left and right cameras are identical, such as... Figure 1 As shown, the method includes:

[0035] S1. According to the detection requirements, obtain the feature contours of the part to be planned from the viewpoint, and calculate the geometric centroid of each feature contour to be planned. Take the feature contour corresponding to the geometric centroid that is closest to the center point of the three-dimensional model of the part as the current cluster.

[0036] S2, determine whether there is a feature contour in the feature contour to be planned that meets the current clustering conditions. If so, cluster it into the current cluster and update the current cluster and the current clustering conditions. Otherwise, proceed to S3.

[0037] The current clustering conditions include: the distance between the geometric centroid of the feature contour and the geometric centroid of the current cluster is less than a distance threshold; the angle between the normals of the feature contour and the current cluster is less than an angle threshold; and after the feature contour is clustered into the current cluster, the length, width, and height of the minimum bounding cube of the current cluster are less than the length, width, and front and rear depth distances in the optimal depth range of the left or right camera, respectively.

[0038] S3: Select the feature contour that is farthest from the geometric centroid of the current cluster from the feature contours to be planned, and update the current cluster accordingly. Return to S2, and continue until all feature contours to be planned are assigned to a cluster.

[0039] Specifically, the geometric centroid of each feature contour to be planned is calculated. The feature contour corresponding to the geometric centroid closest to the center point of the 3D model of the part is taken as the most central feature contour, and this most central feature contour is taken as the current cluster. Feature contours that meet the current clustering conditions are searched among the remaining feature contours to be planned, and they are added to the current cluster. Each time a new feature contour is added, the current cluster is updated, so the current clustering conditions need to be updated synchronously. Then, the feature contour farthest from the geometric centroid of the current cluster is selected from the remaining unassigned feature contours to be planned as the new current cluster, until all feature contours are assigned to a cluster, thus completing the clustering of feature contours.

[0040] In the STP 3D model of the part, the feature contour to be planned is selected. In the STP 3D model, the feature contour is represented by a B-spline curve. Therefore, the B-spline curve C(u) used for the feature contour is first discretized. For the feature contour L, the total number of sampling points N is:

[0041]

[0042] Where dp is the sampling resolution of the 3D measuring instrument, taken as 0.1 mm; l is the length of the feature contour L. This represents the magnitude of the feature contour tangent vector C′(u), where C′(u) is the first derivative of C(u); u min and u max This represents the boundary value of the range of values ​​for parameter u.

[0043] The geometric centroid G of the feature profile is:

[0044]

[0045] Among them, P i P represents the three-dimensional coordinates of each sampling point of the feature contour. i =(x i y i , z i ), i = 1, 2, ..., N.

[0046] Considering the limited camera area, larger features cannot be captured in one shot. Preferably, before step S1, the method further includes: uniformly dividing the feature contour to be planned that has more than the number of sampling points into multiple feature contours to be planned; the number of sampling points threshold is determined based on the planar size of the image acquired by the camera.

[0047] For example, if the sampling point threshold is set to 12000, then for feature contours L with a total sampling number N greater than 12000, the feature contours will be segmented as follows:

[0048]

[0049] Among them, L di For the di-th segment of the curve after segmenting the feature contour L, the sampling points P containing L are... i To facilitate differentiation, the following is adopted: express, Let represent the t-th sampling point of the di-th curve segment; U is the total number of segments, and the formula for calculating U is:

[0050]

[0051] The value of n is determined based on the width of the image captured by the camera; This indicates that the calculation result needs to be rounded up.

[0052] To facilitate determining the starting point of the segmentation, that is Select the point with the largest x-value among the sampling points of this feature contour, i.e., point (x max , y, z), as Then, the endpoint of the first curve L1 after dividing L is... Sampling points

[0053] It is understandable that curve L di+1 The starting point is curve L. di Based on the endpoint of the curve and the sampling points it contains, the centroid and length of each segmented curve can be recalculated. The calculation method is as follows:

[0054]

[0055] G di For L di The center of mass, l di For L di The length.

[0056] The current clustering conditions for the current cluster include the measurement range, normal angle, and contour size, specifically:

[0057]

[0058] g1 represents the measurement range in the clustering conditions. The distance between the geometric centroid G of the feature contour L to be assigned and the geometric centroid of the current cluster must be less than σ (the value of σ is set according to half the length of the photo image, for example: 300mm) to be judged as similar contours. g1 represents the geometric centroid of the current cluster; g2 represents the normal angle in the clustering condition. The angle between the normal of the feature contour L to be assigned and the normal of the current cluster must be less than α (the value of α is set according to the reconstruction accuracy requirements, for example: 10°), where v represents the normal of the feature contour. g3, g4, and g5 represent the normals of the current cluster, and g5 represent the contour dimensions in the clustering conditions. After adding a new feature contour to the current cluster, the minimum bounding cube of the new cluster is... length The length 'a' should be smaller than the camera's optimal depth of field range, for example: 600mm, and the width... The width (b) should be smaller than the camera's optimal depth of field range, for example: 600mm, and the height... It should be less than the camera's front and rear depth of field distance c, for example, 500mm.

[0059] Only if the feature contour L to be assigned meets the current clustering conditions of the current cluster can it be placed into the current cluster.

[0060] After adding the new feature contour, the length, width, and height of the minimum bounding cuboid of the new cluster are:

[0061]

[0062] Where, x′ max and x′ min The x-values ​​represent the maximum and minimum values ​​of x projected onto the plane along the opposite direction of the new cluster normal; y′ max and y′ min The z′ represents the maximum and minimum y-values ​​projected onto the plane along the opposite direction of the new cluster normal; max and z′ min This represents the maximum and minimum z-values ​​projected onto the normal.

[0063] Any point on the feature contour, projected along the reverse direction of the new cluster normal, lands at the origin O. o On the plane, the origin of the model is chosen as the origin O of the projection plane. o Its new coordinates can be represented as:

[0064]

[0065] Among them, P′ i and P i These represent the coordinates of the projected point and its corresponding original coordinates, respectively; e p This represents the unit normal vector of the feature contour. Based on this, the maximum difference along the x-axis and the maximum difference along the y-axis can be obtained, determining the length and width of the smallest circumscribed cuboid.

[0066] Specifically, the normal vector projected onto the cluster from any point on the feature contour can be expressed as:

[0067]

[0068] in, P represents the coordinates projected onto the normal vector. RC For the points on the projection plane mentioned above, select the origin O on the projection plane. o e c This represents the unit normal vector of the new cluster. Based on this, the maximum difference along the z-axis can be obtained, and the height of the smallest circumscribed cuboid can be determined.

[0069] After clustering and segmentation, the geometric centroid and normal of each cluster are calculated as follows:

[0070]

[0071] It is the centroid of the updated cluster. It is the normal of the new cluster after the update, where l sG is the length of the s-th feature contour in the cluster; s v represents the geometric centroid of the s-th feature contour in the cluster; s Let s be the normal to the s-th feature contour in the cluster. s = 1, 2, ..., ssum, where ssum represents the total number of feature contours in the cluster. The total number of feature contours in each cluster may be equal or unequal.

[0072] Furthermore, if there are unassigned feature contours, they can be clustered into the nearest cluster, specifically:

[0073]

[0074] Among them, G n For the centroid of an unassigned feature contour, if its distance to the centroid of a certain cluster is less than μ, for example, 200 mm, it can be clustered into that cluster, and the geometric centroid and normal vector of the cluster are updated again. If G n If none of the above conditions are met, an additional viewpoint needs to be generated for it, that is, it is treated as a cluster, the centroid of which is the centroid of the feature contour, and the normal is the normal of the feature contour.

[0075] Understandably, if the endpoint of an unclosed curve in a cluster is the starting point of another unclosed curve, they can be reconnected to facilitate subsequent calculations.

[0076] S4, each cluster is treated as a target cluster and processed to obtain q candidate viewpoint poses; the processing includes: uniformly generating q candidate viewpoint directions around the normal direction of the target cluster at a certain angle, and generating the poses of the corresponding q candidate viewpoints, determining their reachability based on the poses of each candidate viewpoint, and obtaining a set of reachable viewpoints.

[0077] Specifically, for each cluster obtained by S3, q candidate viewpoint directions are generated around it at a certain angle with its normal, and the poses of the corresponding q candidate viewpoints are generated.

[0078] The pose of the corresponding candidate viewpoint is generated based on the candidate viewpoint direction, using existing technology. For example, the optimal measurement distance is moved according to the generated candidate direction. Simultaneously, the line connecting the optical centers of the left and right cameras of the measuring head is perpendicular to the plane formed by the cluster normal and the viewpoint direction, generating the candidate viewpoint pose. The candidate viewpoint pose includes its position and orientation. The position is the position coordinate of the candidate viewpoint in the part's 3D coordinate system, and the orientation is the transformation matrix that converts the part's 3D coordinate system to the measuring equipment's coordinate system. The geometric centroid of the target cluster is moved by d along each candidate viewpoint direction. opt The poses of q candidate viewpoints can then be generated through calculation; d opt The optimal measuring distance for the measuring equipment.

[0079] The value of q is set according to the measurement precision requirements. The larger the q, the higher the measurement coverage, but the more complex the calculation.

[0080] The values ​​of the aforementioned included angles mainly rely on manual measurement experience, and are usually taken as 30°.

[0081] To further increase randomness and reduce the impact of unreachable motion mechanisms on viewpoint planning, preferably, in step S4, after uniformly generating q candidate viewpoint directions at a certain angle to the normal direction of the target cluster and before further generating the poses of the corresponding q candidate viewpoints, the method further includes:

[0082] Random Gaussian noise is applied to the direction of each candidate viewpoint.

[0083] Taking the angle between the generated viewpoint direction and the cluster normal as 30°, and generating 6 viewpoints in a wraparound manner as an example, this means that the angle between the viewpoint direction and the previous viewpoint direction is 60°. The angles between the viewpoint direction after applying Gaussian noise and the cluster normal and the previous viewpoint direction are:

[0084]

[0085] Where θ represents the angle between the viewpoint direction after applying Gaussian noise and the normal to the cluster; ε represents the angle between the viewpoint direction after applying Gaussian noise and the previous viewpoint direction; θ * With ε * Let θ and ε be the angle between them after applying Gaussian random noise; let ∈ be a random variable whose probability density function follows a standard normal distribution.

[0086]

[0087] Then, the geometric centroid of the cluster is moved along the optimal measurement distance d. opt Its value can be determined based on the measuring equipment used (each measuring equipment has its own optimal measuring distance d). opt In this embodiment of the invention, the diameter is 650mm. Simultaneously, the line connecting the optical centers of the left and right cameras of the measuring head is perpendicular to the plane formed by the normal of the cluster and the viewpoint direction. This means that in the measuring instrument coordinate system, the line connecting the optical centers of the left and right cameras is the X-axis, the viewpoint direction is the Z-axis, and the Y-axis is... Figure 4 As shown, candidate viewpoint poses are generated. Then, all candidate viewpoints are stored in the candidate viewpoint set.

[0088] The accessibility of each candidate viewpoint in the candidate viewpoint set is determined based on the type of the actual motion mechanism (i.e., the mechanism that drives the camera movement). The corresponding inverse kinematic parameters are calculated, and reachable viewpoints are initially selected from the candidate viewpoints and stored in the reachable viewpoint set VP = {V i}

[0089] Those skilled in the art will know that the accessibility of a viewpoint can be determined using IKFast or other similar inverse kinematics solvers to determine whether the viewpoint pose is reachable. For example, if the motion mechanism is a single robotic arm, IKFast can be used directly to solve whether the viewpoint pose is reachable. If it is a dual-arm or multi-arm robotic arm, the viewpoint needs to be segmented according to its spatial position, and the inverse kinematics parameters need to be solved for the viewpoint corresponding to each robotic arm.

[0090] S5: Calculate the measurable points of each reachable viewpoint as the target viewpoint, and obtain the optimal viewpoint set by characterizing the measurement coverage of each reachable viewpoint and planning the optimal measurement path.

[0091] The calculation includes: determining the visibility of each feature contour in the cluster to which the target viewpoint belongs under the target viewpoint; taking the visible feature contour as the original feature contour; translating each sampling point on the original feature contour along the opposite direction of the normal of the original feature contour to form a new sampling point; and taking the new sampling point visible under the target viewpoint as the measurable point of the target viewpoint.

[0092] Specifically, each reachable viewpoint is traversed. First, based on the visibility constraints of each reachable viewpoint, the visibility of the original feature contour sampling points in its cluster is determined. In order to accurately calculate the measurable area of ​​the wall thickness, the visible original feature contour is translated a certain distance in the opposite direction of its normal. Then, based on the occlusion conditions, the measurable area of ​​the new feature contour is calculated.

[0093] Specifically, the visibility constraints for each reachable viewpoint adopt the following three conventional constraints:

[0094] 1) Field of view constraint. The centroid of the feature profile must be within the field of view of the left and right cameras and projectors at the viewpoint location. The field of view of each device can be represented as:

[0095]

[0096] Where c is the camera optical axis direction vector; v is the direction vector from the centroid of the feature contour to the camera center in the camera coordinate system; v xz v yz These are the projection vectors of v onto the XZ and YZ planes, respectively; θ h and θ v These are the horizontal and vertical field of view angles, with values ​​of 35° and 25° respectively.

[0097] 2) Depth of field constraints. The centroid of the feature profile must be within the depth of field measured by the left and right cameras and the projector at the viewpoint. The depth of field constraint for a single device can be expressed as:

[0098]

[0099] in, , respectively, represent the minimum and maximum distances for the camera or projector to achieve clear imaging; d is the distance from the optical center of the camera or projector to its centroid.

[0100] 3) Occlusion constraints mainly fall into two categories: The first is when other objects exist between the measured object and the object being measured, causing occlusion during shooting; the second is when other parts of the measured object itself obstruct the part to be photographed. The first situation is generally avoided in advance in industrial production, while the second situation requires separate occlusion checks for the left and right cameras to ensure both cameras can capture the measured object. Using parametric equations for single-camera occlusion checks, assuming point P is the centroid of the original feature contour and point O is the camera's optical center, then line segment PO can be written as the equation:

[0101] p(k)=k·P+(1-k)·O(0≤k≤1)

[0102] Next, the model is meshed to obtain triangular facets. Therefore, assuming the three vertices of a certain facet in the target model are a, b, and c, the interior point p(m, n) of the facet can be expressed mathematically as:

[0103] p(m,n)=m·a+n·b+(1-mn)·c(0≤m,n≤1)

[0104] Combining the above two equations, we can obtain:

[0105] k·P+(1-k)·O=m·a+n·b+(1-mn)·c

[0106] If the equation has a solution, it means that line segment PO intersects with surface patch Δabc, that is, there is occlusion between the camera and the centroid P; otherwise, there is no occlusion phenomenon.

[0107] It is understandable that the camera optical axis direction and camera optical center in the above constraints can be obtained by rotating and translating the pose of the reachable viewpoint through the measurement instrument's own parameters. That is, the position of the reachable viewpoint is translated a certain distance along the X-axis to obtain the camera optical center, and the Z-axis is rotated a certain angle around the Y-axis to obtain the camera optical axis direction. The above distances and angles are obtained through the measurement instrument's own parameters.

[0108] After performing the above visibility judgment, the feature contours that are not visible from the reachable viewpoint can be identified. When calculating the measurable wall thickness area from the reachable viewpoint in the subsequent calculation, the calculation of the sampling points of the feature contours that are not visible from the reachable viewpoint is skipped to speed up the calculation.

[0109] Each reachable viewpoint is used as the target viewpoint for calculation to obtain the measurable points under the target viewpoint.

[0110] To accurately calculate the measurable area of ​​the wall thickness from the target viewpoint, the visible sampling points in the original feature contour from the target viewpoint are translated a certain distance in the opposite direction of its normal to form new sampling points. Specifically:

[0111]

[0112] in, This represents the three-dimensional coordinates after translation, i.e., the new sampling point; P i The value of dis represents the three-dimensional coordinates of the i-th sampling point on the original feature contour; the value of dis can be obtained according to the sampling resolution of the measuring device. In this embodiment, it is 0.3mm to identify 4 layers of point cloud data; e represents the unit normal vector of the feature contour.

[0113] Then, by using the visible area of ​​the new feature contour under the target viewpoint, the measurable area of ​​the wall thickness region can be deduced. However, due to the occlusion of the original feature contour, it is necessary to calculate the visible area of ​​the new feature contour under the target viewpoint based on the occlusion conditions (this visible area consists of new sampling points visible under the target viewpoint, and the new sampling points visible under the target viewpoint are the measurable points under the target viewpoint, therefore). When calculating the occlusion, calculations need to be performed for the left and right cameras and the projector, specifically:

[0114]

[0115] Among them, S lc P represents the set of new sampling points in the visible area of ​​the left camera. lc S represents the coordinates of each new feature contour (i.e., the new sampling point) in the visible area of ​​the left camera; rc P represents the set of new sampling points within the visible area of ​​the right camera. rc S represents the coordinates of the new feature contours within the visible area of ​​the right camera; p P represents the set of new sampling points within the projectable area of ​​the projector. p Let n be the coordinates of each new feature contour in the projectible region. lc n rc n p The total number of points in the point sets of the visible areas of the left camera, right camera, and projector, respectively.

[0116] Taking the acquisition of coordinates of new feature contours in the visible area of ​​the left camera as an example, let's assume... Point P is the new sampling point, and point O is the optical center of the left camera. Then, line segment P... i O can be written as an equation:

[0117]

[0118] The equation of the plane containing the original feature contour can be determined by three randomly selected sampling points A, B, and C on the contour:

[0119] a(x-x0)+b(y-y0)+c(z-z0)=0

[0120] Where (a,b,c) are vectors and The cross product of x, y, and z; x0, y0, and z0 are the coordinates of point A.

[0121] Solving the above equations simultaneously, we can obtain the line segment. The intersection point PI of the plane containing the original feature contour i The cross-number method is used to determine whether the intersection point is inside the original feature contour. If the intersection point is inside the original feature contour, then the new sampling point is... As seen in the left camera, specifically:

[0122]

[0123] in, This indicates the visibility of the new sampling point; a value of 1 indicates it is visible in the left camera, and a value of 0 indicates it is not visible in the left camera; g(PI i ) represents the intersection point PI i The geometric centroid G of the original feature contour s The formed PI rays i G s The number of intersections with the original feature contour. If the value is odd, the intersection point is inside the original feature contour; if the value is even, the intersection point is outside the original feature contour.

[0124] Understandably, when calculating the segmented feature contour, it is necessary to consider the occlusion of the viewpoint under the unsegmented condition and calculate the intersection of the contour surface before segmentation.

[0125] By iterating through all new sampling points, the final result is S. lc That is, the intersection point PI i New sampling points within the original feature contour gather:

[0126]

[0127] Understandably, for reachable viewpoint V i (i.e., the target viewpoint) For a new sampling point to be considered visible from the left and right cameras and the projector, it must be determined that the wall thickness region of the feature is not obscured by itself at that viewpoint. In other words, the new sampling point is visible and measurable at the target viewpoint. Therefore, at the reachable viewpoint V... i Below, the length of the feature contour in the visible region composed of the newly visible sampling points. for:

[0128]

[0129] At the reachable viewpoint V i Below, the total number of new sampling points in the visible region, which consists of the visible new sampling points. for:

[0130]

[0131] in, express The total number, r = 1, 2, ..., R, where R is the total number of measurable points under the target viewpoint.

[0132] Those skilled in the art will understand that the objective of selecting the optimal viewpoint from the reachable viewpoints is to minimize the required number of reachable viewpoints and maximize the measurement coverage of the reachable viewpoints. In step S5, the measurement coverage of the reachable viewpoints is characterized by the ratio of the measurable points of the reachable viewpoint to all sampling points of all feature contours in its cluster;

[0133] Alternatively, it can be characterized by the sum of the lengths of the feature profiles formed by all adjacent measurable points among the measurable points of the reachable viewpoint.

[0134] Selecting the optimal viewpoint from the reachable viewpoints can be achieved using existing methods such as Markov decision-making or intelligent algorithms. Once the optimal viewpoint set is obtained, existing greedy algorithms or machine learning algorithms can be used to optimize the robot's motion path.

[0135] The following example illustrates how a random genetic algorithm is used to select the optimal viewpoint from reachable viewpoints (where the measurement coverage is represented by the length of the visible feature contour), and then a greedy algorithm is used to optimize the robot's motion path.

[0136] A random genetic algorithm is used to select reachable viewpoints. First, the population size p is set to 150. Viewpoints are randomly selected and added to individuals. Then, the total visible area of ​​the individual is checked to see if it meets the measurement coverage requirement. If it does not meet the requirement, new viewpoints are added to the individual. The fitness value of the individual is calculated to provide a basis for subsequent genetic operations. New individuals are generated until the set population size is reached.

[0137] For an individual to meet the coverage condition, the length of the visible feature contour must be greater than a set value:

[0138]

[0139] in, For individual X, V i The length of the visible feature profile at the viewpoint; σ is the measurement coverage requirement, taken as 60%; Lsum The total length of the selected feature profile.

[0140] Alternatively, the coverage rate can be calculated using the number of new measurable sampling points:

[0141]

[0142] in, For individual X, V i New sampling points visible from the viewpoint (i.e., V) i Measurable points of view), R rsum The total number of samples for the selected feature contour.

[0143] Reachable viewpoints are selected using a random genetic algorithm, and the fitness function is calculated as follows:

[0144] f(X) = count

[0145] Where count is the number of viewpoints in individual X, and a low fitness value indicates that the individual is a high-quality solution.

[0146] Individuals evolve through genetic operations (including selection, crossover, and mutation). Selection employs an elitist strategy, meaning that individuals with a population proportion of ρ... e The elite individuals are directly replicated and passed on to the next generation. e The value is 0.1.

[0147] Crossover is an operation that randomly selects two individuals from the population and, through the exchange of their two chromosomes, passes on the superior traits of the parents to the offspring, thus creating new superior individuals. Specifically, one parent comes from the elite set and the other from the non-elite set. During crossover, the elite parent A and the non-elite parent B are selected according to the crossover probability ρ. c Crossover generates the next generation. Specifically, given parents A and B, the inheritance of genes from which parent is determined by simulating a skewed coin toss, where the probability of heads after the toss is ρ. c The probability of the opposite side appearing is 1-ρ c Assuming the i-th toss results in heads, the offspring inherits the i-th allele from parent A; otherwise, it inherits the i-th allele from parent B. Simultaneously, each time a new individual's genes are generated through crossover, the coverage requirement is calculated. If the coverage requirement is met, the current crossover operation ends; otherwise, the next gene generation continues. Each crossover generates one new individual. When the number of generated new individuals reaches p × (1 - ρ)... e -ρ m When the crossover operation stops, the crossover probability ρ is given. c The value is 0.7; ρ m This represents the percentage of mutated individuals in the population, with a value of 0.3.

[0148] The mutation operation follows the same pattern as the initial population in terms of individual generation, randomly generating p×ρ individuals. m A new individual replaces the same number of individuals with lower fitness values.

[0149] The solution process is as follows Figure 6 As shown. The number of evolutionary iterations is set to 400. An initial population of size p is first initialized. Simultaneously, the population individuals are calculated based on the fitness function. fitness value in, Let represent the i-th individual in the j-th generation population. Then, based on the fitness of the individuals in the j-th generation population... Perform ascending sorting and genetic operations. Once the required number of iterations is reached, output the individual with the lowest fitness value in the final generation as the optimal viewpoint set V. opt .

[0150] For the obtained optimal viewpoint set, the mathematical model for optimizing the robot's motion path using a greedy algorithm is as follows:

[0151]

[0152] Among them, V opt Represents the optimal viewpoint set; t i This indicates the measurement time required at this viewpoint; e(V) i V i+1 ) represents the viewpoint V i and viewpoint V i+1 The cost of movement between them; e(V) h V1) represents the initial position v h Motion cost between viewpoint v1 and viewpoint v1.

[0153] The MoveIt package in ROS is used to generate smooth and collision-free robot trajectories. After acquiring the robot trajectory between viewpoints, the robot's movement time on the trajectory, i.e., the movement cost between optimal viewpoints, can be calculated based on the set robot speed. This is achieved by designing a weighted and directed node graph data structure G(V... opt E) is used to store the above information, where V opt It stores information about all viewpoints; E contains edges e with direction and weight values. ij It stores V i To V j The robot trajectory and motion cost. It's worth noting that since the robot trajectory is directional, the trajectories in the two directions between two viewpoints need to be calculated and stored separately in the edges of the corresponding directions, such as... Figure 7 As shown.

[0154] Randomly select a viewpoint as the initial viewpoint. Based on the node graph, search for the viewpoint closest to it as the next starting point, and continue until all viewpoints are included in the motion path. Calculate the total motion cost.

[0155]

[0156] Among them, V j dm is the ordered set of viewpoints along this path; dm is the total number of viewpoints in the set.

[0157] Understandably, by traversing each viewpoint in the optimal viewpoint set as an initial viewpoint and calculating its motion cost, the robot's motion path is optimized by selecting the ordered viewpoint set with the minimum motion cost, thus achieving automatic viewpoint planning for the feature contours of complex parts.

[0158] This invention provides an electronic device, including: a computer-readable storage medium and a processor;

[0159] The computer-readable storage medium is used to store executable instructions;

[0160] The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any of the above embodiments.

[0161] This invention provides a computer-readable storage medium storing computer instructions that cause a processor to perform the method described in any of the above embodiments.

[0162] This invention provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the method described in any of the above embodiments.

[0163] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A viewpoint planning method for part feature contours based on clustering segmentation, applied to a surface structured light 3D measurement system including left and right cameras and a projector, characterized in that... The method includes: S1, calculate the geometric centroid of each feature contour to be planned for the part, and take the feature contour to be planned corresponding to the geometric centroid closest to the center point of the three-dimensional model of the part as the current cluster; S2, determine whether there is a feature contour to be planned that meets the current clustering conditions. If so, cluster it into the current cluster and update the current cluster and the current clustering conditions. Otherwise, proceed to S3. The current clustering conditions include: the distance between the geometric centroid of the feature contour and the geometric centroid of the current cluster is less than a distance threshold; the angle between the normals of the feature contour and the current cluster is less than an angle threshold; and after the feature contour is clustered into the current cluster, the length, width, and height of the minimum bounding cube of the cluster are less than the length, width, and front and rear depth distances in the optimal depth range of the left or right camera, respectively. S3, take the feature profile to be planned that is farthest from the geometric centroid of the current cluster as the new cluster, update the current cluster, return to S2, until all feature profiles to be planned are assigned to a cluster. S4, each cluster is treated as a target cluster and processed to obtain q candidate viewpoint poses; the processing includes: uniformly generating q candidate viewpoint directions around the normal direction of the target cluster at a certain angle, and generating the poses of the corresponding q candidate viewpoints, determining their reachability based on the poses of each candidate viewpoint, and obtaining a set of reachable viewpoints; S5: Calculate the measurable points of each reachable viewpoint as the target viewpoint, and obtain the optimal viewpoint set by characterizing the measurement coverage of each reachable viewpoint and planning the optimal measurement path. The calculation in S5 includes: determining the visibility of each feature contour in the cluster to which the target viewpoint belongs under the target viewpoint, taking the visible feature contour as the original feature contour, translating each sampling point on the original feature contour along the opposite direction of the normal of the original feature contour to form a new sampling point, and taking the new sampling point visible under the target viewpoint as the measurable point of the target viewpoint.

2. The method as described in claim 1, characterized in that, In step S5, the measurement coverage of the reachable viewpoint is characterized by the ratio of the measurable points of the reachable viewpoint to all sampling points of all feature contours in its cluster. Alternatively, it can be characterized by the sum of the lengths of the feature profiles formed by all adjacent measurable points among the measurable points of the reachable viewpoint.

3. The method as described in claim 1, characterized in that, In step S5, the new sampling points visible from the target viewpoint satisfy the following conditions: The number of intersections between the new sampling point and the optical center of the left or right camera, the intersection point PI of the new sampling point and the geometric centroid G of the original feature contour, and the ray PIG formed by these intersections is odd.

4. The method as described in claim 1, characterized in that, Before step S1, the method further includes: uniformly dividing the planned feature contour with more than the number of sampling points into multiple feature contours.

5. The method as described in claim 1, characterized in that, In step S4, after generating q candidate viewpoint directions uniformly around the normal direction of the target cluster at a certain angle, and before further generating the poses of the corresponding q candidate viewpoints, the method further includes: Random Gaussian noise is applied to the direction of each candidate viewpoint.

6. An electronic device, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in any one of claims 1-5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to perform the method as described in any one of claims 1-5.

8. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the method as described in any one of claims 1-5.

Images