Cylinder detection method and electronic device combined with point cloud registration

Abstract

The present invention provides a cylinder detection method and electronic device combined with point cloud registration, which registers the model point cloud and the collected point cloud of the target workpiece, and converts the model point cloud to the robot base coordinate system. The collected point cloud is plane-detected, and a first reference plane and an effective plane are determined. The first reference plane is the plane closest to the second reference plane in the model point cloud that is parallel to the cylindrical axis, and the effective plane is the plane located above the first reference plane. The collected point cloud is divided into multiple local point clouds based on the first reference plane and the effective plane. For each local point cloud, the normal vector of the first reference plane is used as the axial direction of the cylinder to perform cylinder detection to obtain the cylinder. In this solution, cylinder detection is performed based on point cloud registration. Even when the height of the cylindrical surface in the point cloud is very small, the cylinder can be stably and correctly detected. Compared with the method of the prior art that requires axis detection based on a voting mechanism, it is more stable and accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional point cloud processing, and in particular to a cylinder detection method and electronic equipment combined with point cloud registration. Background Art

[0002] In the existing point cloud cylinder detection algorithm, the plane in the point cloud is usually detected first, and after removing the points on the plane in the point cloud, cylinder detection is performed based on the remaining points. When detecting the cylinder, the axis of the cylinder is detected by voting, and then the points in the point cloud are projected onto a plane perpendicular to the axis of the cylinder, and circle detection is performed on the plane to detect the cylinder.

[0003] The existing point cloud cylinder detection algorithm is more suitable for detecting cylindrical surfaces with a certain height. However, if the height of the cylindrical surface is too small, the detection of the cylinder axis will be unstable, which will lead to unstable cylinder detection. Specifically, in the existing point cloud cylinder detection algorithm, when the height of the cylindrical surface is too small, due to the small number of points on the cylindrical surface, incorrect voting will occur during the cylinder axis voting detection, resulting in the failure to detect the cylinder, or the detection of an incorrect cylinder. Summary of the invention

[0004] The objectives of the present invention include, for example, providing a cylinder detection method and electronic device combined with point cloud registration, which can stably and correctly detect the cylinder.

[0005] The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, the present invention provides a cylinder detection method combined with point cloud registration, the method comprising:

[0007] Obtaining a model point cloud and a collection point cloud of a target workpiece, and registering the model point cloud and the collection point cloud to transform the model point cloud into a robot base coordinate system;

[0008] Performing plane detection on the collected point cloud, and determining a first reference plane and a valid plane in the detected plane, wherein the first reference plane is a plane closest to a second reference plane in the model point cloud, a normal vector of the second reference plane is parallel to an axial direction of a cylinder in the model point cloud, and the valid plane is a plane located above the first reference plane;

[0009] Dividing the collected point cloud into a plurality of local point clouds based on the first reference plane and the effective plane;

[0010] For each of the local point clouds, cylinder detection is performed based on the normal vector of the first reference surface as the axial direction of the cylinder to obtain the cylinder.

[0011] In an optional embodiment, the step of registering the model point cloud and the collected point cloud to convert the model point cloud into a robot base coordinate system includes:

[0012] The model point cloud and the collected point cloud are registered to obtain a transformation matrix from a model coordinate system to a robot base coordinate system, wherein the transformation matrix includes a rotation matrix and a translation vector;

[0013] Determine a second reference surface in the model point cloud, and obtain a normal vector and a distance parameter of the second reference surface, wherein the normal vector of the second reference surface is parallel to the axial direction of the cylinder in the model point cloud;

[0014] The normal vector and distance parameter of the second reference plane are transformed based on the rotation matrix and the translation vector to transform the second reference plane into the robot base coordinate system.

[0015] In an optional implementation, before the step of performing plane detection on the collected point cloud, the method further includes:

[0016] Calculating the centroid of the collected point cloud, and calculating the difference vector between the centroid and the shooting point;

[0017] Calculating an inner product between the transformed normal vector of the second reference surface and the difference vector;

[0018] When the inner product is less than 0, the transformed normal vector and distance parameter of the second reference surface are reversely processed.

[0019] In an optional implementation, the step of determining a first reference plane in the detected plane includes:

[0020] For each of the detected planes, based on the normal vector and distance parameter of the plane and the normal vector and distance parameter of the converted second reference plane, calculate the distance between the plane and the second reference plane;

[0021] A plane corresponding to the minimum distance among the detected planes is determined as the first reference plane.

[0022] In an optional embodiment, the step of determining a valid plane among the detected planes includes:

[0023] For each detected plane, obtaining a normal vector and a centroid point of the plane;

[0024] Based on the normal vector, the center of mass point of the plane and the normal vector and the distance parameter of the first reference plane, it is determined whether the plane is above the first reference plane. If it is determined that the plane is above the first reference plane, the plane is determined to be a valid plane.

[0025] In an optional implementation, the step of determining whether the plane is located above the first reference plane based on the normal vector, the centroid point of the plane and the normal vector and the distance parameter of the first reference plane comprises:

[0026] Calculating the angle between the normal vector of the plane and the normal vector of the first reference surface, calculating the inner product of the centroid point and the normal vector of the first reference surface, and calculating the difference between the inner product and the distance parameter of the first reference surface;

[0027] If the angle is smaller than the preset angle and the difference is greater than 0, it is determined that the plane is located above the first reference surface.

[0028] In an optional implementation, the step of dividing the collected point cloud into a plurality of local point clouds based on the first reference plane and the effective plane includes:

[0029] Arranging the first reference plane and the effective plane in order based on their respective distance parameters;

[0030] For each two adjacent planes of the arrangement, obtain the normal vector and distance parameter of each of the two adjacent planes;

[0031] Points located above a previous plane and below a subsequent plane in the two adjacent planes are screened out from the collected point cloud, and the screened out points are used as a group of local point clouds.

[0032] In an optional implementation, the step of filtering out points in the collected point cloud that are located above a previous plane and below a subsequent plane of the two adjacent planes includes:

[0033] For each point in the collected point cloud, calculate the inner product of the point and the normal vector of a previous plane of the two adjacent planes, and calculate a first difference between the inner product and the distance parameter of the previous plane;

[0034] Calculating the inner product of the point and the normal vector of the latter plane of the two adjacent planes, and calculating a second difference between the inner product and the distance parameter of the latter plane;

[0035] If the first difference is greater than 0 and the second difference is less than 0, it is determined that the point is located above the previous plane and below the next plane.

[0036] In an optional embodiment, the step of performing cylinder detection based on the normal vector of the first reference surface as the axial direction of the cylinder to obtain the cylinder includes:

[0037] Constructing a projection plane perpendicular to the normal vector of the first reference plane;

[0038] Projecting each point in the local point cloud onto the projection surface to obtain a plurality of projection points;

[0039] Performing circle detection based on the multiple projection points;

[0040] A cylinder is detected based on the detected circle and the normal vector of the first reference surface.

[0041] In a second aspect, the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, and when the processor executes the computer program, the steps of the method described in any one of the aforementioned implementation modes are implemented.

[0042] The beneficial effects of the embodiments of the present invention include, for example:

[0043] The present invention provides a cylinder detection method and electronic device combined with point cloud registration. After obtaining the model point cloud and the collected point cloud of the target workpiece, the model point cloud and the collected point cloud are registered to convert the model point cloud to the robot base coordinate system. The collected point cloud is plane-detected, and the first reference plane and the effective plane are determined. The first reference plane is the plane closest to the second reference plane in the model point cloud that is parallel to the axial direction of the cylinder, and the effective plane is the plane located above the first reference plane. The collected point cloud is divided into multiple local point clouds based on the first reference plane and the effective plane. For each local point cloud, the normal vector of the first reference plane is used as the axial direction of the cylinder to perform cylinder detection to obtain the cylinder. In this solution, cylinder detection is performed based on point cloud registration. Even when the height of the cylindrical surface in the point cloud is very small, the cylinder can be stably and correctly detected. Compared with the method of axis detection based on the voting mechanism in the prior art, it is more stable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 A flow chart of a cylinder detection method combined with point cloud registration provided by an embodiment of the present invention;

[0046] Figure 2 for Figure 1 A flowchart of the sub-steps included in S11;

[0047] Figure 3 A schematic diagram of collecting point clouds provided by an embodiment of the present invention;

[0048] Figure 4 for Figure 1 A flowchart of the sub-steps included in S12;

[0049] Figure 5 for Figure 1 Another flow chart of the sub-steps included in S12;

[0050] Figure 6 for Figure 1 A flowchart of the sub-steps included in S13;

[0051] Figure 7 for Figure 6 A flowchart of the sub-steps included in S133;

[0052] Figure 8 for Figure 1 A flowchart of the sub-steps included in S14;

[0053] Fig. 9 A functional module block diagram of a cylinder detection device combined with point cloud registration provided by an embodiment of the present invention;

[0054] Fig.10 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0056] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0057] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0058] In the description of the present invention, it should be noted that the terms “first”, “second”, etc., if used, are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0059] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.

[0060] See also Figure 1 , is a flow chart of a cylinder detection method combined with point cloud registration provided by an embodiment of the present invention, the cylinder detection method combined with point cloud registration can be performed by a cylinder detection device combined with point cloud registration, the cylinder detection device combined with point cloud registration can be implemented by software and / or hardware, and can be configured in an electronic device, the electronic device can be a computer device, a server, a programmable logic controller, etc. The detailed steps of the cylinder detection method combined with point cloud registration are introduced as follows.

[0061] S11, obtaining a model point cloud and a collection point cloud of a target workpiece, and registering the model point cloud and the collection point cloud to transform the model point cloud into a robot base coordinate system.

[0062] S12, performing plane detection on the collected point cloud, and determining a first reference plane and a valid plane in the detected plane, wherein the first reference plane is a plane closest to a second reference plane in the model point cloud, a normal vector of the second reference plane is parallel to an axial direction of a cylinder in the model point cloud, and the valid plane is a plane located above the first reference plane.

[0063] S13: dividing the collected point cloud into multiple local point clouds based on the first reference plane and the effective plane.

[0064] S14: For each of the local point clouds, a normal vector of the first reference surface is used as the axis of the cylinder to perform cylinder detection to obtain a cylinder.

[0065] In this embodiment, the target workpiece is a workpiece with a cylinder, wherein the height of the cylindrical surface of the cylinder may be relatively small. The model point cloud of the target workpiece is the point cloud included in the model of the target workpiece constructed in the software tool, and the acquisition point cloud is the point cloud obtained by photographing the target workpiece with a camera, wherein the camera is set in the robot base coordinate system, and the acquisition point cloud can be understood as the point cloud in the robot base coordinate system, for example, the acquisition point cloud obtained by photographing is the point cloud in the camera coordinate system, and the acquisition point cloud in the camera coordinate system is converted into the point cloud in the robot base coordinate system.

[0066] The point cloud registration algorithm is used to register the model point cloud and the collected point cloud, so as to obtain the transformation matrix from the model coordinate system to the robot base coordinate system. Based on the transformation matrix, the model point cloud can be transformed into the robot base coordinate system.

[0067] In addition, plane detection is performed on the collected point cloud, and multiple planes can be detected. The plane detection algorithm used is not limited, and can be, for example, the Hough plane detection algorithm. A first reference plane is determined from the detected planes, and the first reference plane in the collected point cloud is determined based on the second reference plane in the model point cloud. Specifically, the second reference plane in the model point cloud is a plane perpendicular to the axis of the cylinder in the model point cloud, that is, the normal vector of the second reference plane is parallel to the axial direction of the cylinder in the model point cloud. Furthermore, the second reference plane can intersect with the cylindrical surface of the cylinder in the model point cloud to form an arc.

[0068] On this basis, the first reference plane in the collected point cloud is the plane closest to the second reference plane among the planes detected by the collected point cloud.

[0069] In addition, an effective plane is determined from the planes in the collected point cloud, and the effective plane is a plane located above the first reference plane. The so-called above means that the distance between the centroid of the effective plane and the shooting point is closer than the distance between the centroid of the first reference plane and the shooting point. The shooting point refers to the position of the optical center of the camera in the robot base coordinate system when the camera shoots the point cloud.

[0070] The determined effective plane may be the first one or multiple ones, and the effective plane is approximately parallel to the first reference plane. In this way, the first reference plane and the effective plane can divide the collected point cloud into multiple parts, each of which can be used as a local point cloud. The purpose of using the first reference plane and the effective plane to divide the collected point cloud here is to divide the points on the cylindrical surface in the collected point cloud. Since the points on the cylindrical surface in the collected point cloud are located above the first reference plane, there are points on the cylindrical surface between the first reference plane and an effective plane located above the first reference plane.

[0071] After the above segmentation, one of the local point clouds is a point cloud corresponding to the cylindrical surface, so cylinder detection can be performed on each local point cloud to obtain the cylinder. When performing cylinder detection, the normal vector of the first reference surface is used as the axial direction of the cylinder, that is, the direction vector of the detected axis of the cylinder is parallel to the normal vector of the first reference surface.

[0072] The cylinder detection method combined with point cloud registration provided in this embodiment performs cylinder detection based on point cloud registration. Even when the height of the cylindrical surface in the point cloud is very small, the cylinder can be detected stably and correctly. Compared with the existing method of requiring axis detection based on a voting mechanism, the method is more stable and accurate.

[0073] In this embodiment, the above-mentioned conversion of the model point cloud mainly converts the second reference surface in the model point cloud. For details, please refer to Figure 2 , the model point cloud is converted to the robot base coordinate system, which can be achieved by:

[0074] S111, registering the model point cloud and the collected point cloud to obtain a transformation matrix from the model coordinate system to the robot base coordinate system, wherein the transformation matrix includes a rotation matrix and a translation vector.

[0075] S112, determining a second reference surface in the model point cloud, and obtaining a normal vector and a distance parameter of the second reference surface, wherein the normal vector of the second reference surface is parallel to the axial direction of the cylinder in the model point cloud.

[0076] S113: transform the normal vector and distance parameter of the second reference plane based on the rotation matrix and the translation vector to transform the second reference plane into the robot base coordinate system.

[0077] In this embodiment, by registering the model point cloud and the collected point cloud, the rotation matrix R and the translation vector T from the model coordinate system to the robot base coordinate system can be obtained.

[0078] Since the information of the cylinder in the model point cloud is known, the axial information of the cylinder in the model point cloud, that is, the direction vector of the axis, can be obtained. When multiple planes in the model point cloud are obtained, a plane whose normal vector is parallel to the axial direction of the cylinder and intersects with the cylindrical surface of the cylinder is determined from the multiple planes as the second reference plane. Figure 3 For the point cloud shown in , the reference surface is the plane pointed by the arrow.

[0079] Get the normal vector of the second reference surface n ref and distance parameters ρ ref , where the distance parameter refers to the distance between the plane and the coordinate origin. Based on the rotation matrix and translation vector obtained above, the normal vector and distance parameter of the second reference plane are transformed respectively.

[0080] The normal vector of the second reference surface is transformed as follows:

[0081]

[0082] The distance parameters of the second reference surface are converted as follows:

[0083]

[0084] in, They are respectively the normal vector and distance parameters of the second reference surface transformed to the robot base coordinate system.

[0085] Through the above method, the second reference surface can be converted from the model coordinate system to the robot base coordinate system.

[0086] Since point cloud registration has high accuracy, after the model point cloud is converted based on the above method, the reference surface in the model point cloud is very close to the corresponding reference surface in the acquisition point cloud. However, considering that there may be a reverse situation, the method provided in this embodiment may further include the following steps based on the above:

[0087] Calculate the centroid point of the collected point cloud and the difference vector between the centroid point and the shooting point; calculate the inner product between the transformed normal vector of the second reference surface and the difference vector; when the inner product is less than 0, reversely process the transformed normal vector and distance parameter of the second reference surface.

[0088] In this embodiment, Figure 3 The leftmost point in the image can represent the shooting point. The difference vector between the centroid of the collected point cloud and the shooting point is from the centroid to the shooting point. The shooting point is represented by TCP and the centroid is m Q When , the difference vector between the two is expressed as p m, tcp = TCP - m Q .

[0089] The normal vector of the second reference surface of the model point cloud after transformation is: n trans , the normal vector can be calculated n trans The inner product between the normal vector and the difference vector. If the inner product is less than 0, it means that the direction of the normal vector is opposite to the direction of the difference vector. n trans In the reverse direction, at the same time, the transformed distance parameters of the second reference surface are required ρ trans Do the reverse.

[0090] Based on the above, plane detection is performed on the collected point cloud to obtain multiple planes, and then the first reference plane is determined from the multiple planes. Figure 4 In this embodiment, the first reference plane in the detected plane can be determined by:

[0091] S121, for each detected plane, based on the normal vector and distance parameter of the plane and the normal vector and distance parameter of the converted second reference plane, calculate the distance between the plane and the second reference plane.

[0092] S122: Determine a plane corresponding to the minimum distance among the detected planes as a first reference plane.

[0093] In this embodiment, for each plane, the distance between the two can be calculated based on the normal vector and distance parameter of the plane and the normal vector and distance parameter of the second reference plane according to the following formula:

[0094]

[0095] in, and Respectively represent the first i The normal vector and distance parameters of the plane.

[0096] When implementing, you can initialize the minimum distance min_dis = 1e10 and the plane index min_dis_index = -1. Use the variable i to traverse the planes in the collected point cloud.

[0097] For the traversed plane i , get the plane i The normal vector n i and distance parameters ρ i ,calculate n i and n trans If the inner product is less than 0, it means that the normal vector of the plane n i Normal vector to the second reference surface n trans In the opposite direction, the plane needs to be i The normal vector n i Reverse, and for the plane i The distance parameter ρ i Do the reverse.

[0098] Then, based on the above calculation formula, the plane is calculated i In this embodiment, the normal vector and the distance parameter of the plane are considered simultaneously in the calculation formula, and the calculation result is more accurate.

[0099] When the distance between two planes is very close, the normal vectors between the planes are very close, that is, the inner product between the normal vectors is large, and the distance parameters between the planes are very close, that is, the difference between the distance parameters is small. Therefore, the closer the distance between the two planes is, the smaller the difference between the distance parameters of the two planes is after the above formula is introduced, and the larger the inner product of the normal vectors of the two planes is, the smaller the final result is, and the result is consistent with the actual distance relationship between the planes.

[0100] In this embodiment, the distance is obtained by calculating the difference between the distance parameters of the two planes and then dividing it by the inner product of the two planes. The calculated distance has a high accuracy.

[0101] When calculating the distance for each plane, if the calculated distance is less than the current minimum distance, the current minimum distance is updated to the calculated distance, and min_dis_index = i. That is, the current minimum distance is the minimum distance calculated so far.

[0102] After traversing all planes, the plane with the smallest distance among all planes can be obtained. This plane is the plane closest to the transformed second reference plane, and this plane is the first reference plane in the collected point cloud.

[0103] Based on the determination of the first reference plane, determine the effective plane from multiple planes of the collected point cloud. Figure 5 In this embodiment, determining the valid plane among the detected planes can be achieved by:

[0104] S123, for each detected plane, obtaining the normal vector and the centroid point of the plane.

[0105] S124, based on the normal vector, centroid point and normal vector and distance parameter of the plane and the first reference plane, determine whether the plane is above the first reference plane; if the plane is determined to be above the first reference plane, determine that the plane is a valid plane.

[0106] In this embodiment, when judging whether the plane is located above the first reference plane based on the normal vector of the plane, the centroid point, the normal vector of the first reference plane, and the distance parameter, the judgment can be made in the following manner:

[0107] Calculate the angle between the normal vector of the plane and the normal vector of the first reference plane, calculate the inner product of the centroid point and the normal vector of the first reference plane, and calculate the difference between the inner product and the distance parameter of the first reference plane; if the angle is less than the preset angle and the difference is greater than 0, it is determined that the plane is above the first reference plane.

[0108] In the specific implementation, after the first reference plane is determined, the first reference plane is stored in planes_seg. i Traverse each plane and judge i Is it equal to min_dis_index, where min_dis_index represents the number of the first reference surface. If yes, skip the plane directly; if not, get the plane i The normal vector and center of mass of . And determine whether the following two conditions are met:

[0109] , and the angle between the normal vector ni of plane i and the normal vector n0 of the first reference plane is less than a preset angle. represents the center of mass of plane i.

[0110] If these two conditions are met, it is determined that plane i is located above the first reference plane, and plane i is approximately parallel to the first reference plane, and plane i can be determined to be a valid plane. The valid plane is stored in planes_seg. After traversing all planes, all valid planes can be found.

[0111] After determining the first reference surface and all valid planes in the collected point cloud, the collected point cloud can be segmented using the first reference surface and valid planes, thereby segmenting the cylindrical surface separately. Figure 6 In this embodiment, this step can be implemented in the following way:

[0112] S131: Arrange the first reference plane and the effective plane in order based on their respective distance parameters.

[0113] S132: For every two adjacent arranged planes, obtain normal vectors and distance parameters of the two adjacent planes.

[0114] S133, filtering out points in the collected point cloud that are located above the first plane and below the second plane of the two adjacent planes, and taking the filtered points as a group of local point clouds.

[0115] As can be seen from the above, the first reference plane and all valid planes are saved in planes_seg. First, the normal vectors of each valid plane can be adjusted to make the direction of the normal vector of each plane consistent with the direction of the normal vector of the first reference plane. In other words, the inner product between the normal vector of each valid plane and the normal vector of the first reference plane can be calculated. If the inner product is less than 0, the normal vector and distance parameter of the valid plane can be reversed.

[0116] Then, the first reference plane and all valid planes may be arranged in order, for example, they may be arranged in order from small to large according to their respective distance parameters. In this way, the first reference plane and all valid planes are arranged in a certain order.

[0117] See also Figure 7 , when filtering the points in the collected point cloud that are above the first plane and below the second plane in two adjacent planes, this can be achieved in the following ways:

[0118] S1331, for each point in the collected point cloud, calculate the inner product of the point and the normal vector of the previous plane of the two adjacent planes, and calculate a first difference between the inner product and the distance parameter of the previous plane.

[0119] S1332, calculating the inner product of the point and the normal vector of the latter plane of the two adjacent planes, and calculating the second difference between the inner product and the distance parameter of the latter plane.

[0120] S1333: If the first difference is greater than 0 and the second difference is less than 0, determine that the point is above the previous plane and below the next plane.

[0121] In specific implementation, variable j is used to traverse each plane (including the first reference plane and all valid planes) to obtain the normal vector of the jth plane n j and distance parameters ρ j and the normal vector of the j+1th plane n j+1 and distance parameters ρ j+1 Then, the points located above the jth plane and below the j+1th plane are selected from the collected point cloud, and the selected points constitute a group of local point clouds. For example, if the number of the first reference plane and the valid plane is 3, the number of groups of local point clouds is 2; if the number of the first reference plane and the valid plane is 4, the number of groups of local point clouds is 3.

[0122] In this embodiment, the variable k can be used to traverse each point in the collected point cloud. p k, x First, we can judge p k, x Whether it is a non-number, that is, whether the point has been segmented into a local point cloud. For example, if it has been segmented into a local point cloud, the point can be marked. If it is a non-number, the point can be skipped directly. If it is not a non-number, the following two conditions can be judged: ,and , that is, the difference between the product of the point and the normal vector of the j-th plane and the distance parameter of the j-th plane is greater than 0, and the difference between the product of the point and the normal vector of the j+1-th plane and the distance parameter of the j+1-th plane is less than 0.

[0123] If the above two conditions are met, it means that the point p k Located above plane j and below plane j+1, the point can be stored in the jth local point cloud of seg_i, and p k, x = Non-number, indicating that the point has been stored in a local point cloud.

[0124] After traversing all points in the collected point cloud, the jth local point cloud that has been segmented can be obtained. After traversing all valid planes, the multiple local point clouds that have been segmented can be obtained.

[0125] Since the points on the cylindrical surface are located above the first reference surface and between the first reference surface and a valid plane located above the first reference surface, by segmenting the point cloud, one of the local point clouds includes the points on the cylindrical surface.

[0126] Therefore, cylinder detection can be performed on each local point cloud to obtain the cylinder. Figure 8 In this embodiment, this step can be implemented in the following way:

[0127] S141, constructing a projection plane perpendicular to the normal vector of the first reference plane.

[0128] S142, projecting each point in the local point cloud onto the projection surface to obtain a plurality of projection points.

[0129] S143: Perform circle detection based on the multiple projection points.

[0130] S144: Detect and obtain a cylinder based on the detected circle and the normal vector of the first reference surface.

[0131] In this embodiment, the normal vector of the first reference surface can be obtained, and the normal vector of the first reference surface is used as the normal vector, and a projection surface is constructed through a certain point, such as the origin of the coordinate system. Since the first reference surface corresponds to the second reference surface in the model point cloud, and the second reference surface is perpendicular to the axis of the cylinder in the model point cloud, the constructed projection surface is perpendicular to the axis of the cylinder in the acquisition point cloud.

[0132] Project the points in the local point cloud onto the projection surface. If the local point cloud includes a cylindrical surface, the projection points obtained by projecting the points on the cylindrical surface onto the projection surface will form a circle.

[0133] Therefore, the projected points on the projection surface can be subjected to circle detection to obtain the detected circle, which may specifically include information such as the center position and radius of the circle. Based on the detected circle and the axis of the cylinder, the cylinder in the collected point cloud can be determined.

[0134] The cylinder detection method combined with point cloud registration provided in this embodiment combines point cloud registration information, first converts the model point cloud to the robot base coordinate system, then detects the reference surface in the collected point cloud, and determines the effective plane, and uses the reference surface and the effective plane to segment the collected point cloud to segment the points on the cylindrical surface into a local point cloud. Finally, the normal vector of the reference surface is used as the direction vector of the cylinder axis to detect the cylinder.

[0135] In this solution, the point cloud registration method is combined to realize cylinder detection, and the cylinder can be stably and correctly detected even when there is a cylindrical surface with a very small height in the point cloud.

[0136] Based on the same inventive concept, please refer to Fig. 9 , showing a schematic diagram of functional modules of a cylinder detection device combined with point cloud registration provided in an embodiment of the present invention. This embodiment can divide the functional modules of the cylinder detection device combined with point cloud registration according to the above method embodiment. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic, which is only a logical functional division, and there may be other division methods in actual implementation.

[0137] For example, when each functional module is divided into corresponding functional modules, Fig. 9 The cylinder detection device combined with point cloud registration shown is only a schematic diagram of the device. The cylinder detection device combined with point cloud registration can include an acquisition module, a plane detection module, a segmentation module and a cylinder detection module. The functions of each functional module of the cylinder detection device combined with point cloud registration are described in detail below.

[0138] An acquisition module, used for acquiring a model point cloud and a collection point cloud of a target workpiece, and registering the model point cloud and the collection point cloud to transform the model point cloud into a robot base coordinate system;

[0139] A plane detection module, used to perform plane detection on the collected point cloud, and determine a first reference plane and a valid plane in the detected plane, wherein the first reference plane is a plane closest to a second reference plane in the model point cloud, a normal vector of the second reference plane is parallel to an axial direction of a cylinder in the model point cloud, and the valid plane is a plane located above the first reference plane;

[0140] A segmentation module, configured to segment the collected point cloud into a plurality of local point clouds based on the first reference plane and the effective plane;

[0141] The cylinder detection module is used to detect each of the local point clouds based on the normal vector of the first reference surface as the axial direction of the cylinder to obtain the cylinder.

[0142] It can be understood that the above-mentioned acquisition module, plane detection module, segmentation module and cylinder detection module can be used to execute the above-mentioned S11 to S14. The detailed implementation method of the acquisition module, plane detection module, segmentation module and cylinder detection module can refer to the relevant contents of the above-mentioned S11 to S14.

[0143] In a possible implementation, the above-mentioned obtaining module may be used to:

[0144] The model point cloud and the collected point cloud are registered to obtain a transformation matrix from a model coordinate system to a robot base coordinate system, wherein the transformation matrix includes a rotation matrix and a translation vector;

[0145] Determine a second reference surface in the model point cloud, and obtain a normal vector and a distance parameter of the second reference surface, wherein the normal vector of the second reference surface is parallel to the axial direction of the cylinder in the model point cloud;

[0146] The normal vector and distance parameter of the second reference plane are transformed based on the rotation matrix and the translation vector to transform the second reference plane into the robot base coordinate system.

[0147] In a possible implementation, the detection module may also be used to:

[0148] Calculating the centroid of the collected point cloud, and calculating the difference vector between the centroid and the shooting point;

[0149] Calculating an inner product between the transformed normal vector of the second reference surface and the difference vector;

[0150] When the inner product is less than 0, the transformed normal vector and distance parameter of the second reference surface are reversely processed.

[0151] In a possible implementation, the detection module may be specifically used for:

[0152] For each of the detected planes, based on the normal vector and distance parameter of the plane and the normal vector and distance parameter of the converted second reference plane, calculate the distance between the plane and the second reference plane;

[0153] A plane corresponding to the minimum distance among the detected planes is determined as the first reference plane.

[0154] In a possible implementation, the detection module may be specifically used for:

[0155] For each detected plane, obtaining a normal vector and a centroid point of the plane;

[0156] Based on the normal vector, the center of mass point of the plane and the normal vector and the distance parameter of the first reference plane, it is determined whether the plane is above the first reference plane. If it is determined that the plane is above the first reference plane, the plane is determined to be a valid plane.

[0157] In a possible implementation, the detection module may be specifically used for:

[0158] Calculating the angle between the normal vector of the plane and the normal vector of the first reference surface, calculating the inner product of the centroid point and the normal vector of the first reference surface, and calculating the difference between the inner product and the distance parameter of the first reference surface;

[0159] If the angle is smaller than the preset angle and the difference is greater than 0, it is determined that the plane is located above the first reference surface.

[0160] In a possible implementation, the above segmentation module may be used to:

[0161] Arranging the first reference plane and the effective plane in order based on their respective distance parameters;

[0162] For each two adjacent planes of the arrangement, obtain the normal vector and distance parameter of each of the two adjacent planes;

[0163] Points located above a previous plane and below a subsequent plane in the two adjacent planes are screened out from the collected point cloud, and the screened out points are used as a group of local point clouds.

[0164] In a possible implementation, the segmentation module may be specifically used for:

[0165] For each point in the collected point cloud, calculate the inner product of the point and the normal vector of a previous plane of the two adjacent planes, and calculate a first difference between the inner product and the distance parameter of the previous plane;

[0166] Calculating the inner product of the point and the normal vector of the latter plane of the two adjacent planes, and calculating a second difference between the inner product and the distance parameter of the latter plane;

[0167] If the first difference is greater than 0 and the second difference is less than 0, it is determined that the point is located above the previous plane and below the next plane.

[0168] In a possible implementation, the plane detection module may be specifically used for:

[0169] Constructing a projection plane perpendicular to the normal vector of the first reference plane;

[0170] Projecting each point in the local point cloud onto the projection surface to obtain a plurality of projection points;

[0171] Performing circle detection based on the multiple projection points;

[0172] A cylinder is detected based on the detected circle and the normal vector of the first reference surface.

[0173] See also Fig.10 , is a block diagram of an electronic device provided in an embodiment of the present invention, the electronic device may be a computer device, etc., and the electronic device includes a memory, a processor, and a communication module. The memory, the processor, and the communication module are electrically connected to each other directly or indirectly to achieve data transmission or interaction. For example, these components can be electrically connected to each other via one or more communication buses or signal lines.

[0174] The memory is used to store computer programs or data. The memory can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0175] The processor is used to read / write data or programs stored in the memory, and execute the cylinder detection method combined with point cloud registration provided by any embodiment of the present invention.

[0176] The communication module is used to establish a communication connection between the electronic device and other communication terminals through the network, and is used to send and receive data through the network.

[0177] It should be understood that Fig.10 The structure shown is only a schematic diagram of the structure of the electronic device. The electronic device may also include Fig.10 More or fewer components as shown, or with Fig.10 Different configurations are shown.

[0178] Furthermore, an embodiment of the present invention also provides a computer-readable storage medium, which stores machine-executable instructions. When the machine-executable instructions are executed, the cylinder detection method combined with point cloud registration provided in the above embodiment is implemented.

[0179] Specifically, the computer-readable storage medium can be a general storage medium, such as a mobile disk, a hard disk, etc. When the computer program on the computer-readable storage medium is executed, the above-mentioned cylinder detection method combined with point cloud registration can be executed. Regarding the process involved when the computer-readable storage medium and its executable instructions are executed, reference can be made to the relevant description in the above-mentioned method embodiment, which will not be described in detail here.

[0180] In summary, the cylinder detection method and electronic device combined with point cloud registration provided by the embodiment of the present invention, after obtaining the model point cloud and the collected point cloud of the target workpiece, align the model point cloud and the collected point cloud to convert the model point cloud to the robot base coordinate system. Plane detection is performed on the collected point cloud, and the first reference plane and the effective plane are determined. The first reference plane is the plane closest to the second reference plane in the model point cloud that is parallel to the axis of its cylinder, and the effective plane is the plane located above the first reference plane. Based on the first reference plane and the effective plane, the collected point cloud is divided into multiple local point clouds. For each local point cloud, the normal vector of the first reference plane is used as the axis of the cylinder to perform cylinder detection to obtain the cylinder. In this solution, cylinder detection is performed based on point cloud registration. Even when the height of the cylindrical surface in the point cloud is very small, the cylinder can be stably and correctly detected. Compared with the method of axis detection based on the voting mechanism in the prior art, it is more stable and accurate.

[0181] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A cylinder detection method combined with point cloud registration, It is characterized in that The method comprises: Obtaining a model point cloud and a collection point cloud of a target workpiece, and registering the model point cloud and the collection point cloud to transform the model point cloud into a robot base coordinate system; Performing plane detection on the collected point cloud, and determining a first reference plane and a valid plane in the detected plane, wherein the first reference plane is a plane closest to a second reference plane in the model point cloud, a normal vector of the second reference plane is parallel to an axial direction of a cylinder in the model point cloud, and the valid plane is a plane located above the first reference plane; Dividing the collected point cloud into a plurality of local point clouds based on the first reference plane and the effective plane; For each of the local point clouds, a normal vector of the first reference surface is used as the axis of the cylinder to perform cylinder detection to obtain a cylinder; The step of converting the model point cloud into the robot base coordinate system includes: The model point cloud and the acquisition point cloud are registered to obtain a transformation matrix from the model coordinate system to the robot base coordinate system, wherein the transformation matrix includes a rotation matrix and a translation vector; a second reference plane in the model point cloud is determined, and a normal vector and a distance parameter of the second reference plane are obtained, wherein the normal vector of the second reference plane is parallel to the axial direction of the cylinder in the model point cloud; the normal vector and the distance parameter of the second reference plane are transformed based on the rotation matrix and the translation vector to transform the second reference plane to the robot base coordinate system; The step of determining a first reference surface in the detected plane comprises: For each of the detected planes, based on the normal vector and distance parameter of the plane and the normal vector and distance parameter of the converted second reference plane, the distance between the plane and the second reference plane is calculated according to the following formula: in, and They represent the first i The normal vector and distance parameters of the plane, are respectively the normal vector and distance parameter of the second reference surface transformed to the robot base coordinate system, and the distance parameter refers to the distance between the plane and the coordinate origin; A plane corresponding to the minimum distance among the detected planes is determined as the first reference plane.

2. The cylinder detection method combined with point cloud registration according to claim 1, It is characterized in that Before the step of performing plane detection on the collected point cloud, the method further includes: Calculating the centroid of the collected point cloud, and calculating the difference vector between the centroid and the shooting point; Calculating an inner product between the transformed normal vector of the second reference surface and the difference vector; When the inner product is less than 0, the transformed normal vector and distance parameter of the second reference surface are reversely processed.

3. The cylinder detection method combined with point cloud registration according to claim 1, It is characterized in that The step of determining a valid plane among the detected planes comprises: For each detected plane, obtaining a normal vector and a centroid point of the plane; Based on the normal vector, the center of mass point of the plane and the normal vector and the distance parameter of the first reference plane, it is determined whether the plane is above the first reference plane. If it is determined that the plane is above the first reference plane, the plane is determined to be a valid plane.

4. The cylinder detection method combined with point cloud registration according to claim 3, It is characterized in that The step of judging whether the plane is located above the first reference plane based on the normal vector, the centroid point of the plane and the normal vector and the distance parameter of the first reference plane comprises: Calculating the angle between the normal vector of the plane and the normal vector of the first reference surface, calculating the inner product of the centroid point and the normal vector of the first reference surface, and calculating the difference between the inner product and the distance parameter of the first reference surface; If the angle is smaller than the preset angle and the difference is greater than 0, it is determined that the plane is located above the first reference surface.

5. The cylinder detection method combined with point cloud registration according to claim 1, It is characterized in that The step of dividing the collected point cloud into a plurality of local point clouds based on the first reference plane and the effective plane comprises: Arranging the first reference plane and the effective plane in order based on their respective distance parameters; For each two adjacent planes of the arrangement, obtain the normal vector and distance parameter of each of the two adjacent planes; Points located above a former plane and below a latter plane in the two adjacent planes are screened out from the collected point cloud, and the screened out points are used as a group of local point clouds.

6. The cylinder detection method combined with point cloud registration according to claim 5, It is characterized in that The step of filtering out points in the collected point cloud that are located above the first plane and below the second plane of the two adjacent planes includes: For each point in the collected point cloud, calculate the inner product of the point and the normal vector of a previous plane of the two adjacent planes, and calculate a first difference between the inner product and the distance parameter of the previous plane; Calculating the inner product of the point and the normal vector of the latter plane of the two adjacent planes, and calculating a second difference between the inner product and the distance parameter of the latter plane; If the first difference is greater than 0 and the second difference is less than 0, it is determined that the point is located above the previous plane and below the next plane.

7. The cylinder detection method combined with point cloud registration according to claim 1, It is characterized in that The step of performing cylinder detection based on the normal vector of the first reference surface as the axial direction of the cylinder to obtain the cylinder comprises: Constructing a projection plane perpendicular to the normal vector of the first reference plane; Projecting each point in the local point cloud onto the projection surface to obtain a plurality of projection points; Performing circle detection based on the multiple projection points; A cylinder is detected based on the detected circle and the normal vector of the first reference surface.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images