Visual segmentation and positioning method of bolt heads in rolling mill

Abstract

The visual segmentation and positioning method of the bolt head of a rolling mill includes the following steps: Step 1, the original three-dimensional point cloud is sequentially subjected to through-filtering and voxel filtering to complete preprocessing; Step 2, the top plane point cloud of the target bolt head is segmented from the preprocessed point cloud; Step 3, the robot teaches the standard bolt with the same specifications as the target bolt, obtains the standard bolt pose matrix, visually collects and segments, obtains the top plane point cloud of the standard bolt head, and uses the top plane point cloud of the standard bolt head as the template bolt point cloud; Step 4, identifies and locates the pose of the target bolt head. This method realizes the precise segmentation of the top plane point cloud of the bolt head of the rolling mill according to the position characteristics of the bolts of the rolling mill, establishes a template bolt library according to the specifications and sizes of the bolts of the rolling mill, guides the robot to move in place, and has high positioning accuracy and fast speed.

Description

Technical Field

[0001] The invention relates to a three-dimensional point cloud segmentation and positioning method, in particular to a three-dimensional point cloud segmentation and positioning method for a bolt head of a steel rolling mill, and belongs to the technical field of image processing. Background Art

[0002] With the continuous development of the economy and society, my country's demand for bar products is increasing day by day, and the quality requirements for bars are getting higher and higher. This puts forward new requirements for the relevant equipment for producing bars, especially rolling mills. In the production of bars, the hole profile of the rolls will have different degrees of wear, and the roughness cannot meet the rolling requirements, resulting in the size of the rolled bars being out of tolerance, affecting the quality of the bars. Therefore, fast and timely roll replacement is of great significance to improving production efficiency and product quality. The time and workload of disassembling and assembling the bolts of the rolling mill can account for more than half of the work of changing the rolls. In the process of changing the rolls, the hexagonal bolts that fix the frame and the rolls must be removed first.

[0003] At present, the disassembly of rolling mill bolts worldwide mainly relies on semi-automatic mechanical equipment to assist manual work, which takes a long time, is inefficient, and has high labor intensity and low safety factor. It can be seen that the disassembly of rolling mill bolts needs to be automated and intelligent to replace manual operation. The automated disassembly of rolling mill bolts requires the introduction of robots, and guiding robots to intelligently identify and locate bolt heads requires the introduction of machine vision technology. Therefore, according to the industrial production environment, a segmentation and positioning method for the head image of rolling mill bolts is proposed to lay the foundation for the industrial application of automatic disassembly of rolling mill bolts. Summary of the invention

[0004] In order to solve the above technical problems, the present invention proposes a visual segmentation and positioning method for the head of a rolling mill bolt, which provides a technical basis for the automated disassembly of the rolling mill bolts.

[0005] The visual segmentation and positioning method of the bolt head of a rolling mill comprises the following steps:

[0006] Step 1: In the steel production preparation workshop environment, the hand-eye vision system is used to obtain the original three-dimensional point cloud of the rolling mill bolt part, which includes the casing point cloud around the rolling mill bolt, the overall point cloud of the rolling mill bolt head, and some noise points. The obtained original three-dimensional point cloud is sequentially subjected to through-filtering and voxel filtering to complete preprocessing;

[0007] Step 2: Segment the target bolt head top plane point cloud from the preprocessed point cloud. The specific steps are as follows: (1) Use a plane model-based random sampling consensus algorithm to fit and segment the point cloud of the bolt bottom casing, and delete the point cloud of the casing; (2) Cluster and segment the overall point cloud of the bolt head based on the specification size of the target bolt and the difference in Euclidean distance between each point; (3) Use a plane model-based random sampling consensus algorithm to fit and segment to obtain the bolt head top plane point cloud;

[0008] Step 3: The robot teaches a standard bolt with the same specifications as the target bolt, obtains the standard bolt pose matrix, collects and segments the standard bolt head top plane point cloud, and uses the standard bolt head top plane point cloud as the template bolt point cloud;

[0009] Step 4 is to identify and locate the target bolt head pose. The specific steps are as follows: (1) Use the principal component analysis method to roughly align the target bolt point cloud and the template bolt point cloud to obtain the initial rigid transformation matrix; (2) Use the iterative nearest neighbor method to accurately align the point cloud with the initial rigid transformation matrix to obtain the target bolt pose in the camera coordinate system; (3) Calculate the product of the target bolt pose matrix in the camera coordinate system and the standard bolt pose matrix to obtain the bolt pose in the robot base coordinate system.

[0010] Beneficial effects of the method of the present invention:

[0011] (1) According to the characteristics of the bolt position of the rolling mill, the three-dimensional point cloud of the bolt part of the rolling mill is collected. Based on the hexagonal bolt top plane characteristics shared by the bolt point clouds of different angles and specifications, the bolt as a whole and the bolt top plane are gradually separated to achieve accurate segmentation of the rolling mill bolt head top plane point cloud;

[0012] (2) A template bolt library is established according to the different specifications and sizes of rolling mill bolts. When locating the target bolt, the transformation matrix is ​​obtained by aligning it with the template bolts of the same specifications, and then the bolt position in the robot base coordinate system is calculated, which is convenient for guiding the robot to move into place, with high positioning accuracy and fast speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 Schematic diagram of the structure of the robot hand-eye vision system used in the method of the present invention;

[0014] Figure 2 It is a flow chart of the visual segmentation and positioning method of the bolt head of a rolling mill according to the present invention. DETAILED DESCRIPTION

[0015] Combined with Figure 1-2 The system structure and operation steps of the present invention are described.

[0016] The visual segmentation and positioning method of the bolt head of a rolling mill comprises the following steps:

[0017] Step 1: In the steel production preparation workshop environment, the hand-eye vision system is used to obtain the original three-dimensional point cloud of the rolling mill bolt part, which includes the casing point cloud around the rolling mill bolt, the overall point cloud of the rolling mill bolt head, and some noise points. The obtained original three-dimensional point cloud is sequentially subjected to through-filtering and voxel filtering to complete preprocessing;

[0018] Step 2: Segment the target bolt head top plane point cloud from the preprocessed point cloud. The specific steps are as follows: (1) Use a plane model-based random sampling consensus algorithm to fit and segment the point cloud of the bolt bottom casing, and delete the point cloud of the casing; (2) Cluster and segment the overall point cloud of the bolt head based on the specification size of the target bolt and the difference in Euclidean distance between each point; (3) Use a plane model-based random sampling consensus algorithm to fit and segment to obtain the bolt head top plane point cloud;

[0019] Step 3: The robot teaches a standard bolt with the same specifications as the target bolt, obtains the standard bolt pose matrix, collects and segments the standard bolt head top plane point cloud, and uses the standard bolt head top plane point cloud as the template bolt point cloud;

[0020] Step 4 is to identify and locate the target bolt head pose. The specific steps are as follows: (1) Use the principal component analysis method to roughly align the target bolt point cloud and the template bolt point cloud to obtain the initial rigid transformation matrix; (2) Use the iterative nearest neighbor method to accurately align the point cloud with the initial rigid transformation matrix to obtain the target bolt pose in the camera coordinate system; (3) Calculate the product of the target bolt pose matrix in the camera coordinate system and the standard bolt pose matrix to obtain the bolt pose in the robot base coordinate system.

[0021] The method of the present invention adopts a robot hand-eye vision system including a robot track 1, an industrial robot 2, a line laser stereo camera 3, a wrench 4 and a rolling mill bolt library 5. The robot track 1 is horizontally arranged on one side of the rolling mill 6, the industrial robot 2 is placed on the robot track 1, the line laser stereo camera 3 and the wrench 4 are arranged at the end of the industrial robot 2, and the rolling mill bolt library 5 is arranged in the working space of the industrial robot 2 for storing the rolling mill bolts. The robot track adds an extra degree of freedom to the six-degree-of-freedom industrial robot and expands the working range of the robot. A line laser stereo camera is installed at the end of the robot, and the Eye-in-Hand working mode is adopted. After visual segmentation and positioning are completed, the robot end wrench is guided to realize bolt removal.

[0022] The present invention aims at the bolts that need to be disassembled when changing rolls of RRCA445-HS finishing mill, taking M20 bolts as samples, simulates the steel plant production environment in the laboratory to verify the feasibility of the visual segmentation and positioning method, and obtains multiple groups of three-dimensional point cloud images of the bolt parts of the rolling mill through the robot hand-eye vision system. In order to locate the position of the target bolt, a template bolt library with the same specification as the target bolt is established, and the original point cloud is preprocessed, segmented, identified and positioned, and finally the visual positioning of the rolling mill bolt is completed.

[0023] 1. Preprocessing of original point cloud

[0024] Due to the complex industrial site environment, the original point cloud collected by the line laser stereo camera contains some irrelevant noise points. The present invention successively pre-processes the original point cloud by means of through-filtering and voxel filtering to obtain a point cloud with clear boundaries and easy segmentation. The specific steps are as follows:

[0025] (1) Pass-through filtering

[0026] Since there are a lot of fixed point clouds in the point cloud background that are irrelevant to the main structure, they will affect the target extraction. Straight-through filtering is usually the easiest way to remove fixed point clouds. x m , y m , z m ) Set upper and lower thresholds x axis( X 1, X 2), y axis( Y 1, Y 2), z axis( Z 1, Z 2), retain the points within the range, then the coordinates of the points after direct filtering are:

[0027]

[0028] The present invention needs to filter out irrelevant point clouds in the background, using z For straight-through filtering, the upper and lower filter thresholds are (200, 800).

[0029] (2) Voxel filtering

[0030] The amount of point cloud data collected directly by the stereo camera is relatively large, which seriously affects the processing efficiency and will cause redundant points when describing shape features. Voxel filtering is used to simplify the original point cloud and complete the downsampling operation of the point cloud without destroying the target's geometric structure and shape features.

[0031] Divide the point cloud range into cubes of the same size, calculate the center of gravity of each cube, and replace all points in the cube with the center of gravity to simplify the point cloud. d The choice is extremely important. d If the value is too large, some important points containing target edge features will be filtered out, resulting in over-simplification of the point cloud; d If the value is too small, the number of point clouds in the cube will be too small, which will not help to simplify the point clouds. d =0.75 mm.

[0032] 2. Target bolt head top plane point cloud segmentation

[0033] Due to the different angles between the camera lens and the bolt, the side elevation point cloud of the bolt head will be missing to varying degrees, and only the plane point cloud at the top of the bolt will remain intact. Therefore, only the complete plane point cloud of the bolt top is segmented and retained to facilitate the identification and positioning of the bolt.

[0034] The preprocessed point cloud contains the entire bolt head, the housing plane, and some adhesion points. The direct clustering method cannot separate the bolt head alone. If statistical filtering is used, the sparse corner points on the top surface of the bolt will be mistakenly removed, resulting in the loss of edge detail information of the bolt point cloud, which will cause difficulties in subsequent positioning work. Therefore, it is necessary to fit the housing plane according to the plane model. Removing this plane can separate the bolt head point cloud independently, and finally fit and segment the bolt head top plane point cloud. The specific steps are as follows:

[0035] (1) The random sampling consistency method based on the plane model is used to fit and remove the casing plane

[0036] Random Sample Consensus (RANSAC) estimates mathematical model parameters iteratively from a set of observation data containing outliers. The data that conforms to the assumed model parameters are called inliers, and the remaining points are called outliers. The algorithm requires that the minimum number of samples of the basic subset and the probability of obtaining at least one benign subset be guaranteed under a certain confidence probability. P ( P > t ) satisfies the following relationship:

[0037]

[0038] in, t is the probability of an interior point in the data set, that is , N The minimum amount of data required to calculate the model parameters, m is the number of iterations.

[0039] According to the plane characteristics of the casing around the bolt, the plane model is selected as the mathematical model for estimation:

[0040] Ax + By + C = D

[0041] Calculate plane parameters by picking three points A , B , C , D , fit the optimal plane, and use the outer points after fitting as the reserved points, thereby removing the casing plane and achieving complete independence of the bolt head as a whole. Error threshold selected during RANSAC fitting T and iteration threshold m 0 directly affects the optimal plane estimation effect. Error threshold T It mainly limits the range of the up and down jumps of the midpoint of the estimated plane. The value will affect the flatness of the fitted plane. The maximum number of iterations varies depending on the point cloud, so when setting the iteration threshold m If 0 is too small, the iteration will often stop before the optimal effect is achieved, thus outputting an erroneous result. T is 2 mm and m0 is 50.

[0042] (2) Extract the entire bolt head using Euclidean clustering

[0043] After the segmentation by the random sampling consistency algorithm based on the plane model, the overall point cloud of the bolts in the point cloud is no longer connected with other point clouds, so the Euclidean clustering segmentation can be used to cluster each bolt into one category.

[0044] Euclidean clustering is a clustering algorithm based on the Euclidean distance between adjacent points. Euclidean distance is the Euclidean metric. m The true distance between two points in dimensional space. Assume that the point cloud S 0={ p 1, p 2,…, p n}, where any two points p i and p j of k Neighborhood Euclidean distance d ij for:

[0045]

[0046] The distance is less than the distance threshold dThe points with a value of 0 are clustered together, and this process is repeated until the number of points in the cluster no longer increases, and the entire clustering process ends.

[0047] In the Euclidean clustering algorithm, the distance threshold d 0 and the upper and lower thresholds of the number of points contained in the cluster S max , S min It is particularly important to determine the distance threshold. d 0 helps improve computing efficiency, the upper and lower thresholds of clustering S max , S min The number of point clouds that need to include the target. If the range is too small, the target point cloud cluster may be missed, while if the range is too large, other irrelevant point cloud clusters will be included. Therefore, the present invention determines through experiments d 0 is 2 mm, S min 800, S max is 1400.

[0048] (3) Fitting and segmenting the bolt top plane point cloud using the random sampling consistency method based on the plane model

[0049] The top of the hexagonal bolt consists of a circular plane and six triangular arc surfaces. Since the curvature of the arc surface is small and the maximum height difference is only 1.2 mm, the top can be approximately regarded as a plane, thus obtaining a hexagonal plane on the top.

[0050] The random sampling consistency algorithm based on the plane model is used again to fit the hexagonal plane on the top of the bolt to ensure that the error threshold is minimized as much as possible without destroying the details of the six corners of the hexagon. T , the present invention determines the error threshold through experiments T is 1.5 mm, the iteration threshold m 0 is 50.

[0051] 3. Establish standard template bolts

[0052] According to the target bolt specifications, standard position bolts with the same size are selected as templates. The robot is used to teach the template bolts so that the robot end sleeve matches the template bolts. At this time, the TCP coordinate system of the robot coincides with the template bolt coordinate system. The tool position information on the teaching box display screen is recorded, which is the posture of the template bolt in the robot base coordinate system. To facilitate subsequent calculations, the rotation vector in the posture is converted into the expression of the rotation matrix through the Rodriguez formula, and the rotation and translation matrix is ​​homogenized to obtain the template bolt posture matrix.

[0053] The three-dimensional point cloud of the template bolt is collected through the robot hand-eye vision system, and preprocessed and segmented to obtain the template bolt top plane point cloud with clear edge contour and high image quality.

[0054] 4. Identification and positioning of target bolt head

[0055] The hexagonal plane point cloud at the top of the bolt after segmentation contains the real space pose information of the bolt. It is difficult to directly obtain the three-dimensional pose information of the plane. It is relatively simple to match it with the standard pose bolt and calculate the transformation matrix.

[0056] The method of combining coarse registration with fine registration is adopted. First, the template bolt point cloud is moved to the nearby position of the target bolt point cloud by coarse registration, so that the two point clouds are roughly aligned in the same coordinate system. Then, the rigid body transformation matrix of the two point clouds is calculated by fine registration, and the target bolt position and posture in the robot base coordinate system are calculated together with the template bolt position and posture matrix. The specific steps are as follows:

[0057] (1) Coarse registration using principal component analysis

[0058] Coarse registration is mainly used to register two point clouds whose initial relative positions are unknown, roughly align the two point clouds, and provide a better initial pose for subsequent fine registration.

[0059] The principal component analysis (PCA) algorithm is a commonly used dimensionality reduction method. The dimensionality of three-dimensional point clouds is relatively high. PCA dimensionality reduction can be used to convert multiple indicators into a few comprehensive indicators while maintaining the overall variance structure of the data set. The PCA-based coarse registration method mainly uses the principal axis direction of three-dimensional point cloud data for registration. First, the covariance matrix of the two groups of point clouds is calculated, and the main eigenvectors, that is, the principal axis direction of the point cloud data, are calculated based on the covariance matrix. Then, the rotation matrix is ​​solved based on the principal axis direction. R , the offset value of the centroid coordinates of the two sets of point clouds is the translation vector T ,Will( R , T ) is applied to the template point cloud to complete the coarse registration.

[0060] (2) Iterative nearest neighbor method for precise registration

[0061] The purpose of fine point cloud registration is to estimate a rotation and translation matrix with better coincidence based on rough registration. The iterative nearest neighbor point (ICP) registration algorithm is an optimal matching algorithm based on the least squares method, which continuously iterates and corrects the rigid body transformation of two point clouds to minimize the distance between all point sets.

[0062] Template point cloud after coarse registration and rigid body transformation , target point cloud , we need to find a set of rigid body transformation matrices ( R , T ) to determine the transformation relationship between the two point clouds:

[0063]

[0064] But in fact P’ , Q It is impossible for all points on the same point to coincide completely, so we need to find the point according to certain constraints. p i exist Q The nearest point in q i , define the error function:

[0065]

[0066] (3) Calculate the bolt position in the robot base coordinate system

[0067] Solving the position and posture of the target bolt is to solve the transformation matrix from the target bolt coordinate system to the robot base coordinate system. base H object , the template bolt pose is the transformation matrix of the template bolt coordinate system relative to the robot base coordinate system base H template , which is obtained by teaching the template bolt position by the robot. The transformation matrix between the target bolt point cloud and the template bolt point cloud is solved by point cloud registration. template H object , then the position of the target bolt is:

[0068] base H object = base H template template H object .

Claims

1. A method for visual segmentation and positioning of a bolt head of a rolling mill, comprising the following steps: Step 1: In the steel production preparation workshop environment, the hand-eye vision system is used to obtain the original three-dimensional point cloud of the rolling mill bolt part, which includes the casing point cloud around the rolling mill bolt, the overall point cloud of the rolling mill bolt head and noise points. The obtained original three-dimensional point cloud is sequentially subjected to through-filtering and voxel filtering to complete preprocessing; Step 2: Segment the target bolt head top plane point cloud from the preprocessed point cloud. The specific steps are as follows: (1) The point cloud of the casing part at the bottom of the bolt is fitted and segmented using a random sampling consistency algorithm based on a plane model, and the point cloud of the casing part is deleted; (2) Cluster and segment the overall point cloud of the bolt head based on the size of the target bolt and the difference in Euclidean distance between each point. (3) A random sampling consistency algorithm based on a plane model is used to fit and segment the top plane point cloud of the bolt head; Step 3: The robot teaches a standard bolt with the same specifications as the target bolt, obtains the standard bolt pose matrix, collects and segments the standard bolt head top plane point cloud, and uses the standard bolt head top plane point cloud as the template bolt point cloud; Step 4: Identify and locate the target bolt head position. The specific steps are as follows: (1) The principal component analysis method is used to roughly align the target bolt point cloud and the template bolt point cloud to obtain the initial rigid transformation matrix; (2) Based on the initial rigid transformation matrix, the iterative nearest neighbor method is used to perform precise point cloud registration to obtain the target bolt pose in the camera coordinate system; (3) Calculate the product of the target bolt pose matrix in the camera coordinate system and the standard bolt pose matrix to obtain the bolt pose in the robot base coordinate system.

2. The method for visual segmentation and positioning of bolt heads of rolling mills according to claim 1, characterized in that: The robot hand-eye vision system used in step 1 of the method comprises a robot ground track (1), an industrial robot (2), a line laser stereo camera (3), a wrench (4) and a rolling mill bolt library (5). The robot ground track (1) is horizontally arranged on one side of the rolling mill to be disassembled and assembled, the industrial robot (2) is placed on the robot ground track (1), the line laser stereo camera (3) and the wrench (4) are arranged at the end of the industrial robot (2), and the rolling mill bolt library (5) is arranged in the working space of the industrial robot (2) for storing rolling mill bolts.

3. The method for visual segmentation and positioning of bolt heads of rolling mills according to claim 1, characterized in that: Step 4: The formula for calculating the bolt pose in the robot base coordinate system is: base H object = base H template template H object ,in, base H object is the transformation matrix from the target bolt coordinate system to the robot base coordinate system, base H template is the transformation matrix of the template bolt coordinate system relative to the robot base coordinate system, template H object It is the transformation matrix between the target bolt point cloud and the template bolt point cloud.

Images