Visual positioning method and system for robot for polishing in local discontinuous areas

Abstract

The present application belongs to the field of machine vision and robot application, and in particular to a visual positioning method for polishing in a local discontinuous area by a robot, comprising the following steps: scanning a workpiece by a linear array camera to obtain point cloud of the surface of the workpiece; obtaining the size information of the bounding box of the workpiece, and dividing the bounding box into multiple planar cubes along the maximum and minimum of the edge length of the bounding box; fitting a fitting plane for each planar cube; calculating the distance of each point cloud in each planar cube to the fitting plane, and the average of all distances is the roughness; counting the roughness of all planar cubes and dividing them into different grades; selecting all planar cubes that need to be polished, setting a polishing threshold, and transmitting the polishing threshold, the centroid coordinates of the point cloud in the planar cube and the fitted surface normal vector to the robot controller to make the robot polish; the present application expands from two-dimensional image data to three-dimensional point cloud data, the data volume is improved, and the accuracy of the obtained roughness is improved.

Description

Technical Field

[0001] This invention belongs to the fields of machine vision and robotics applications, specifically a visual positioning method and system for robots to polish discontinuous areas. Background Technology

[0002] Currently, the fine grinding technology for workpieces with complex textures has become a cutting-edge research field in automated processing. While China's grinding manufacturing industry has begun to use robotic arms to replace manual labor for automated production, some workpiece grinding production lines still rely on manual inspection. Manual inspection is not only inefficient but also results in inconsistent quality. Current research in this technology is focused on achieving higher speed, higher precision, simpler system operation, lower maintenance costs, better integration, and less susceptibility to external influences.

[0003] Sun Rui et al. utilized the different degrees of influence of surface roughness on the phase of light to construct a three-dimensional shape model based on the reflection of the detected light wave, displaying the roughness at different locations (Sun Rui, Xue Shaolong. A material surface quality detection method, system and storage device [P]. Guangdong Province: CN113970551A, 2022-01-25.). Li Shuncai et al. extracted the root mean square of sound pressure level and the root mean square of reduced-dimensional vibration acceleration as feature values ​​from the time-domain signals of milling noise and triaxial milling vibration acceleration, and obtained the surface texture feature value of the workpiece using Tamura texture features (Li Shuncai, Li Songyuan, Liu Zhi, Hu Yuting, Shao Minghui, Song Guolu. A method for predicting surface roughness based on acoustic vibration and texture features [P]. Jiangsu Province: CN113704922A, 2021-11-26.). Tian Jianyan et al. performed image preprocessing, grayscale conversion, and filtering on the images, and then extracted the texture features of the images based on the grayscale co-occurrence matrix to calculate the roughness (Tian Jianyan, Dong Liangzhen, Wei Wanzhen, Gao Yunsong, Guo Hengkuan, Yang Shengqiang. Image-based Support Vector Machine Detection Method and System for Part Surface Roughness [P]. Shanxi Province: CN113989233A, 2022-01-28.). Wang Menghui designed a color distribution statistical matrix containing five matrix indicators: number of non-zero points, contrast, homogeneity, information entropy, and energy, to represent roughness (Wang Menghui. Research on Roughness Recognition Based on Color Distribution Statistical Matrix and Variable Prediction Model [D]. Hunan University, 2016.). Guo Bian used computer microscopy vision as the detection method and adopted the light and dark recovery shape algorithm to realize the three-dimensional reconstruction and roughness detection of the microscopic morphology of the processed surface (Guo Bian. Three-dimensional Reconstruction and Roughness Detection of Processed Surface Microscopic Vision Images [D]. Xi'an University of Technology, 2010.).

[0004] Currently, based on differences in sampling methods, roughness measurement methods can be broadly categorized into two types: contact and non-contact. Contact measurement accuracy, speed, and coverage are limited by the equipment itself, and samples are easily damaged. Non-contact roughness measurement methods include optical measurement, acoustic emission detection, and visual inspection. Among these, optical inspection and acoustic emission are susceptible to environmental influences and are relatively expensive. Current visual inspection methods typically employ two-dimensional image processing, but feature extraction from two-dimensional images relies on the contrast (edge ​​data) of the measured object, requires specific lighting conditions, and necessitates subsequent localization methods to determine areas where roughness exceeds a threshold. Summary of the Invention

[0005] The purpose of this invention is to provide a visual positioning method and system for robots to polish discontinuous areas, so as to overcome the shortcomings of existing visual detection methods that are easily affected by other scene factors such as light and color and are difficult to achieve ideal results.

[0006] The technical solution adopted by the present invention to achieve the above objectives is: a visual positioning method for robots to polish locally discontinuous areas, comprising the following steps:

[0007] 1) Scan the workpiece that needs to be polished with a linear scan camera to obtain the point cloud of the workpiece surface;

[0008] 2) Use PCA to obtain the bounding box size information of the workpiece, and divide it into multiple planar cubes along the maximum and minimum side lengths of the bounding box;

[0009] 3) Fit the contour line of each planar cube using least squares, and fit the fitted plane using the discreteness;

[0010] 4) Calculate the distance from each point cloud acquired by the linear scan camera in each planar cube to the fitting plane, and obtain the average value of all distances as the roughness R. a ;

[0011] 5) Calculate the R-value of all planar cubes. a The values ​​are adjusted according to the polishing process requirements. a Divided into different levels;

[0012] 6) Select all the planar cubes that need to be polished, set the polishing threshold t according to the different levels, and transmit the polishing threshold t, the centroid coordinates of the point cloud of the planar cube, and the fitted surface normal vector to the robot controller so that the robot can perform the polishing operation; after one polishing is completed, determine the relationship between the roughness Ra of the entire workpiece surface and the polishing threshold t. If Ra≤t, the polishing is complete; otherwise, repeat steps 1) to 6).

[0013] The side length of the planar cube is determined according to the robot's grinding process and requirements.

[0014] Step 3) specifically includes:

[0015] Since a point cloud is composed of countless points, by fitting a plane using the discreteness property, we can find a plane that is closest to all points, i.e., the plane with the shortest sum of distances to each point.

[0016] ax + by + cz + d = 0

[0017] Where a, b, c, and d are the coefficients of the plane equation, and x, y, and z are the variables of the plane equation;

[0018] By using the matrix form of the least squares method, we can obtain the values ​​of a, b, and c, and substitute them into the plane formula to obtain the fitted plane.

[0019] Step 4) specifically includes:

[0020] Calculate the distance from each point in each planar cube to the fitted plane, that is, fit the fitted plane formula to the two-dimensional data R. a In the expression, the average distance between each point is taken as the roughness R. a for:

[0021]

[0022] Where a, b, c, and d are the coefficients of the plane equation, x i y i , z i These are the variables of the plane equation, and n is the number of sampling points.

[0023] The fitted surface normal vector is:

[0024] A vision-based positioning system for robots used in grinding discontinuous areas includes:

[0025] The point cloud acquisition module is used to scan the workpiece that needs to be polished with a line scan camera to acquire the point cloud of the workpiece surface.

[0026] The plane fitting module is used to obtain the bounding box size information of the workpiece using the PCA method, divide the bounding box into multiple planar cubes along the maximum and minimum values ​​of the side length; fit the contour line of each planar cube using least squares, and fit the fitting plane through dispersion.

[0027] The roughness construction module is used to calculate the distance from each point cloud acquired by the linear scan camera to the fitted plane in each planar cube, and to obtain the average value of all distances as the roughness R. a ; Calculate the R-value of all planar cubes a The values ​​are adjusted according to the polishing process requirements. aDivided into different levels;

[0028] The grinding control module is used to select all the planar cubes that need to be ground, set the grinding threshold t according to the different levels of classification, and transmit the grinding threshold t, the centroid coordinates of the point cloud of the planar cube, and the fitted surface normal vector to the robot controller so that the robot can perform the grinding operation.

[0029] The present invention has the following beneficial effects and advantages:

[0030] 1. The linear scan camera used in this invention has a large scanning range and is not sensitive to changes in lighting or interference from ambient light. Compared with the traditional RGB recognition algorithm, it improves the algorithm's adaptability to the environment and enhances its robustness.

[0031] 2. The linear array camera in this invention can acquire sequence data in real time and efficiently, and therefore can process the real-time data in real time. Compared with the traditional step of scanning first and then processing, it can process weld data in real time more efficiently and reduce process cycle time.

[0032] 3. This invention extends from two-dimensional image data to three-dimensional point cloud data, which increases the amount of data, but also improves the accuracy of the algorithm.

[0033] 4. The point cloud data of the present invention can send three-dimensional shape data classified according to different roughness to the display end, which can capture the specific shape and location of defects more quickly. Attached Figure Description

[0034] Figure 1 This is a flowchart of the method of the present invention;

[0035] Figure 2 This is a schematic diagram of the cube segmentation principle of the present invention;

[0036] Figure 3 This is a schematic diagram of the cube division of the present invention;

[0037] Figure 4 This is a schematic diagram of the fitting contour line of the present invention;

[0038] Figure 5 This is a schematic diagram illustrating the principle of calculating roughness Ra according to the present invention.

[0039] Figure 6 This is a schematic diagram of the workpiece surface roughness within a unit sampling length according to the present invention;

[0040] Figure 7 This is a schematic diagram of the workpiece surface roughness within a unit sampling length according to the present invention. Detailed Implementation

[0041] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0042] When surface roughness and smoothness are used as metrics, roughness R a The arithmetic mean of the absolute values ​​of the relative heights of an object's surface within a sampling length of 1 is an important measure of roughness, especially since the polished surface is large and has a three-dimensional shape in space. Therefore, this invention uses roughness R... a The concept is extended to the three-dimensional plane.

[0043] like Figure 1 The diagram shown is a flowchart of the method of the present invention, and the specific steps are as follows:

[0044] Step 1) Scan the workpiece that needs to be polished with a line scan camera to obtain the point cloud of the workpiece surface;

[0045] Step 2) as Figure 2 The diagram shown illustrates the principle of cube segmentation in this invention. The PCA method is used to calculate the workpiece bounding box, and the sampling plane cube is divided along the maximum and minimum values ​​of the bounding box's side lengths. Figure 3 The diagram shown illustrates the cubic division of the invention, where the bounding box is divided into several planar cubes. The side length of each cube is determined based on the robot's polishing process and requirements.

[0046] Step 3) After dividing it into small squares, as follows: Figure 4 The diagram shown is a schematic diagram of the principle of fitting the ideal contour surface of the present invention. The ideal contour surface is obtained by fitting each cube using least squares, that is, the corresponding least squares fitted ideal contour line.

[0047] Since a point cloud is composed of countless points, we fit a plane using the discreteness formula, which means finding a plane that is closest to all the points. The plane formula is:

[0048] ax + by + cz + d = 0 #(4-1)

[0049] Where a, b, c, and d are the coefficients of the plane equation, and x, y, and z are the variables of the plane equation;

[0050] According to the least squares method, we have:

[0051] S=∑(ax i +by i +cz i ) 2 (4-2)

[0052] If we want to minimize the value of S, then we have:

[0053]

[0054] Substituting equation 4-2 into equation 4-3, we get:

[0055]

[0056] Converting it into a matrix equation, we get:

[0057]

[0058] The values ​​of a, b, and c are obtained by multiplying both sides of equation (4-5) by the inverse coefficient matrix, thus yielding the formula for fitting the plane. Simultaneously, using the values ​​of a, b, and c, the surface normal vector is fitted as follows:

[0059] Step 4), such as Figure 5 The diagram illustrates the principle of roughness calculation in this invention. It calculates the distance from each point in each cube to the fitted plane, and the average of these distances is the roughness R. a :

[0060] R in two-dimensional data a The calculation formula is as follows:

[0061]

[0062] Where lr is the sampling length, and Z(x) is the ordinate of all contours within the sampling length;

[0063] The discrete expression of formula (4-6) is as follows:

[0064]

[0065] Where n is the number of sampling points, and Z(i) is the distance from the sampling point to the midline fitted by the sampling point on the contour surface;

[0066] like Figure 7 The diagram shown illustrates the surface roughness of the workpiece within a unit sampling length according to the present invention. The present invention will use R... a Extending the concept to a three-dimensional plane, to calculate the distance from each point in each cube to the fitted plane, we need to substitute the fitted plane formula from equation (4-1) into equation (4-6), then we have:

[0067]

[0068] Where a, b, c, and d are the coefficients of the plane equation, x i y i , z i These are the variables in the plane equation.

[0069] Surface roughness of workpiece per unit sampling length, such as Figure 6 As shown, l is the sampling length, which can measure different R values ​​on the surface of the object. aDifferent peaks, since the actual surface profile includes three geometric shape errors: roughness, waviness and macroscopic shape error, when measuring the surface roughness profile, the measurement should be limited to a sufficiently short length h to suppress and reduce the influence of surface waviness and macroscopic shape error on the surface roughness profile measurement results. This length is called the sampling length, and in this embodiment, it is taken as 1.

[0070] The distance between two wave crests is called the wave length.

[0071] Wavelength within 1 mm is called micro-roughness;

[0072] The wave pitch changes periodically within the range of 1-10 mm, which is called surface waviness.

[0073] When the wavelength is above 10 mm and there is no periodic change, it is called macroscopic geometric error.

[0074] Step 5): Calculate the R-value of all planar cubes. a The values ​​are adjusted according to the polishing process requirements. a Divided into different levels;

[0075] In this embodiment, the roughness R of the turnout is adjusted according to the set grinding process. a The system is divided into several levels, and the calculated proportions are visualized, making it convenient for process engineers to query and debug on the display.

[0076] Among them, 0.00mm-0.05mm, 0.05mm-0.10mm, 0.10mm-0.20mm, 0.20mm-0.40mm, 0.40mm-0.60mm, and ≥0.80mm;

[0077] This invention, according to a set grinding process, reduces the roughness R of the turnout. a The following levels are divided, and the proportions are calculated and visualized. Additionally, the roughness R at different locations can be analyzed. a By comparing the results, a visual image of the comparison is obtained.

[0078] 6) Select all the planar cubes that need to be polished, set the polishing threshold t according to the different levels defined above, and transmit the polishing threshold t, the centroid coordinates of the point cloud of the planar cube, and the fitted surface normal vector to the robot controller so that the robot can perform the polishing operation; after one polishing is completed, determine the relationship between the roughness Ra of the entire workpiece surface and the polishing threshold t. If Ra≤t, the polishing is complete; otherwise, repeat steps 1) to 6).

[0079] This invention relates to a vision positioning system for robots used for grinding locally discontinuous areas, comprising:

[0080] The point cloud acquisition module is used to scan the workpiece that needs to be polished with a line scan camera to acquire the point cloud of the workpiece surface.

[0081] The plane fitting module is used to obtain the bounding box size information of the workpiece using the PCA method, divide the bounding box into multiple planar cubes along the maximum and minimum values ​​of the side length; fit the contour line of each planar cube using least squares, and fit the fitting plane through dispersion.

[0082] The roughness construction module is used to calculate the distance from each point cloud acquired by the linear scan camera to the fitted plane in each planar cube, and to obtain the average value of all distances as the roughness R. a ; Calculate the R-value of all planar cubes a The values ​​are adjusted according to the polishing process requirements. a Divided into different levels;

[0083] The grinding control module is used to select all the planar cubes that need to be ground, set the grinding threshold t according to the different levels of classification, and transmit the grinding threshold t, the centroid coordinates of the point cloud of the planar cube, and the fitted surface normal vector to the robot controller so that the robot can perform the grinding operation.

Claims

1. A visual positioning method for robots used in grinding locally discontinuous areas, characterized in that, Includes the following steps: 1) Scan the workpiece that needs to be polished with a linear scan camera to obtain the point cloud of the workpiece surface; 2) Use PCA to obtain the bounding box size information of the workpiece, and divide it into multiple planar cubes along the maximum and minimum side lengths of the bounding box; 3) Fit the contour line of each planar cube using least squares, and fit the fitted plane using the discreteness; Since a point cloud is composed of countless points, by fitting a plane using the discreteness property, we can find a plane that is closest to all the points. ax + by + cz + d = 0 Where a, b, c, and d are the coefficients of the plane equation, and x, y, and z are the variables of the plane equation; Using the matrix form of the least squares method, we obtain the values ​​of a, b, and c, and substitute them into the plane formula to obtain the fitted plane. According to the least squares method, we have: S=Σ(ax i +by i +c-z i ) 2 (4-2) If we want to minimize the value of S, then we have: Substituting equation 4-2 into equation 4-3, we get: Replacing it with a matrix equation, we get: The values ​​of a, b, and c are obtained by multiplying both sides of equation (4-5) by the inverse coefficient matrix, thus yielding the formula for fitting the plane. Simultaneously, using the values ​​of a, b, and c, the surface normal vector is fitted as follows: 4) Calculate the distance from each point cloud acquired by the linear scan camera in each planar cube to the fitting plane, and obtain the average value of all distances as the roughness R. a ; R in two-dimensional data a The calculation formula is as follows: Where lr is the sampling length, and Z(x) is the ordinate of all contours within the sampling length; The discrete expression of formula (4-6) is as follows: Where n is the number of sampling points, and Z(i) is the distance from the sampling point to the midline fitted by the sampling point on the contour surface; Calculate the distance from each point in each planar cube to the fitted plane, that is, fit the fitted plane formula to the two-dimensional data R. a In the expression, the average distance between each point is taken as the roughness R. a for: Where a, b, c, and d are the coefficients of the plane equation, x i y i , z i These are the variables of the plane equation, and n is the number of sampling points; 5) Calculate the R-value of all planar cubes. a The values ​​are adjusted according to the polishing process requirements. a Divided into different levels; 6) Select all the planar cubes that need to be polished, set the polishing threshold t according to the different levels, and transmit the polishing threshold t, the centroid coordinates of the point cloud of the planar cube, and the fitted surface normal vector to the robot controller so that the robot can perform the polishing operation; after one polishing is completed, determine the relationship between the roughness Ra of the entire workpiece surface and the polishing threshold t. If Ra≤t, the polishing is complete; otherwise, repeat steps 1) to 6).

2. The visual positioning method for robot grinding of locally discontinuous areas according to claim 1, characterized in that, The side length of the planar cube is determined according to the robot's grinding process and requirements.

3. The visual positioning system of the visual positioning method for grinding locally discontinuous areas using a robot according to any one of claims 1-2, characterized in that, include: The point cloud acquisition module is used to scan the workpiece that needs to be polished with a line scan camera to acquire the point cloud of the workpiece surface. The plane fitting module is used to obtain the bounding box size information of the workpiece using the PCA method, and divide it into multiple planar cubes along the maximum and minimum values ​​of the bounding box side length. For each planar cube, the contour line is fitted using least squares, and the fitted plane is fitted using the discreteness; The roughness construction module is used to calculate the distance from each point cloud acquired by the linear scan camera to the fitted plane in each planar cube, and to obtain the average value of all distances as the roughness R. a ; Calculate the R-value of all planar cubes a The values ​​are adjusted according to the polishing process requirements. a Divided into different levels; The grinding control module is used to select all the planar cubes that need to be ground, set the grinding threshold t according to the different levels of classification, and transmit the grinding threshold t, the centroid coordinates of the point cloud of the planar cube, and the fitted surface normal vector to the robot controller so that the robot can perform the grinding operation.

Images