Ceramic bathroom surface glazing robot track automatic generation method based on 3D vision

Abstract

The invention relates to a method for automatically generating a track of a ceramic bathroom surface glazing robot based on 3D vision. The method comprises the following steps: acquiring a surface point cloud model of a target ceramic bathroom; inputting the surface point cloud model into a pre-trained point cloud segmentation model, and outputting S1 and S2 type curved surfaces; generating a preliminary fitting path curve based on the S1 and S2 curved surfaces; performing interpolation processing on the preliminary fitting path curve to obtain path scatter points on the path curve after interpolation; performing path attitude estimation based on path scatter points on the interpolated path curve to obtain a path attitude estimation result; and generating the track of the glazing robot based on the path attitude estimation result. According to the method, a personalized path generation scheme can be intelligently formulated according to different types of glazing curved surfaces on the surface of the ceramic bathroom workpiece. The key influence of the position and posture of the spray gun on the glazing effect is also considered, and the uniformity and high quality of the glazing process are ensured through fine position and posture control and path optimization.

Description

Technical Field

[0001] The present invention relates to the technical field related to machine vision, and in particular to a method for automatically generating trajectories of a ceramic bathroom surface glazing robot based on 3D vision. Background Art

[0002] Glazing is an important process in the production of ceramic sanitary ware. The glazing layer has a beautifying and protective effect on the surface of ceramic sanitary ware. In the early days, manual glazing was mostly used. Although this method is more flexible, it is inefficient, the quality is unstable, and the high-dust working environment can easily affect the health of workers. In recent years, with the development of robotics technology, robotic glazing has gradually replaced manual work. However, most traditional robotic glazing methods rely on manual teaching or offline programming. The operation process is complicated and the teaching cycle is long. It requires workers to master robot operation technology, industrial software application skills, and glazing technology. There is a severe shortage of skilled talents in related industries. Robot programming-free applications based on artificial intelligence technology will become the mainstream direction of future applications such as robotic glazing.

[0003] The existing automatic trajectory generation method usually first obtains the surface point cloud model of the workpiece through 3D camera scanning, then slices the point cloud, extracts a series of path scattered points, and then curve fits these scattered points to generate the working path, and finally obtains the robot glazing trajectory through robot inverse kinematics calculation. However, this method has essential problems when facing complex ceramic bathroom surfaces. It cannot fully adapt to the characteristics of a variety of complex surfaces, and the glazing path planning requirements between different surfaces vary greatly. If the point cloud model is not divided into regions before path planning, the path planning effect will be poor, which will affect the glazing quality. In addition, the posture of the spray gun has a direct impact on the deposition distribution of the glaze. If the trajectory planning is not smooth or accurate enough, the glazing uniformity will be affected, making it difficult to guarantee the final glazing quality. Summary of the invention

[0004] The purpose of the present invention is to solve at least one of the deficiencies of the prior art and to provide a method for automatically generating trajectories of a ceramic sanitary ware surface glazing robot based on 3D vision.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] Specifically, a method for automatically generating trajectories of a ceramic sanitary ware surface glazing robot based on 3D vision is proposed, including the following:

[0007] Obtain the surface point cloud model of the target ceramic bathroom;

[0008] Input the surface point cloud model into a pre-trained point cloud segmentation model, and output S1 and S2 surfaces;

[0009] Generate preliminary fitting path curves based on the S1-type surface and the S2-type surface respectively;

[0010] Performing interpolation processing on the preliminary fitting path curve to obtain path scatter points on the interpolated path curve;

[0011] The path attitude estimation is performed based on the path scattered points on the interpolated path curve to obtain the path attitude estimation result;

[0012] A trajectory of the glazing robot is generated based on the path posture estimation result.

[0013] Further, specifically, the preliminary fitting path curve is generated based on the S1 type surface and the S2 type surface, including:

[0014] Build a glazing spray gun model;

[0015] Based on the glazing spray gun model, a first cross section for the S1 type curved surface and a second cross section for the S2 type curved surface are respectively generated;

[0016] Extracting first path scatter points for the S1 type surface and extracting second path scatter points for the S2 type surface respectively;

[0017] Sorting the first path scatter points and the second path scatter points respectively to obtain sorted first path scatter points and sorted second path scatter points;

[0018] The first path scatter points after sorting and the second path scatter points after sorting are subjected to path curve fitting to obtain a preliminary fitting path curve.

[0019] Further, specifically, a glazing spray gun model is established, including:

[0020] The elliptical double β model of the glazing spray gun is established. During the operation of the glazing spray gun, the accumulation thickness of the glaze satisfies the β distribution function on both the x-axis and the y-axis. a and b are the major and minor semi-axis of the ellipse, respectively. The glazing boundary can be expressed as formula (1):

[0021]

[0022] The distribution function of the glaze deposition thickness per unit time at a point (x, y) in the elliptical glazing area of ​​the glazing spray gun is shown in equation (2), where δ max is the maximum thickness of glaze deposited in the elliptical glazing area per unit time, β1 and β2 represent the β values ​​in the x-axis and y-axis directions respectively,

[0023] .

[0024] Further, specifically, based on the glazing spray gun model, a first cross section for the S1 type curved surface and a second cross section for the S2 type curved surface are respectively generated, including:

[0025] For S1-type surfaces, an OBB bounding box is used, while for S2-type surfaces, an AABB bounding box is used. φ0, φ1, and φ2 are the three-axis directions of the bounding box, respectively. φ0 is the shortest axis direction, φ1 is the second longest axis direction, and φ2 is the longest axis direction. φ0 is the normal vector of the S1-type surface point cloud, φ1 is the section normal vector, φ0 and φ2 constitute the section plane, and the section plane is generated at an equal distance in the φ1 direction. For S2-type point clouds, φ0 is the section normal vector, φ1 and φ2 constitute the section plane, and the section plane is generated at an equal distance in the φ0 direction.

[0026] Considering that the glazing profile of the glazing spray gun is an ellipse, the x-axis direction, i.e., the long axis direction, is the glazing forward direction, and the short axis is the glazing cross-sectional distance. In formula (3), Δd N Indicates the distance of the cross section in its normal direction N, K indicates the proportional coefficient, and b indicates the short semi-axis of the glazing ellipse profile. Since the distribution of the glaze accumulation thickness is not uniform, a proportional coefficient should be set to make the glaze accumulation of the two glazing paths overlap at the boundary to achieve a relatively uniform glaze accumulation thickness.

[0027] Δd N =2Kb(K<1)#(3)

[0028] Since the number of sections m is determined by the section distance Δd N and the length d of the surface point cloud in the cross-section normal direction N source The formula is shown in (4). In formula (5), μ represents d source With Δd N Remainder, μ max Represents the remainder threshold. When μ is less than μ max When μ is greater than μ, round down to avoid repeated glazing between different surfaces. max When applying glaze, round up to avoid under-glazing.

[0029] .

[0030] Further, specifically, extracting the first path scatter points for the S1 type surface and extracting the second path scatter points for the S2 type surface respectively includes:

[0031] Assume that the point cloud of the surface to be glazed is P, take N as the section normal vector and construct the section group M with equal distance, take an error range δ, when the point cloud point P of the glazed surface i Distance d to the cross section iWhen the distance is not greater than δ, that is, when formula (6) is satisfied, the point is projected onto the cross section. When the distance is greater than δ, the point is discarded, and the path scatter point P extracted can be obtained. i ∈P(i=0,1,…,n), and then extract the first path scatter points and the second path scatter points based on this,

[0032] |d i |≤δ#(6).

[0033] Further, specifically, sorting the first path scatter points and the second path scatter points respectively to obtain sorted first path scatter points and sorted second path scatter points includes:

[0034] The longest axis direction φ2 of the bounding box OBB corresponding to the first path scatter point is taken as the direction of the path, and the projection length L of the path scatter point in this direction is calculated. i To sort, when the cross section M i After the scatter point sorting is completed, the cross section M i+1 The sorting direction is changed to -φ2, and the calculation formula of the projection length is shown in formula (7), where P0 represents the initial position point in the direction of the path, P i represents the scattered points in the direction of the path,

[0035] L i =(P i -P0)·φ2#(7)

[0036] For the AABB bounding box corresponding to the second path scatter point, φ1 and φ2 form a cross-section plane, and fit the center O of the S2-type surface, take φ2 as the starting direction vector, and then according to the angle α between the vector formed by the scatter point and the center and the starting normal vector i Sort the scattered points by size, and Respectively represent the coordinate values ​​of scattered point i in the φ2 and φ1 directions, and They represent the coordinates of the center O in the φ2 and φ1 directions respectively, and the angle is calculated by formula (8):

[0037] .

[0038] Further, specifically, path curve fitting is performed on the sorted first path scatter points and the sorted second path scatter points by cubic non-uniform rational B-spline NURBS curve fitting to obtain a preliminary fitting path curve.

[0039] Further, specifically, interpolation processing is performed on the preliminary fitting path curve to obtain path scatter points on the interpolated path curve, including:

[0040] First, the preliminary fitting path curve is divided into num parts with equal step lengths, that is, there are num+1 interpolation points, and then the chord height difference between two adjacent interpolation points is calculated. If the chord height difference e exceeds the specified maximum error range ε, the step length is reduced until e is within the error range, and then the interpolation point coordinates are updated until the last interpolation point is calculated. The specific calculation is shown in formula (10), where R i is the radius of the fitting circle between two adjacent points, and L is approximately equal to the Euclidean distance between two adjacent points;

[0041] .

[0042] Further, specifically, the path attitude estimation is performed based on the path scattered points on the interpolated path curve to obtain the path attitude estimation result, including:

[0043] The path pose is estimated by the glazing path feeding direction t Tool and normal direction n Tool Determine, the calculation formula is shown in formula (11),

[0044] Feed direction t Tool This is the first-order derivative of the NURBS curve. The normalized vector of .

[0045]

[0046] A kd tree is established around a point C(u) on the interpolated path curve, and a set of neighborhood points are obtained by searching the point cloud near the point, and the centroid P of the neighborhood point is calculated. centroid , and use the neighborhood points to calculate the covariance matrix C, and then use the principal component analysis method PCA to decompose the covariance matrix. V is the eigenvector matrix, and the column vectors v1, v2, and v3 of V are the eigenvectors of the covariance matrix, and are unit vectors. Λ is the eigenvalue diagonal matrix, and λ1, λ2, and λ3 are the corresponding eigenvalues. The eigenvector corresponding to the smallest eigenvalue is the normal vector n Tool ;

[0047]

[0048] C=VΛV T #(14)

[0049] V=(v1 v2 v3)#(15)

[0050]

[0051] The x-axis of the TCP coordinate of the path point in the path posture estimation result is set as the feed direction t at the point C(u). Tool , the z-axis is set to the negative direction of the normal vector of the neighborhood point cloud at this point - n Tool, and then according to the right-hand rule, the y-axis direction of the point can be obtained, and the TCP coordinates of any point on the path curve can be calculated.

[0052] Further, specifically, generating a trajectory of the glazing robot based on the path posture estimation result includes:

[0053] Since the glazing spray gun of the glazing robot works in a non-contact manner and is in the z-axis direction of the end, the TCP coordinates of the path point are converted to the TCP coordinates of the end flange. The length d of the spray gun needs to be added to the z-axis. Tool and glazing height h, and then the TCP coordinates are converted into robot joint angle coordinates through robot inverse kinematics to generate the glazing robot trajectory.

[0054] The beneficial effects of the present invention are:

[0055] The present invention proposes a method for automatically generating the trajectory of a ceramic sanitary ware surface glazing robot based on 3D vision, which does not require manual intervention and improves the working speed and automation level of the ceramic sanitary ware surface glazing production line, which is specifically embodied in the following aspects:

[0056] 1. The surface of ceramic sanitary ware workpieces is composed of free complex surfaces. The simple parallel section method is difficult to adapt to the diverse surface features. Its slices may not accurately reflect the true geometric features of the surface and affect the accuracy of path planning. According to the characteristics of complex surfaces, a surface classification method is designed to divide the surfaces into "non-annular surfaces" S1 and "annular surfaces" S2, and use OBB and AABB bounding box algorithms respectively to better fit the true shape of the surface and ensure the accuracy of the scattered section direction.

[0057] 2. Combined with the glaze thickness distribution model of the glazing spray gun, a dynamic scattered section generation distance is designed to prevent over-glazing and under-glazing, and ensure the uniformity of glazing path generation and glazing effect.

[0058] 3. According to the S1 and S2 surfaces, the projection distance sorting method and polar coordinate sorting method are designed respectively, and the cubic non-uniform rational B-spline is combined to generate a smooth glazing path, ensuring the smoothness of the glazing robot's motion trajectory.

[0059] 4. According to the curvature change of the fitting curve, an adaptive curvature interpolation method was designed to solve the problem of large fluctuations in the glaze deposition thickness on the glazed surface. During the glazing process, the robot can evenly deposit the glaze on the surface of the ceramic sanitary workpiece, thereby improving the glazing quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The above and other features of the present disclosure will become more apparent by describing in detail the embodiments shown in the accompanying drawings. The same reference numerals in the accompanying drawings of the present disclosure represent the same or similar elements. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure. For those skilled in the art, other accompanying drawings can be obtained based on these accompanying drawings without creative work. In the accompanying drawings:

[0061] Attached Figure 1 It is the overall workflow diagram of the automatic generation of robot trajectory for ceramic bathroom surface glazing based on 3D vision;

[0062] Attached Figure 2 It is a flow chart of the surface point cloud of type S1 and S2 from classification to generation of interpolation points;

[0063] Attached Figure 3 It is a schematic diagram of the ceramic bathroom glazing trajectory generation system;

[0064] Attached Figure 4 It is a schematic diagram of the glazing spray gun model;

[0065] Attached Figure 5 It is the flow chart of point cloud segmentation of S1 and S2 types;

[0066] Attached Figure 6 is a schematic diagram of surface categories, with the left side showing the S1 surface category in an instance and the right side showing the S2 surface category in an instance;

[0067] Attached Figure 7 It is a schematic diagram of AABB and OBB bounding boxes;

[0068] Attached Figure 8 It is a schematic diagram of path scatter point extraction. The left side is the schematic diagram of section method generation, and the right side is the schematic diagram of scatter point projection to section.

[0069] Attached Fig. 9 It is a schematic diagram of path scatter point sorting. The left side is the scatter point sorting of S1 type surface, and the right side is the scatter point sorting of S2 type surface;

[0070] Attached Fig.10 This is a schematic diagram of the equal chord height difference interpolation method;

[0071] Attached Fig.11 It is a schematic diagram of the path after scatter point fitting. The left side is a schematic diagram of the S1 type surface path, and the right side is a schematic diagram of the S2 type surface path;

[0072] Attached Fig.12 It is a schematic diagram of interpolation point pose estimation;

[0073] Attached Fig.13 This is a schematic diagram of TCP coordinates of interpolation points. DETAILED DESCRIPTION

[0074] The following will be combined with the embodiments and drawings to clearly and completely describe the concept, specific structure and technical effects of the present invention, so as to fully understand the purpose, scheme and effect of the present invention. It should be noted that the embodiments in this application and the features in the embodiments can be combined with each other without conflict. The same reference numerals used throughout the drawings indicate the same or similar parts.

[0075] Example 1, reference Figure 1 as well as Figure 2 The present invention proposes a method for automatically generating a trajectory of a ceramic bathroom surface glazing robot based on 3D vision, comprising the following steps:

[0076] Obtain the surface point cloud model of the target ceramic bathroom;

[0077] Input the surface point cloud model into a pre-trained point cloud segmentation model, and output S1 and S2 surfaces;

[0078] Generate preliminary fitting path curves based on the S1-type surface and the S2-type surface respectively;

[0079] Performing interpolation processing on the preliminary fitting path curve to obtain path scatter points on the interpolated path curve;

[0080] The path attitude estimation is performed based on the path scattered points on the interpolated path curve to obtain the path attitude estimation result;

[0081] A trajectory of the glazing robot is generated based on the path posture estimation result.

[0082] The specific implementation points are as follows:

[0083] (I) Construction and analysis of ceramic bathroom glazing trajectory generation system

[0084] The overall system for generating glazing trajectories for ceramic sanitary ware is as follows Figure 3 As shown in the figure, it is mainly composed of ceramic sanitary ware workpieces, motion platform, 3D point cloud acquisition module, glazing module and machine vision processing system. The motion platform can rotate and translate the workpiece to make it reach the processing position better. The 3D point cloud acquisition module mainly includes a robot and a 3D camera, which is responsible for collecting the surface information of ceramic sanitary ware on the production line and reconstructing the point cloud model. The glazing module mainly includes a robot and a glazing spray gun, which is responsible for glazing the surface of ceramic sanitary ware. The machine vision processing module is integrated in the host computer and is responsible for data acquisition, unified coordinate system, point cloud segmentation, point cloud trajectory generation, and controlling robot movement and glazing. The reconstructed point cloud model is unified to the base coordinate system {B} of the glazing robot, and subsequent work can be carried out.

[0085] (ii) Glazing spray gun model

[0086] like Figure 4 As shown, an elliptical double β model of the glazing spray gun is established. The accumulation thickness of the glaze during the operation of the glazing spray gun satisfies the β distribution function on both the x-axis and the y-axis. a and b are the major and minor semi-axis of the ellipse, respectively. The glazing boundary can be expressed as formula (1).

[0087]

[0088] The distribution function of the glaze deposition thickness per unit time at a point (x, y) in the elliptical glazing area of ​​the glazing spray gun is shown in equation (2). max is the maximum thickness of glaze deposition in the elliptical glazing area per unit time, and β1 and β2 represent the β values ​​in the x-axis and y-axis directions, respectively.

[0089]

[0090] (III) Segmentation of ceramic bathroom surface point cloud model

[0091] After obtaining the surface point cloud model of the ceramic bathroom, in order to meet the needs of subsequent path generation, the point cloud model needs to be segmented into "non-annular surface" S1 category and "annular surface" S2 category. Figure 5 As shown in the figure, first construct a point cloud model dataset of ceramic sanitary ware artifacts, then build a point cloud segmentation network for training, and obtain a trained network model. Then use the network model to segment the reconstructed point cloud model to obtain S1 and S2 surface point clouds. This method can run end-to-end without tedious and complex calculations in the middle. Some examples of S1 and S2 surface point clouds are shown in the figure. Figure 6 shown.

[0092] (IV) Section generation

[0093] like Figure 6 As shown, for S1 type surfaces, a compact bounding box OBB is used, while S2 type surfaces use an AABB bounding box. They are the three-axis directions of the bounding box, is the shortest axis direction, is the second major axis direction, is the longest axis direction. Figure 7 As shown, is the normal vector of the S1 surface point cloud, is the cross-section normal vector, and Construct a cross-sectional plane and Generate section planes at equal distances in the direction. Figure 8 As shown, for the S2 point cloud, is the cross-section normal vector, and Construct a cross-sectional plane and Generate section planes at equal distances in the direction.

[0094] like Figure 4 As shown in the figure, the glazing profile of the glazing spray gun is an ellipse, with the x-axis direction (long axis direction) as the glazing forward direction, and the short axis is the glazing cross-sectional distance. N Indicates the distance of the cross section in its normal direction N, K indicates the proportional coefficient, and b indicates the short semi-axis of the glazing ellipse profile. Since the distribution of the glaze accumulation thickness is not uniform, a proportional coefficient should be set to make the glaze accumulation of the two glazing paths overlap at the boundary to achieve a relatively uniform glaze accumulation thickness.

[0095] Δd N =2Kb(K<1)#(3)

[0096] Then the number of sections m is determined by the section distance Δd N and the length d of the surface point cloud in the cross-section normal direction N source The formula is shown in (4). In formula (5), μ represents d source With Δd N Remainder, μ max Indicates the remainder threshold. When μ is less than μ max When μ is greater than μ, round down to avoid repeated glazing between different surfaces. max When applying glaze, round up to avoid under-glazing.

[0097]

[0098] (V) Path Scatter Point Extraction

[0099] Since both S1 and S2 surfaces are non-parametric surfaces, it is impossible to directly extract path scatter points on the discrete ceramic bathroom surface point cloud in a parametric way. The present invention designs a method based on the cutting plane method to extract path scatter points. Figure 8 As shown in the figure, the point cloud of the surface to be glazed is P, N is the section normal vector and the section group M is constructed with equal distance. Since the point cloud data is discrete, the point cloud of the glazed surface may not fall completely on the section, so an error range δ is taken, such as Figure 8 As shown, when the point cloud point P of the glazed surface i Distance d to the cross section i When the distance is less than δ, the point is projected onto the cross section. When the distance is greater than δ, the point is discarded, and the path scatter point P can be extracted. i ∈P(i=0,1,…,n).

[0100] |d i |≤δ#(6)

[0101] (VI) Path Scattering Sorting

[0102] like Fig. 9 As shown in FIG. 1 , the present invention designs two sorting methods according to the different characteristics of the path scatter points extracted from the S1 and S2 surfaces. Fig. 9 As shown, take the longest axis direction of the S1 surface OBB bounding box As the direction of the path, according to the projection length L of the path scatter points in this direction i To sort, when the cross section M i After the scatter point sorting is completed, the cross section M i+1 The sorting direction of This ensures the continuity of the path and the continuity of the robot's motion. The calculation formula for the projection length is shown in formula (7), where P0 represents the initial position point in the direction of the path, P i Represents scattered points in the direction of this path.

[0103]

[0104] like Fig. 9 As shown, for S2 surface, and Construct the cross-sectional plane, fit the center O of the "annular surface", and take is the starting direction vector, and then the angle α between the vector formed by the scattered point and the center and the starting normal vector i Sort the scatter points by size. and Respectively represent the scattered point i in and Coordinate value in direction. and Respectively represent the center but O is and The coordinate value in the direction. The angle can be obtained by formula (8).

[0105]

[0106] (VII) Path curve fitting

[0107] In order to meet the glazing quality, the extracted path scatter points need to be fitted into a smooth and continuous curve. The present invention adopts cubic non uniform rational B-splines (CubicNURBS) curve fitting scatter points. The first and second order derivatives of its continuous curve are both continuous and differentiable, which can well ensure the smooth change of displacement and speed during the glazing process, and ensure that the glaze deposition on the glazed surface meets the requirements. The definition of NURBS curve is as follows:

[0108]

[0109] In formula (9), k represents the degree of the curve (k=3 in the present invention), ω i Point P i The weight factor, N i,k (u) is the B-spline basis function.

[0110] (VIII) Glazing path interpolation

[0111] After fitting the curve, it is necessary to interpolate the curve and let the robot move along the interpolation point path. Since the glazed surfaces targeted by the present invention are all free-form surfaces, the curvature of the fitted curve varies. If the interpolation step is constant, the glaze deposition thickness of the glazed surface will fluctuate greatly when the curvature is large. Therefore, the present invention uses a method of equal chord height error for interpolation. Fig.10 As shown, first divide the curve into num parts with equal step length, that is, there are num+1 interpolation points, and then calculate the chord height difference between two adjacent interpolation points. If the chord height difference e exceeds the specified maximum error range ε, reduce the step length until e is within the error range, and then update the interpolation point coordinates until the last interpolation point is calculated. In formula (10), R i is the radius of the fitting circle between two adjacent points, and L is approximately equal to the Euclidean distance between two adjacent points. By constructing interpolation points by the above method, the interpolation points of the curve can be sparse when the curvature changes slowly, and dense when the curvature changes rapidly, so that the glaze deposition thickness of the glazed surface tends to be smooth, thereby improving the glazing quality. Fig.11 As shown, the paths of some S1 and S2 type surfaces are exemplified.

[0112]

[0113] 9. Glazing Path Posture Estimation

[0114] By using the above method to fit the path scatter points to a curve, and then interpolating on the curve, the position mapping from the curve parameter space to the robot task space is completed. If the robot is to perform glazing work, it is also necessary to obtain the posture of the robot end glazing tool. The posture of the end glazing tool is determined by the glazing path feed direction t Tool and normal direction n Tool As shown in formula (11), the feed direction t Tool This is the first-order derivative of the NURBS curve. The normalized vector of .

[0115]

[0116] A kd tree is built around a point C(u) on the NURBS curve, and a set of neighborhood points are obtained by searching the point cloud near the point. The centroid P of the neighborhood point is calculated. centroid, and use the neighborhood points to calculate the covariance matrix C. Then use principal component analysis (PCA) to decompose the covariance matrix. V is the eigenvector matrix. The column vectors v1, v2, and v3 of V are the eigenvectors of the covariance matrix and are unit vectors. The value of N is equal to P i The total number of eigenvalues. Λ is the eigenvalue diagonal matrix, λ1, λ2, λ3 are the corresponding eigenvalues. The eigenvector corresponding to the minimum eigenvalue is the normal vector n Tool .

[0117]

[0118] C=VΛV T #(14)

[0119] V=(v1 v2 v3)#(15)

[0120]

[0121] like Fig.12 As shown, the x-axis of the robot TCP at point C(u) is set as the feed direction t at that point Tool , the z-axis is set to the negative direction of the normal vector of the neighborhood point cloud at this point - n Tool Then, according to the right-hand rule, the y-axis direction of the point can be obtained, and the TCP coordinates of any point on the path curve can be calculated. Some examples are as follows Fig.13 shown.

[0122] (10) Glazing robot trajectory generation

[0123] The above method can be used to obtain the TCP coordinates of the interpolation points on the path of the glazing robot. Since the glazing spray gun works non-contact and the spray gun is in the z-axis direction of the end, the TCP coordinates of the path point are converted to the TCP coordinates of the end flange. The length d of the spray gun needs to be added to the z-axis. Tool and glazing height h. Then, the TCP coordinates are converted into robot joint angle coordinates through robot inverse kinematics to generate the robot's motion trajectory. Through the above method, the automatic generation of robot trajectory for ceramic sanitary surface glazing based on 3D vision can be realized.

[0124] Reference Figure 3 The basic components of the hardware part of the present invention include: a three-dimensional camera, a spray gun, two robots, a motion platform and a machine vision processing system.

[0125] The 3D camera is used to obtain the point cloud model of the ceramic bathroom surface;

[0126] The spray gun is used to atomize the glaze so that it is evenly deposited on the surface of the ceramic bathroom;

[0127] The end flange of robot 1 is fixed with a three-dimensional camera, which is responsible for moving the three-dimensional camera to perceive the surface information of ceramic sanitary ware.

[0128] The end flange of robot 2 is fixed with a glazing spray gun, which is responsible for moving the spray gun to each glazing surface;

[0129] The motion platform is responsible for moving the ceramic sanitary ware to the appropriate working position. It is used to assist the robot in completing the surface glazing when the robot's working range cannot completely cover the ceramic sanitary ware workpiece;

[0130] The robot vision processing system is responsible for the automatic generation of the robot glazing trajectory, which mainly includes the ceramic bathroom surface point cloud model acquisition module, the point cloud segmentation module, and the point cloud trajectory generation module.

[0131] The main implementation process of the present invention is:

[0132] 1. Fix the 3D camera on the end flange of robot 1, move the robot arm and the motion platform to fully perceive the surface information of the ceramic bathroom, reconstruct the point cloud model of the ceramic bathroom surface, and convert its coordinate system to the base coordinate system of robot 2;

[0133] In order to simplify the glazing program generation process, for new products, it is necessary to scan the complete point cloud model and regenerate the glazing program; for products in the same batch, the camera position is fixed to perform 3D positioning of the product, and the generated glazing program is fine-tuned according to the 3D position of the product;

[0134] 2. Establish a glazing spray gun model, establish the correlation between the distance h from the nozzle to the glazing surface and the glazing spray profile and glazing thickness σ, and dynamically adjust the position of the glazing spray gun;

[0135] 3. Since the point cloud model of ceramic sanitary ware workpiece is composed of complex free-form surfaces, it is difficult to design a general path generation method for complex free-form surfaces, so it is necessary to divide the point cloud model into regions. The present invention designs a segmentation network model to perform point cloud segmentation on the point cloud model obtained in process one, and divides the smooth curved surfaces into "non-annular curved surfaces" S1 category, such as the outer surface of the water tank of a ceramic toilet, and divides the curved surfaces with large curvature changes into "annular curved surfaces" S2 category, such as the inner cavity surface of a ceramic toilet. The annular curved surface here does not necessarily mean a ring shape. The present invention classifies the curved surfaces with large curvature changes into S2 category; Four.

[0137] 1. The "non-annular surface" S1 class uses the OBB (Oriented bounding box) bounding box algorithm to form a compact bounding box; the "annular surface" S2 class uses the AABB bounding box (Axis-aligned bounding box) algorithm to form a bounding box.

[0138] 2. Generate a projection section, project the scattered points near the section onto the plane, and sort the scattered points of the S1 and S2 surfaces using the sorting methods based on projection distance and polar coordinates respectively. Fit the sorted scattered points with cubic non uniform rational B-splines (Cubic NURBS) curves;

[0139] 5. Interpolate the path fitted in process 4, control the density of interpolation points according to the change of its curvature, take the forward direction of the glazing spray gun as the x-axis direction, the negative direction of the normal vector of the point cloud fitting around the point as the z-axis direction, and the cross product of the x-axis and the z-axis is the y-axis direction, and the robot end coordinates and posture can be obtained. Then, the robot joint angle combination is obtained through the robot inverse kinematics, and the mapping to the robot is the trajectory of the ceramic bathroom surface glazing robot.

[0140] Although the description of the present invention has been quite detailed and specifically describes several described embodiments, it is not intended to be limited to any of these details or embodiments or any particular embodiment, but should be regarded as providing a broad possible interpretation of these claims in view of the prior art by reference to the appended claims, thereby effectively covering the intended scope of the present invention. In addition, the above description of the present invention is based on the embodiments foreseeable by the inventor, and its purpose is to provide a useful description, and those non-substantial changes to the present invention that have not yet been foreseen may still represent equivalent changes to the present invention.

[0141] The above is only a preferred embodiment of the present invention. The present invention is not limited to the above implementation. As long as the technical effect of the present invention is achieved by the same means, it should belong to the protection scope of the present invention. Within the protection scope of the present invention, its technical scheme and / or implementation method can have various modifications and changes.

Claims

1. A method for automatically generating trajectories of a ceramic bathroom surface glazing robot based on 3D vision, characterized in that: Includes the following: Obtain the surface point cloud model of the target ceramic bathroom; Input the surface point cloud model into a pre-trained point cloud segmentation model, and output S1 and S2 surfaces; Generate preliminary fitting path curves based on the S1-type surface and the S2-type surface respectively; Performing interpolation processing on the preliminary fitting path curve to obtain path scatter points on the interpolated path curve; The path attitude estimation is performed based on the path scattered points on the interpolated path curve to obtain the path attitude estimation result; A trajectory of the glazing robot is generated based on the path posture estimation result.

2. The method for automatically generating trajectories of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 1, characterized in that: Specifically, the preliminary fitting path curves are generated based on the S1-type surface and the S2-type surface, including: Build a glazing spray gun model; Based on the glazing spray gun model, a first cross section for the S1 type curved surface and a second cross section for the S2 type curved surface are respectively generated; Extracting first path scatter points for the S1 type surface and extracting second path scatter points for the S2 type surface respectively; Sorting the first path scatter points and the second path scatter points respectively to obtain sorted first path scatter points and sorted second path scatter points; The first path scatter points after sorting and the second path scatter points after sorting are subjected to path curve fitting to obtain a preliminary fitting path curve.

3. The method for automatically generating trajectories of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 2, characterized in that: Specifically, a glazing spray gun model is established, including: The elliptical double β model of the glazing spray gun is established. During the operation of the glazing spray gun, the accumulation thickness of the glaze satisfies the β distribution function on both the x-axis and the y-axis. a and b are the major and minor semi-axis of the ellipse, respectively. The glazing boundary can be expressed as formula (1): The distribution function of the glaze deposition thickness per unit time at a point (x, y) in the elliptical glazing area of ​​the glazing spray gun is shown in equation (2), where δ max is the maximum thickness of glaze deposited in the elliptical glazing area per unit time, β1 and β2 represent the β values ​​in the x-axis and y-axis directions respectively, 。 4. The method for automatically generating trajectories of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 3, characterized in that: Specifically, based on the glazing spray gun model, a first cross section for the S1 type curved surface and a second cross section for the S2 type curved surface are generated respectively, including: For S1-type surfaces, an OBB bounding box is used, while for S2-type surfaces, an AABB bounding box is used. φ0, φ1, and φ2 are the three-axis directions of the bounding box, respectively. φ0 is the shortest axis direction, φ1 is the second longest axis direction, and φ2 is the longest axis direction. φ0 is the normal vector of the S1-type surface point cloud, φ1 is the section normal vector, φ0 and φ2 constitute the section plane, and the section plane is generated at an equal distance in the φ1 direction. For S2-type point clouds, φ0 is the section normal vector, φ1 and φ2 constitute the section plane, and the section plane is generated at an equal distance in the φ0 direction. Considering that the glazing profile of the glazing spray gun is an ellipse, the x-axis direction, i.e., the long axis direction, is the glazing forward direction, and the short axis is the glazing cross-sectional distance. In formula (3), Δd N Indicates the distance of the cross section in its normal direction N, K indicates the proportional coefficient, and b indicates the short semi-axis of the glazing ellipse profile. Since the distribution of the glaze accumulation thickness is not uniform, a proportional coefficient should be set to make the glaze accumulation of the two glazing paths overlap at the boundary to achieve a relatively uniform glaze accumulation thickness. Δd N =2Kb(K<1)#(3) Since the number of sections m is determined by the section distance Δd N and the length d of the surface point cloud in the cross-section normal direction N source The formula is shown in (4). In formula (5), μ represents d source With Δd N Remainder, μ max Represents the remainder threshold. When μ is less than μ max When μ is greater than μ, round down to avoid repeated glazing between different surfaces. max When applying glaze, round it up to avoid under-glazing. 。 5. The method for automatically generating trajectories of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 2, characterized in that: Specifically, extracting the first path scatter points for the S1 type surface and extracting the second path scatter points for the S2 type surface respectively includes: Assume that the point cloud of the surface to be glazed is P, take N as the section normal vector and construct the section group M with equal distance, take an error range δ, when the point cloud point P of the glazed surface i Distance d to the cross section i When the distance is not greater than δ, that is, when formula (6) is satisfied, the point is projected onto the cross section. When the distance is greater than δ, the point is discarded, and the path scatter point P extracted can be obtained. i ∈P(i=0,1,…,n), and then extract the first path scatter points and the second path scatter points based on this, |d i |≤δ#(6)。 6. The method for automatically generating the trajectory of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 4, characterized in that: Specifically, the first path scatter points and the second path scatter points are sorted to obtain sorted first path scatter points and sorted second path scatter points, respectively, including: The longest axis direction φ2 of the bounding box OBB corresponding to the first path scatter point is taken as the direction of the path, and the projection length L of the path scatter point in this direction is calculated. i To sort, when the cross section M i After the scatter point sorting is completed, the cross section M i+1 The sorting direction of the path is changed to -φ2, and the calculation formula of the projection length is shown in formula (7), where P0 represents the initial position point in the direction of the path, P i represents the scattered points in the direction of the path, L i =(P i -P0)·φ2#(7) For the AABB bounding box corresponding to the second path scatter point, φ1 and φ2 form a cross-section plane, and fit the center O of the S2-type surface, take φ2 as the starting direction vector, and then according to the angle α between the vector formed by the scatter point and the center and the starting normal vector i Sort the scattered points by size, and Respectively represent the coordinate values ​​of scattered point i in the φ2 and φ1 directions, and They represent the coordinates of the center O in the φ2 and φ1 directions respectively, and the angle is calculated by formula (8): 。 7. The method for automatically generating trajectories of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 2, characterized in that: Specifically, path curve fitting is performed on the sorted first path scatter points and the sorted second path scatter points by cubic non-uniform rational B-spline NURBS curve fitting to obtain a preliminary fitting path curve.

8. The method for automatically generating a trajectory of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 1, characterized in that: Specifically, interpolation processing is performed on the preliminary fitting path curve to obtain path scatter points on the interpolated path curve, including: First, the preliminary fitting path curve is divided into num parts with equal step lengths, that is, there are num+1 interpolation points, and then the chord height difference between two adjacent interpolation points is calculated. If the chord height difference e exceeds the specified maximum error range ε, the step length is reduced until e is within the error range, and then the interpolation point coordinates are updated until the last interpolation point is calculated. The specific calculation is shown in formula (10), where R i is the radius of the fitting circle between two adjacent points, and L is approximately equal to the Euclidean distance between two adjacent points; 。 9. The method for automatically generating the trajectory of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 8, characterized in that: Specifically, the path attitude estimation is performed based on the path scattered points on the interpolated path curve to obtain the path attitude estimation result, including: The path pose is estimated by the glazing path feeding direction t Tool and normal direction n Tool Determine, the calculation formula is shown in formula (11), Feed direction t Tool That is, the normalized vector of the first-order derivative C(u) of the NURBS curve; A kd tree is established around a point C(u) on the interpolated path curve, and a set of neighborhood points are obtained by searching the point cloud near the point, and the centroid P of the neighborhood point is calculated. centroid , and use the neighborhood points to calculate the covariance matrix C, and then use the principal component analysis method PCA to decompose the covariance matrix. V is the eigenvector matrix, and the column vectors v1, v2, and v3 of V are the eigenvectors of the covariance matrix, and are unit vectors. Λ is the eigenvalue diagonal matrix, and λ1, λ2, and λ3 are the corresponding eigenvalues. The eigenvector corresponding to the smallest eigenvalue is the normal vector n Tool ; C=VΛV T #(14) V=(v1 v2 v3)#(15) ； The x-axis of the TCP coordinate of the path point in the path posture estimation result is set as the feed direction t at this point. Tool , the z-axis is set to the negative direction of the normal vector of the neighborhood point cloud at this point -n Tool , and then according to the right-hand rule, the y-axis direction of the point can be obtained, and the TCP coordinates of any point on the path curve can be calculated.

10. The method for automatically generating trajectory of a ceramic sanitary ware surface glazing robot based on 3D vision according to claim 9, characterized in that: Specifically, generating a trajectory of the glazing robot based on the path posture estimation result includes: Since the glazing spray gun of the glazing robot works in a non-contact manner and is in the z-axis direction of the end, the TCP coordinates of the path point are converted to the TCP coordinates of the end flange. The length d of the spray gun needs to be added to the z-axis. Tool and glazing height h, and then the TCP coordinates are converted into robot joint angle coordinates through robot inverse kinematics to generate the glazing robot trajectory.

Images