An automatic gluing method based on 3D vision

Abstract

The application relates to a 3D vision-based automatic gluing method, which comprises the following steps: moving a glue outlet of an automatic gluing machine to the vicinity of a to-be-glued hole according to a preset position, moving a 3D camera at the end of a mechanical arm to above the to-be-glued hole to obtain point cloud data of the to-be-glued hole; pretreating the point cloud of the to-be-glued hole; extracting circular hole depth information based on three-dimensional point cloud of the bottom of the circular hole in the pretreated point cloud; sequentially performing closed operation, bilateral filtering and circular hole edge contour extraction processing on the pretreated point cloud of the to-be-glued hole; performing circular hole fitting by using a circular hole fitting algorithm according to the extracted circular hole contour edge, and outputting the center coordinates and the radius of the fitted circle; calculating the gluing amount according to the circular hole radius, the circular hole depth and the gluing thickness; and finely adjusting the glue outlet of the automatic gluing machine according to the coordinates in the mechanical arm base coordinate system, aligning the circular hole and gluing according to the calculated gluing amount. The application realizes a 3D vision-based automatic gluing method for irregularly-shaped circular holes on curved surfaces.

Description

Technical Field

[0001] This invention relates to the field of automatic adhesive application technology, and more particularly to an automatic adhesive application method based on 3D vision for irregular holes on curved surfaces. Background Technology

[0002] Currently, the task of sealing round holes on product surfaces with adhesive is primarily performed manually. However, manual operation is limited by the operator's skill level and working conditions, resulting in low bonding efficiency, high workload, long work cycles, and significant adhesive waste. To address the issue of automated adhesive application, two widely adopted methods are robot-taught adhesive application and vision-guided robot adhesive application. Manual "multi-level teaching" uses a fixed adhesive application trajectory; if the product changes, the trajectory needs to be re-taught manually. Vision-guided robot adhesive application can apply adhesive based on real-time image information, eliminating the need for human intervention and saving time and effort. However, the key to vision-guided robot adhesive application lies in the accuracy of the vision system's ability to identify the product to be coated.

[0003] In existing technologies, vision-guided robot adhesive application often suffers from poor versatility, cumbersome processes, time-consuming point cloud spatial model building, and the quality of the spatial model directly affects the quality of the adhesive application trajectory, resulting in poor stability. From the perspective of visual image acquisition methods, 3D laser sensors acquire information through scanning, which, compared to point clouds obtained from 3D vision, suffers from disadvantages such as disorder, sparsity, limited information content, and slow acquisition speed, significantly impacting the efficiency of automated adhesive application. Some automated adhesive application methods only allow for adhesive application on the surface of circular holes using camera-captured images. Another common problem is that when the point cloud data acquired by 3D vision is incomplete, the adhesive application trajectory on the product to be coated becomes unclear, leading to application failure.

[0004] Furthermore, there is a lack of existing research on automatic gluing systems specifically addressing the application of adhesive to round holes in products, particularly the issue of automatic gluing for irregularly shaped round holes with uneven distribution. Summary of the Invention

[0005] Based on the above analysis, the present invention aims to provide an automatic adhesive application method based on 3D vision to solve the problem of automatic adhesive application to irregularly shaped circular holes with uneven distribution.

[0006] This invention provides an automated adhesive application method based on 3D vision, comprising:

[0007] The glue dispensing machine moves its glue outlet to a preset position near the glue-to-be-applied hole, and the robotic arm's end effector moves the 3D camera above the glue-to-be-applied hole to acquire point cloud data of the glue-to-be-applied hole.

[0008] The point cloud of the hole to be coated with adhesive is pre-processed;

[0009] Extract the depth information of the circular hole from the 3D point cloud at the bottom of the circular hole in the preprocessed point cloud;

[0010] The pre-processed point cloud of holes to be coated with adhesive is sequentially subjected to closing operation, bilateral filtering, and extraction of the edge contour of the circular holes.

[0011] The circular hole is fitted using a circular hole fitting algorithm based on the extracted circular hole contour edge, and the center coordinates and radius of the fitted circle are output. The amount of adhesive is calculated based on the circular hole radius, circular hole depth and adhesive thickness.

[0012] The glue dispensing nozzle of the automatic glue applicator is finely adjusted according to the coordinates of the robotic arm's base coordinate system, and glue is applied according to the calculated amount of glue applied to the circular hole.

[0013] Furthermore, the preprocessing includes:

[0014] Region of interest delineation;

[0015] Downsampling is performed on the point cloud within the region of interest;

[0016] The downsampled point cloud is then filtered.

[0017] Furthermore, the point cloud filtering algorithm employs a conditional filtering algorithm, including:

[0018] Calculate the normal vector of each point in the point cloud, delete points whose normal vectors are greater than a set adjustable threshold, and use the remaining points as the target point cloud for further processing.

[0019] Furthermore, the step of extracting the depth information of the circular hole based on the three-dimensional point cloud at the bottom of the circular hole in the preprocessed point cloud includes:

[0020] Extract the 3D point cloud of the bottom of the circular hole in the camera coordinate system, transform the coordinates of the extracted 3D point cloud of the bottom of the circular hole to the robot arm coordinate system, and calculate the average depth value of the 3D point cloud of the bottom of the circular hole as the actual depth of the circular hole.

[0021] Furthermore, the edge contour of the circular hole is extracted from the bilaterally filtered image, including:

[0022] The image gradient is calculated on the circular hole image after image closing operation and bilateral filtering to identify the approximate outline edge of the hole. Then, the Qstu method is used to set two hysteresis thresholds, high and low, to remove false edges, thereby extracting the actual edge contour of the hole image.

[0023] Furthermore, a closing operation is performed on the circular hole image, including:

[0024] The edge contour of the hole in the circular hole image is dilated and traversed using a 3×3 convolution kernel; the edge contour of the hole in the dilated circular hole image is then eroded using the same 3×3 convolution kernel.

[0025] Furthermore, bilateral filtering is performed on the circular hole image after the closing operation, including:

[0026]

[0027]

[0028]

[0029] w = w d (i, j, k, l) × w r (i, j, k, l),

[0030] Where f(k, l) represents the grayscale value function of the corresponding pixel within the sliding window range, w d For the position domain Gaussian function, w r The grayscale Gaussian function is given by δ, where 1 ≤ k ≤ 3, 1 ≤ l ≤ 3, k and l are integers, i and j represent the x and y coordinates of the point cloud in the pixel coordinate system, respectively. d δ r represents the variances of the position domain and grayscale domain, respectively, and p(i,j) is the grayscale value after bilateral filtering.

[0031] Furthermore, the calculation of the image gradient includes:

[0032] By calculating the horizontal gradient information I x Vertical gradient information I y 45° gradient information I 45° and 135° gradient information I 135° Calculate the gradient in the horizontal and vertical directions;

[0033] The gradients in the horizontal and vertical directions are:

[0034]

[0035]

[0036] Furthermore, the edge extraction using the Qtsu thresholding method includes:

[0037] The image is divided into foreground and background by using a grayscale threshold Th. The average grayscale value of the foreground is m1, the average grayscale value of the background is m2, the ratio of pixels classified as foreground to all pixels in the image is p1, and the ratio of pixels classified as background to all pixels in the image is p2. The inter-class variance σ of all pixels in the image is calculated. The maximum value of the inter-class variance is used as the high threshold Th for image segmentation, and Th / 2 is used as the low threshold for image segmentation. Points with grayscale values ​​above the high threshold are identified as edge points; points with grayscale values ​​between the high and low thresholds that are connected to points considered as edge points are also identified as edge points; and points with grayscale values ​​below the low threshold are identified as false edge points and are removed.

[0038] Furthermore, the calculation of the adhesive application amount based on the radius of the hole, the depth of the hole, and the adhesive thickness includes:

[0039] When the radius R of the circular hole is greater than 10 mm, adhesive is first applied to the side surface and bottom of the hole, and then a cylindrical plug with m semi-cylindrical grooves on its side is inserted into the hole using the gripper at the end of the robotic arm. When the radius R of the circular hole is less than or equal to 10 mm, adhesive is only applied to the side surface of the hole, and then a cylindrical plug with m semi-cylindrical grooves on its side is inserted into the hole using the gripper at the end of the robotic arm. The formula for the adhesive volume f is:

[0040]

[0041] s represents the volume of adhesive remaining at the bottom of the plug after uniform application, and s is equal to the volume V of the flow channels around the plug.

[0042]

[0043] Where t is the thickness of the adhesive coating on the sidewall of the hole, r is the radius of the semi-cylinder of the guide channel, h is the hole depth, s is the volume of the remaining adhesive at the bottom of the plug after uniform coating, s is equal to the volume V of the guide channel around the plug, and m≤4.

[0044] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0045] 1. This automated glue application method enables automatic calculation of the glue application trajectory for the robot without manual teaching, saving labor costs, shortening the overall production cycle, improving production efficiency, and facilitating quick and convenient process changes. Since it eliminates the need for manual teaching of the robot's glue application trajectory, it saves pre-production preparation time.

[0046] 2. The method acquires the three-dimensional point cloud model data of the product to be coated in real time. The accuracy of the point cloud model data can be guaranteed by a high-precision three-dimensional camera. The method of the present invention can accurately identify the outline and position of the product to be coated and perform circular hole fitting in the case of three-dimensional point cloud defects, thereby improving the robustness of the method.

[0047] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0048] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0049] Figure 1 This is a flowchart of an automatic glue application method provided in one embodiment of the present invention;

[0050] Figure 2 This is a flowchart of a visual algorithm provided in one embodiment of the present invention. Detailed Implementation

[0051] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0052] A specific embodiment of the present invention discloses an automated adhesive application method based on 3D vision, such as... Figure 1 As shown. The method includes the following steps:

[0053] Step S1: The glue outlet of the automatic glue applicator moves to the vicinity of the glue-to-be-applied hole according to the preset position, and the end of the robotic arm drives the 3D camera to move above the glue-to-be-applied hole to obtain the point cloud data of the glue-to-be-applied hole.

[0054] Step S11: The automatic glue applicator is controlled by a computer. After the computer sets the glue application position, the glue outlet of the automatic glue applicator moves to the vicinity of the glue application hole according to the instructions transmitted by the computer.

[0055] Step S12: The computer-controlled robotic arm end effector moves the 3D camera above the hole to be coated with adhesive. The parameters of the 3D camera are adjusted and set, such as the projection brightness, number of exposures, exposure time, white balance, etc., in order to obtain the best point cloud shooting effect. The 3D camera acquires the three-dimensional point cloud model data of the circular hole on the surface of the product to be coated with adhesive and transmits it to the computer.

[0056] Step S2: Preprocess the point cloud of the holes to be coated. The preprocessing includes: region of interest delineation, downsampling, and point cloud filtering. Specifically, it includes steps S21-S23.

[0057] Step S21: Delineate the region of interest.

[0058] Defining the region of interest ensures that the point cloud only covers information about the circular holes, filtering out useless point clouds. Filtering out useless point clouds can improve the speed of system operation in the later stages.

[0059] First, the 3D point cloud is converted into a NumPy array. The point cloud coordinate information acquired by the 3D camera is referenced to the camera coordinate system. The region of interest (ROI) is a cylindrical area defined by the computer, with the center of the circular hole as the center, radius R, and height H. The point cloud within this ROI is used as the target point cloud for subsequent processing. The radius R and height H can be adjusted by the computer according to the actual working conditions on site.

[0060] Step S22: Downsample the point cloud within the region of interest.

[0061] The downsampling method used is voxel grid downsampling.

[0062] Point cloud downsampling reduces the density of dense point clouds to sparser ones, improving the system's operating speed in later stages. The voxel grid downsampling method divides the point cloud into multiple voxel grids, retaining only one point from the point cloud in each grid. After downsampling, the number of points in the point cloud is checked. If the number of points is less than a preset threshold, the voxel grid size is adjusted until the number of points exceeds the preset threshold; this threshold can be adjusted according to the actual on-site conditions.

[0063] Step S23: Filter the downsampled point cloud.

[0064] The point cloud filtering algorithm employs a conditional filtering algorithm. The normal vector of each point in the point cloud is calculated, and points whose normal vectors are greater than a set adjustable threshold are deleted. The remaining points are used as the target point cloud for subsequent processing.

[0065] Specifically, the normal vector of each point in the downsampled point cloud is calculated as follows:

[0066] By fitting a surface to the points around each point whose normal vector is to be calculated that are within a set radius threshold, and then calculating the normal vector of that point and the normal vectors of its neighboring points on the fitted surface, points whose normal vectors are greater than the set threshold are deleted, and the remaining points are used as the target point cloud for further processing; the set threshold can be adjusted according to the actual working conditions on site.

[0067] Step S3: Extract the depth information of the circular hole from the three-dimensional point cloud at the bottom of the circular hole in the preprocessed point cloud.

[0068] Specifically, the extraction of the hole depth based on the 3D point cloud at the bottom of the hole in the preprocessed point cloud includes:

[0069] Extract the 3D point cloud of the bottom of the circular hole in the camera coordinate system, transform the coordinates of the extracted 3D point cloud of the bottom of the circular hole to the robot arm coordinate system, and calculate the average depth value of the 3D point cloud of the bottom of the circular hole as the actual depth of the circular hole.

[0070] In the camera coordinate system, with the center of the point cloud as the origin, extract the points in the preprocessed point cloud located at the bottom of the circular hole within the radius R preset by the computer. Transform the extracted points to the robotic arm coordinate system. Calculate the z-coordinate of the extracted preprocessed points at the bottom of the circular hole in the robotic arm coordinate system and take the average value, which is the actual depth of the circular hole.

[0071] The origin of the robotic arm's coordinate system is the robotic arm's base, and its coordinates in the lower coordinate system are [X...]. w Y w Z w The coordinate transformation formula between the camera coordinate system and the robot arm coordinate system is as follows:

[0072]

[0073] Where R' is a 3×3 matrix representing spatial coordinate rotation; T is a 3×1 matrix representing spatial coordinate translation, [XC,Y] C Z C [ ] represents the coordinates in the camera coordinate system.

[0074] Depend on By reversing this process, the 3D coordinates of the robotic arm can be derived from the coordinates in the camera coordinate system.

[0075] Step S4: Perform closing operation, bilateral filtering, and extraction of the circular hole edge contour on the pre-processed point cloud of the hole to be coated. Specifically, this includes steps S41-S44.

[0076] S41. Convert the preprocessed point cloud of the hole to be coated to the pixel coordinate system to obtain the circular hole image.

[0077] A camera system has three coordinate systems: pixel coordinates, image coordinates, and camera coordinates. The pixel coordinate system is represented as [u,v], where the origin is the top-left corner of the image, the u-axis is horizontal to the right, and the v-axis is vertically downwards. The image coordinate system, established below the pixel coordinate system and expressed in physical units, is generally represented as (x,y), giving the pixel scale physical meaning. Its origin is the camera principal point, the intersection of the camera's optical axis and the image plane, typically located at the center of the image plane. The x-axis is parallel to the u-axis, and the y-axis is parallel to the v-axis. The camera coordinate system describes the relative position of an object to the camera and is represented as [X...]. C ,Y C Z C ], where the origin is the optical center O of the camera, X CThe axis is parallel to the x-axis, and the y-axis is parallel to the x-axis. C The Z-axis is parallel to the y-axis. C The axis is parallel to the camera's optical axis and perpendicular to the image plane;

[0078] The following formula can be used to sample the coordinates [X] in the camera coordinate system. C ,Y C Z C Transform to pixel coordinates [u,v]:

[0079]

[0080] Where f' is the camera focal length, dx represents the width of one pixel in the x direction, dy represents the width of one pixel in the y direction, and (u0, v0) are the coordinates of the pixel corresponding to the center of the image plane in the pixel coordinate system.

[0081] Step S42: Perform a closing operation on the circular hole image.

[0082] The closing operation includes:

[0083] The edge contour of the hole in the circular hole image is dilated and traversed using a 3×3 convolution kernel; the edge contour of the hole in the dilated circular hole image is then eroded using the same 3×3 convolution kernel.

[0084] Closing operations refer to the sequential processing of an image through dilation and erosion. Dilation followed by erosion helps to bridge narrow gaps, eliminate small holes, and fill breaks in the contour lines.

[0085] Since the preprocessed 3D point cloud still has defects, it can significantly interfere with subsequent algorithm processing. Therefore, these defects need to be repaired to achieve the best detection results. These defects mainly manifest as discontinuities caused by inconsistent reflection at the hole edges, small voids caused by damage to the hole surface, and point cloud breaks or odd shapes caused by other data acquisition issues. To address the interference of these undesirable features on subsequent image processing and edge detection, a closing operation is used to further repair the preprocessed 3D point cloud. This closing operation first uses a 3×3 convolution kernel to dilate and traverse the hole edge contour. This process effectively repairs the broken features at the edge and eliminates small voids, making the point cloud features continuous. Then, by using the same convolution kernel to erode the continuous point cloud features, small cracks are bridged while maintaining the overall size and shape of the hole edge, resulting in a complete and continuous edge contour. These operations achieve the effect of repairing hole-sealing defects such as damage and burrs using an algorithm, effectively improving the adaptability to hole quality during automatic glue application.

[0086] Step S43: Perform bilateral filtering on the circular hole image after the closing operation; including:

[0087]

[0088]

[0089]

[0090] w = w d (i, j, k, l) × w r (i, j, k, l),

[0091] Where f(k, l) represents the grayscale value function of the corresponding pixel within the sliding window range, w d For the position domain Gaussian function, w r The grayscale Gaussian function is given by δ, where 1 ≤ k ≤ 3, 1 ≤ l ≤ 3, k and l are integers, i and j represent the x and y coordinates of the point cloud in the pixel coordinate system, respectively. d δ r The variances in the position domain and grayscale domain are represented sequentially. In a specific embodiment of the present invention, the empirical value of 0.05 obtained through experiments is used, and p(i,j) is the grayscale value of the output after bilateral filtering.

[0092] Bilateral filtering can simultaneously consider the spatial domain information and value domain information of pixels, thus it can preserve complete edges while denoising the image.

[0093] Step S44: Extract the edge contour of the circular hole from the filtered circular hole image;

[0094] Specifically, it includes:

[0095] The Canny edge detection operator is used to calculate the image gradient of the circular hole image after image closing operation and bilateral filtering to identify the approximate outline edge of the hole. Then, the Qstu method is used to set two hysteresis thresholds, high and low, to remove false edges, thereby extracting the actual edge contour of the hole image.

[0096] The calculation of image gradients includes:

[0097] By calculating the horizontal gradient information I x Vertical gradient information I y 45° gradient information I 45° and 135° gradient information I 135° The specific formula for calculating the gradient in the horizontal and vertical directions is as follows:

[0098] I x ={[p(i+1,j-1)+2p(i+1,j)+p(i+1,j+1)]-[p(i-1,j-1)+2p(i-1,j)+p(i-1,j+1)]} / 4,

[0099] I y ={[p(i-1,j+1)+2p(i,j+1)+p(i+1,j+1)]-[p(i-1,j-1)+2p(i,j-1)+p(i+1,j-1)]} / 4,

[0100] I 45° ={[2p(i+1,j+1)+p(i,j+1)+p(i+1,j)]-[2p(i-1,j-1)+p(i-1,j)+p(i,j-1)]} / 4,

[0101] I 135° ={[2p(i+1,j-1)+p(i,j-1)+p(i+1,j)]-[2p(i-1,j+1)+p(i-1,j)+p(i,j+1)]} / 4,

[0102] The formulas for calculating the gradient in the horizontal and vertical directions are:

[0103]

[0104]

[0105] By adding gradient information in both 45° and 135° directions, we can minimize the detection of a large number of false edges while losing true edge information, thus improving the accuracy of edge detection.

[0106] After the gradient calculation is completed, the approximate edge contour of the hole image can be found. However, this contour contains pseudo-edges such as isolated lines, broken lines, and bifurcation lines, which need to be removed by hysteresis thresholding to extract the actual edge contour of the hole image. The hysteresis threshold is divided into two thresholds, high and low. Points with gray values ​​above the high threshold are considered edge points; points connected to points considered as edge points are also considered edge points if they are between the high and low thresholds; points below the low threshold are considered as pseudo-edge points and are removed. Setting the high and low thresholds using the Qstu method can improve computational efficiency. This algorithm divides the image into foreground and background parts through image gray-level thresholding. The gray-level threshold for image segmentation is set to Th. The average gray level of the foreground is m1, and the ratio of pixels classified as foreground to all pixels in the image is p1; the average gray level of the background is m2, and the ratio of pixels classified as background to all pixels in the image is p2; let the global gray-level mean of the image be mG, and the inter-class variance be denoted as σ, then:

[0107] p1*m1+p2*m2=mG (1)

[0108] p1+p2=1 (2)

[0109] σ=p1(m1-mG) 2+p2(m2-mG) 2 (3)

[0110] From equations (1)-(3), we can derive:

[0111] σ = p1 * p2(m1 - m2) 2 (4)

[0112] The inter-class variance of all pixels is calculated according to the above formula. The inter-class variance represents the degree of fluctuation between the foreground and the background. The greater the fluctuation, the more obvious the distinction between the foreground and the background. Taking the maximum value of the inter-class variance as the gray threshold Th for image segmentation can maximize the segmentation of the foreground and the background, thereby obtaining the required high threshold. The low threshold Ti is half of the high threshold. Since mG is a constant for an image, the maximum value of the inter-class variance can be obtained by transforming equation (4) into a function of m1. The maximum value of the function can be obtained by taking the derivative or finding the extreme value of the function. After obtaining the high and low thresholds, the approximate edge contour of the image can be further refined to obtain the true edge. The edge contour has eliminated redundant line segments or unclosed branches. After non-maximum suppression, non-edge points can be effectively suppressed, and the edge contour refinement is finally completed, providing a good foundation for contour fitting.

[0113] Step S5: Fit the extracted circular hole contour edge using a circular hole fitting algorithm, and output the center coordinates and radius of the fitted circle. Calculate the amount of adhesive applied based on the circular hole radius, circular hole depth, and adhesive thickness.

[0114] The steps of the circular hole fitting algorithm are as follows:

[0115] Step S51: Convert the image data in the pixel coordinate system obtained in step S4 into an M×N array K in the camera coordinate system. The element values ​​of K represent the three-dimensional coordinates of each point in the camera coordinate system.

[0116] Step S52: Calculate the average Y-axis coordinate of each column element in array K. Calculate the variance σ of the points in the extracted point cloud. If σ is greater than the preset threshold S, take another picture; otherwise, continue with the next steps.

[0117] If σ is greater than the preset threshold S, the photo is considered to be significantly skewed, and a new photo is taken.

[0118] Step S53, with The resulting 1×N array is denoted as L. Calculate the average value of array L.

[0119] Step S54: Calculate the average X-axis coordinate of each element in array K. by The resulting 1×M array is denoted as Q. Calculate the average value of array Q.

[0120] Step S55 The coordinates projected onto the robotic arm's coordinate system are the coordinates of the center point O of the fitted circle.

[0121] Step S56: Convert the points located on the edge of the hole in array K to the coordinate system of the robotic arm, calculate the Euclidean distance r' between the points located on the edge of the hole in array K and the center O of the fitted circle, and store it in the set Ri; the number with the largest proportion in the set Ri is the radius R of the fitted circle.

[0122] The calculation of adhesive application amount based on the radius of the hole, the depth of the hole, and the adhesive thickness includes:

[0123] When the radius R of the circular hole is greater than 10 mm, adhesive is first applied to the side surface and bottom of the hole, and then a cylindrical plug with m semi-cylindrical grooves on its side is inserted into the hole using the gripper at the end of the robotic arm. When the radius R of the circular hole is less than or equal to 10 mm, adhesive is only applied to the side surface of the hole, and then a cylindrical plug with m semi-cylindrical grooves on its side is inserted into the hole using the gripper at the end of the robotic arm. The formula for the adhesive volume f is:

[0124]

[0125] s represents the volume of adhesive remaining at the bottom of the plug after uniform application, and s is equal to the volume V of the flow channels around the plug.

[0126]

[0127] Where t is the thickness of the adhesive coating on the sidewall of the hole, r is the radius of the semi-cylinder of the guide channel, h is the hole depth, s is the volume of the remaining adhesive at the bottom of the plug after uniform coating, s is equal to the volume V of the guide channel around the plug, and m≤4.

[0128] Step S6: The glue outlet of the automatic glue applicator is finely adjusted according to the coordinates of the robotic arm's base coordinate system, and glue is applied according to the calculated amount of glue applied to the circular hole.

[0129] Compared with existing technologies, the 3D vision-based automatic glue application method provided in this embodiment realizes automatic calculation of the glue application trajectory for a robot that requires no manual teaching, saving labor costs, shortening the overall production cycle, improving production efficiency, and allowing for quick and convenient process changes. Since no manual teaching of the robot's glue application trajectory is needed, pre-production preparation time is saved. By acquiring the 3D point cloud model data of the product to be glued in real time, the accuracy of this point cloud model data can be guaranteed by a high-precision 3D camera. The method of this invention can accurately identify the contour and position of the product to be glued, and perform circular hole fitting even in the case of 3D point cloud defects, improving the robustness of the method.

[0130] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0131] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An automated adhesive application method based on 3D vision, characterized in that, include: The glue dispensing machine moves its glue outlet to a preset position near the glue-to-be-applied hole, and the robotic arm's end effector moves the 3D camera above the glue-to-be-applied hole to acquire point cloud data of the glue-to-be-applied hole. The point cloud of the hole to be coated with adhesive is pre-processed; Extract the depth information of the circular hole from the 3D point cloud at the bottom of the circular hole in the preprocessed point cloud; The pre-processed point cloud of holes to be coated with adhesive is sequentially subjected to closing operation, bilateral filtering, and extraction of the edge contour of the circular holes. The circular hole is fitted using a circular hole fitting algorithm based on the extracted circular hole contour edge, and the center coordinates and radius of the fitted circle are output. The amount of adhesive is calculated based on the circular hole radius, circular hole depth and adhesive thickness. The steps of the circular hole fitting algorithm are as follows: Step S1: Convert the image data in the pixel coordinate system into an M×N array K in the camera coordinate system. The element values ​​of K represent the three-dimensional coordinates of each point in the camera coordinate system. Step S2: Calculate the average Y-coordinate of each column element in array K. Calculate the variance σ of the points in the extracted point cloud. If σ is greater than the preset threshold S, take another picture; otherwise, continue with the next steps. If σ is greater than the preset threshold S, it is considered that the photograph is too skewed and a new photograph is taken. Step S3, with The resulting 1×N array is denoted as L. Calculate the average value of array L. Step S4: Calculate the average X-axis coordinate of each element in array K. by The resulting 1×M array is denoted as Q. Calculate the average value of array Q. Step S5 The coordinates projected onto the robot arm's coordinate system are the coordinates of the center point O of the fitted circle. Step S6: Transform the points in array K located on the edge of the circular hole into the robotic arm coordinate system, calculate the Euclidean distance r' between the points in array K located on the edge of the circular hole and the center O of the fitted circle, and store the result in the array R. i Data set R i The number that accounts for the largest proportion is the radius R of the fitted circle; The glue dispensing nozzle of the automatic glue applicator is finely adjusted according to the coordinates of the robotic arm's base coordinate system, and glue is applied according to the calculated amount of glue applied to the circular hole.

2. The automatic adhesive application method based on 3D vision according to claim 1, characterized in that, The preprocessing includes: Region of interest delineation; Downsampling is performed on the point cloud within the region of interest; The downsampled point cloud is then filtered.

3. The automatic adhesive application method based on 3D vision according to claim 2, characterized in that, The point cloud filtering algorithm employs a conditional filtering algorithm, including: Calculate the normal vector of each point in the point cloud, delete points whose normal vectors are greater than a set adjustable threshold, and use the remaining points as the target point cloud for further processing.

4. The automatic adhesive application method based on 3D vision according to claim 1, characterized in that, The extraction of hole depth information based on the 3D point cloud at the bottom of the hole in the preprocessed point cloud includes: Extract the 3D point cloud of the bottom of the circular hole in the camera coordinate system, transform the coordinates of the extracted 3D point cloud of the bottom of the circular hole to the robot arm coordinate system, and calculate the average depth value of the 3D point cloud of the bottom of the circular hole as the actual depth of the circular hole.

5. The automatic adhesive application method based on 3D vision according to claim 1, characterized in that, Extract the edge contour of the circular hole from the bilaterally filtered image, including: The image gradient is calculated on the circular hole image after image closing operation and bilateral filtering to identify the approximate outline edge of the hole. Then, the Qstu method is used to set two hysteresis thresholds, high and low, to remove false edges, thereby extracting the actual edge contour of the hole image.

6. The automatic adhesive application method based on 3D vision according to claim 5, characterized in that, Performing a closing operation on a circular hole image includes: The edge contour of the hole in the circular hole image is dilated and traversed using a 3×3 convolution kernel; the edge contour of the hole in the dilated circular hole image is then eroded using the same 3×3 convolution kernel.

7. The automatic adhesive application method based on 3D vision according to claim 1, characterized in that, The closed-loop image of the circular hole is subjected to bilateral filtering, including: w=w d (i,j,k,l)×w r (i,j,k,l), Where f(k, l) represents the grayscale value function of the corresponding pixel within the sliding window range, w d For the position domain Gaussian function, w r The grayscale Gaussian function is given by δ, where 1 ≤ k ≤ 3, 1 ≤ l ≤ 3, k and l are integers, i and j represent the x and y coordinates of the point cloud in the pixel coordinate system, respectively. d δ r represents the variances of the position domain and grayscale domain, respectively, and p(i,j) is the grayscale value after bilateral filtering.

8. The automatic adhesive application method based on 3D vision according to claim 5, characterized in that, The calculation of image gradients includes: By calculating the horizontal gradient information I x Vertical gradient information I y 45° gradient information I 45° and 135° gradient information I 135° Calculate the gradient in the horizontal and vertical directions; The gradients in the horizontal and vertical directions are:

9. The automatic adhesive application method based on 3D vision according to claim 5, characterized in that, The step of removing false edges by setting two hysteresis thresholds (high and low) using the Qstu method to extract the actual edge contour of the hole image includes: The image is divided into foreground and background by using a grayscale threshold Th. The average grayscale value of the foreground is m1, the average grayscale value of the background is m2, the ratio of pixels classified as foreground to all pixels in the image is p1, and the ratio of pixels classified as background to all pixels in the image is p2. The inter-class variance σ of all pixels in the image is calculated. The maximum value of the inter-class variance is used as the high threshold Th for image segmentation, and Th / 2 is used as the low threshold for image segmentation. Points with grayscale values ​​above the high threshold are identified as edge points; points with grayscale values ​​between the high and low thresholds that are connected to points considered as edge points are also identified as edge points; and points with grayscale values ​​below the low threshold are identified as false edge points and are removed.

10. The automatic adhesive application method based on 3D vision according to claim 1, characterized in that, The calculation of adhesive application amount based on the radius of the hole, the depth of the hole, and the adhesive thickness includes: When the radius R of the circular hole is greater than 10 mm, adhesive is first applied to the side surface and bottom of the hole, and then a cylindrical plug with m semi-cylindrical grooves on its side is inserted into the hole using the gripper at the end of the robotic arm. When the radius R of the circular hole is less than or equal to 10 mm, adhesive is only applied to the side surface of the hole, and then a cylindrical plug with m semi-cylindrical grooves on its side is inserted into the hole using the gripper at the end of the robotic arm. The formula for the adhesive volume f is: s represents the volume of adhesive remaining at the bottom of the plug after uniform application, and s is equal to the volume V of the flow channels around the plug. Where t is the thickness of the adhesive coating on the sidewall of the hole, r is the radius of the semi-cylinder of the guide channel, h is the hole depth, s is the volume of the remaining adhesive at the bottom of the plug after uniform coating, s is equal to the volume V of the guide channel around the plug, and m≤4.

Images