Method, device and robot for controlling a glue gun

Abstract

The application discloses a kind of glue gun control method, device and robot, applied to industrial automation and intelligent manufacturing technical field, the method includes: by in the process of glue gun nozzle to workpiece is carried out glue spraying operation, control laser to workpiece surface projection laser line, and obtain the laser line image that camera is photographed;According to laser line image, determine the three-dimensional point cloud information of laser line center line;According to three-dimensional point cloud information and the current nozzle coordinate information of glue gun nozzle, determine the local geometric feature parameter of workpiece surface;According to local geometric feature parameter and current nozzle coordinate information, determine the current gap height between glue gun nozzle and workpiece surface, and according to current gap height, glue gun nozzle height is controlled. Realize the real-time measurement and adaptive control of the gap between glue gun nozzle and workpiece, improve the following ability of glue gun nozzle under complex curved surface, prevent glue gun collision damage.

Description

Technical Field

[0001] This invention relates to the field of industrial automation and intelligent manufacturing technology, and in particular to a method, device and robot for controlling a glue gun. Background Technology

[0002] In the manufacturing process of automotive body-in-white, adhesive application robots are widely used for sealing, bonding, and vibration damping of sheet metal parts. During the adhesive application process, the gap height between the glue gun nozzle and the sheet metal surface is strictly controlled (typically 2-5mm). Too small a gap can easily cause the nozzle to collide with the sheet metal, resulting in equipment damage; too large a gap can cause adhesive strip breakage, hairline adhesive pulling, or insufficient adhesive application, severely affecting the quality of the finished product. Therefore, real-time monitoring and stable control of the gap between the glue gun nozzle and the sheet metal is a crucial step in ensuring adhesive application quality.

[0003] In existing technologies, fixed height settings, laser displacement sensors, contact probes, or 3D scanning systems are typically used to detect the height of the workpiece surface. However, these methods affect process stability and real-time performance, making it impossible to achieve online high-precision adaptive adjustment. Summary of the Invention

[0004] In view of the above problems, a method, apparatus, and robot for controlling glue guns are proposed to overcome or at least partially solve the above problems, including: In a first aspect, embodiments of the present invention provide a method for controlling a glue gun, wherein the glue gun is provided with a glue gun nozzle, and the glue gun nozzle is provided with a camera and a laser, the method comprising: During the process of applying glue to the workpiece using the glue gun nozzle, the laser is controlled to project a laser line onto the surface of the workpiece, and the laser line image is captured by the camera. Based on the laser line image, determine the three-dimensional point cloud information of the laser line centerline; Based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle, the local geometric feature parameters of the workpiece surface are determined; Based on the local geometric feature parameters and the current nozzle coordinate information, the current gap height between the glue gun nozzle and the workpiece surface is determined, and the height of the glue gun nozzle is controlled according to the current gap height.

[0005] Optionally, based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle, the local geometric feature parameters of the workpiece surface are determined, including: Based on the current nozzle coordinate information, determine the coordinate information of the projection point of the glue gun nozzle onto the workpiece surface; Based on the coordinate information of the projection points, the neighborhood of the workpiece surface is determined; Based on the 3D point cloud information, a surface model is performed on the neighborhood to obtain the local geometric feature parameters of the neighborhood.

[0006] Optionally, the three-dimensional point cloud information includes the three-dimensional coordinate information of multiple laser points on the workpiece surface. Based on the three-dimensional point cloud information, surface modeling is performed on the neighborhood to obtain the local geometric feature parameters of the neighborhood, including: From the three-dimensional point cloud information, determine the target laser point corresponding to the neighborhood and the three-dimensional coordinate information of the target laser point; Determine the weight information of the target laser point; Based on the weight information and the three-dimensional coordinate information of the target laser point, a surface model is performed on the neighborhood to obtain the local geometric feature parameters of the neighborhood.

[0007] Optionally, determining the current gap height between the glue gun nozzle and the workpiece surface based on the local geometric feature parameters and the current nozzle coordinate information includes: Based on the local geometric feature parameters, determine the elevation information corresponding to the projection point; Based on the local geometric feature parameters, determine the gradient information corresponding to the projection point, and based on the gradient information, obtain the surface unit normal vector information corresponding to the projection point; Based on the elevation information, the surface unit normal vector information, and the current nozzle coordinate information, the current gap height between the glue gun nozzle and the workpiece surface is obtained.

[0008] Optionally, based on the laser line image, the three-dimensional point cloud information of the laser line centerline is determined, including: Based on the laser line image, determine the pixel coordinates of the center line of the laser line on the workpiece surface; Based on the camera's calibration data, the pixel coordinates are converted into three-dimensional coordinates in the laser coordinate system to obtain the three-dimensional point cloud information of the laser line centerline.

[0009] Optionally, the height of the glue gun nozzle is controlled according to the current gap height, including: Obtain the target gap height; The height compensation amount is determined based on the current gap height and the target gap height; The height of the glue gun nozzle is controlled according to the height compensation amount.

[0010] Optionally, the height compensation amount is determined based on the current gap height and the target gap height, including: Determine the height error between the current gap height and the target gap height; Based on the height error, determine the height compensation amount.

[0011] Optionally, the method further includes: During the process of controlling the height of the glue gun nozzle, the glue gun nozzle is protected against impact.

[0012] Optionally, based on the current gap height, anti-collision protection is provided for the glue gun nozzle, including: A multi-level threshold is obtained, the current gap height is compared with the multi-level threshold, a comparison result is obtained, and the glue gun nozzle is protected against collision based on the comparison result.

[0013] And / or, determine the remaining time before the glue gun nozzle collides with the workpiece surface, and provide anti-collision protection for the glue gun nozzle based on the remaining time.

[0014] Secondly, embodiments of the present invention provide a glue gun control device, wherein the glue gun is provided with a glue gun nozzle, and the glue gun nozzle is provided with a camera and a laser, the device comprising: The laser line image acquisition module is used to control the laser to project a laser line onto the surface of the workpiece during the glue gun nozzle spraying glue onto the workpiece, and to acquire the laser line image captured by the camera. A three-dimensional point cloud information determination module is used to determine the three-dimensional point cloud information of the center line of the laser line based on the laser line image; The geometric feature parameter acquisition module is used to determine the local geometric feature parameters of the workpiece surface based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle. The height control module is used to determine the current gap height between the glue gun nozzle and the workpiece surface based on the local geometric feature parameters and the current nozzle coordinate information, and to control the height of the glue gun nozzle based on the current gap height.

[0015] The embodiments of the present invention have the following advantages: In this embodiment of the invention, during the glue spraying operation on the workpiece using a glue gun nozzle, a laser is controlled to project a laser line onto the workpiece surface, and a laser line image captured by a camera is acquired. Based on the laser line image, the three-dimensional point cloud information of the laser line centerline is determined. Based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle, the local geometric feature parameters of the workpiece surface are determined. Based on the local geometric feature parameters and the current nozzle coordinate information, the current gap height between the glue gun nozzle and the workpiece surface is determined, and the height of the glue gun nozzle is controlled based on the current gap height. This achieves visual inspection through the combination of laser and camera to model the local curved surface of the workpiece, and simultaneously uses height compensation to control the glue gun nozzle, realizing real-time measurement and adaptive control of the gap between the glue gun nozzle and the workpiece. This improves the following ability of the glue gun nozzle on complex curved surfaces and prevents glue gun collision damage. Attached Figure Description

[0016] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a flowchart of the steps of a glue gun control method provided in some embodiments of the present invention; Figure 2 This is a schematic diagram of an adhesive coating measuring device provided in some embodiments of the present invention; Figure 3 This is a schematic diagram of a laser line measurement for adhesive application provided in some embodiments of the present invention; Figure 4 This is a schematic diagram of a laser measurement cross-section of an adhesive strip provided in some embodiments of the present invention; Figure 5 This is a structural block diagram of a glue gun control device provided in some embodiments of the present invention. Detailed Implementation

[0018] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0019] In this embodiment of the invention, a model of the local curved surface of the workpiece is created by visual inspection combining laser and camera. At the same time, the height compensation amount is used to control the glue gun nozzle, realizing real-time measurement and adaptive control of the gap between the glue gun nozzle and the workpiece. This improves the glue gun nozzle's ability to follow on complex curved surfaces and prevents the glue gun from being damaged by collision.

[0020] Reference Figure 1 The diagram shows a flowchart of a glue gun control method according to some embodiments of the present invention. The glue gun is equipped with a glue gun nozzle, and the glue gun nozzle is equipped with a camera and a laser.

[0021] The camera and laser are mounted in a hexagonal disk above the glue gun nozzle and move with the nozzle. Figure 2 As shown, six laser beams arranged at 60-degree angles to each other are configured to project laser stripes onto the surfaces of the adhesive strip and the workpiece. A camera captures images in real time. Figure 3 As shown.

[0022] In some examples, the laser coverage area can include both the adhesive strip surface and the workpiece surface, allowing for simultaneous acquisition of the adhesive strip morphology and the gap between the glue gun nozzle and the workpiece. The laser on the adhesive strip surface can acquire the real-time three-dimensional morphology of the adhesive strip, from which the width, height, and cross-sectional shape of the applied adhesive strip can be extracted, such as... Figure 4 As shown.

[0023] Specifically, it may include the following steps: Step 101: During the glue gun nozzle's glue spraying operation on the workpiece, the laser is controlled to project a laser line onto the workpiece surface, and the laser line image captured by the camera is obtained.

[0024] In some examples, the glue gun nozzle spraying glue onto the workpiece refers to the process where the glue gun nozzle is in the glue dispensing state and sprays glue onto the workpiece's glue application area according to a preset trajectory. During this process, the glue gun nozzle needs to maintain a preset height. The laser line image refers to the image of the laser line falling on the workpiece surface.

[0025] When the glue gun nozzle is detected to be operating, a signal is sent to the laser, controlling it to project a laser line onto the surface of the workpiece being processed, and controlling an industrial camera to capture the laser line image of the laser stripe in real time. In some examples, the laser line image can be a laser distortion image, which refers to a two-dimensional photograph captured by the camera after the vertical laser line is projected onto the workpiece surface and bends or deforms due to the unevenness of the workpiece surface. In some examples, a frame rate greater than 60fps (frames per second) is required to ensure that no frames are dropped during movement.

[0026] Step 102: Determine the three-dimensional point cloud information of the center line of the laser line based on the laser line image.

[0027] In some examples, a 3D point cloud refers to a set of discrete point data with 3D spatial coordinates. In this invention, the coordinate information is assumed to be a coordinate system with the laser as the origin.

[0028] After acquiring the laser line image captured by the camera, the points on the workpiece surface illuminated by the laser line in the image can be determined. Then, the pixel coordinates of these points in the image are determined, and coordinate transformation is performed to obtain the three-dimensional point cloud information of the laser line centerline.

[0029] In some examples, the acquired raw laser line images can be preprocessed to eliminate the influence of environment and motion on accuracy. For example, the acquired raw image (the image acquired here is a grayscale image) can be preprocessed using a dual filtering scheme of Gaussian filtering and median filtering to eliminate the influence of ambient light interference and motion jitter on the accuracy of laser stripe recognition. The following formula can be used:

[0030] in, This represents the grayscale value of the original image at pixel location (x, y). The grayscale value of the image after filtering (convolution operation); The weights of the filter kernel (such as a Gaussian kernel or a mean kernel); i, j are the horizontal and vertical offsets of the convolution window, respectively; k is half the size of the convolution window, for example... =1 indicates a 3×3 convolution window size.

[0031] In some embodiments of the present invention, determining the three-dimensional point cloud information of the center line of the laser line based on the laser line image includes: Sub-step 11: Determine the pixel coordinates of the center line of the laser line on the workpiece surface based on the laser line image.

[0032] In some examples, the laser line centerline refers to a laser line composed of the strongest laser point (centroid); pixel coordinates refer to the position coordinates of each pixel in the image on the two-dimensional image plane.

[0033] After acquiring the laser line image, the two-dimensional coordinate information of each point on the workpiece surface illuminated by the center line of the laser line in the image can be obtained first. Based on the two-dimensional coordinate information of each point, the pixel coordinates of the center line of the laser line on the workpiece surface can be determined.

[0034] In some examples, after preprocessing the original laser line image, the sub-pixel accuracy of the laser centerline can be achieved by using the gray-scale centroid method to locate the pixel coordinates of the laser line centerline, line by line. In some examples, the following formula can be used:

[0035] In this formula, The coordinates (center position) of the centroid to be calculated; Let be the position (coordinates) of the i-th pixel; Let be the grayscale value (intensity) of the i-th pixel.

[0036] In some embodiments of the present invention, determining the three-dimensional point cloud information of the center line of the laser line based on the laser line image includes: Sub-step 12: Based on the camera calibration data, the pixel coordinates are converted into three-dimensional coordinates in the laser coordinate system to obtain the three-dimensional point cloud information of the laser line centerline.

[0037] In some examples, calibration data refers to the camera's intrinsic and extrinsic parameters, with the camera and laser mounted on the same rigid bracket, fixed to the end of the glue gun nozzle.

[0038] In practical applications, the intrinsic and extrinsic parameters of the camera are obtained. Using the camera's imaging geometry model, the correspondence between pixel coordinates and sensor coordinates is established through camera calibration. The pixel coordinates are then transformed into three-dimensional coordinates in the laser coordinate system, yielding the three-dimensional point cloud information of the laser line's centerline. In some examples, the following formula can be used:

[0039] K is the camera intrinsic parameter matrix, including focal length and principal point position, used to complete the projection from pixel coordinates to camera coordinates; The extrinsic parameter matrix describes the rotation and translation relationship between the camera coordinate system and the physical coordinate system; s is the homogeneous coordinate scaling factor; (u,v) are the pixel coordinates, ( , , ) represents the coordinates in the laser coordinate system.

[0040] Step 103: Determine the local geometric feature parameters of the workpiece surface based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle.

[0041] In some examples, local geometric feature parameters refer to parameters of surface morphology, including the tilt, bending, and twisting characteristics of the local surface.

[0042] In practical applications, the nozzle coordinate information of the glue gun nozzle is obtained, and the nozzle coordinate information is combined with the three-dimensional point cloud information on the corresponding workpiece surface to model the local workpiece surface corresponding to the nozzle coordinate, and the local geometric feature parameters of the workpiece surface near the glue gun nozzle are obtained.

[0043] In some embodiments of the present invention, the local geometric feature parameters of the workpiece surface are determined based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle, including: Sub-step 21: Based on the current nozzle coordinate information, determine the coordinate information of the projection point of the glue gun nozzle onto the workpiece surface.

[0044] In some examples, the glue gun nozzle is mounted on the glue-applying robot, and the current nozzle coordinate information refers to the coordinate information in the sensor coordinate system.

[0045] In practical applications, the current coordinates of the glue gun nozzle are obtained, the coordinates of the nozzle's center point in the sensor coordinate system are determined, the nozzle center is projected onto the workpiece plane, and the coordinates of the projected point are obtained. In some examples, for instance, it is projected onto a workpiece reference plane, and its XY plane components are obtained.

[0046] in, This refers to the coordinate information of the projection point. Let be the coordinates of the glue gun nozzle center on the x-axis of the sensor coordinate system. The coordinates of the glue gun nozzle center on the y-axis of the sensor coordinate system are given.

[0047] In some examples, three coordinate systems are involved, such as the laser coordinate system Σ_s: using the hexagonal disk device as the reference coordinate system, the laser and camera are fixedly mounted, maintaining a rigid and constant pose relationship. The robot base coordinate system Σ_r: defined by the robot control system, used to describe the spatial pose of the robot's end effector. The glue gun nozzle TCP coordinate system: Σ_tcp is located at the working point of the glue gun nozzle, and the robot outputs its position and orientation in the robot base coordinate system Σ_r in real time.

[0048] The 3D point cloud data is obtained in the sensor coordinate system, and the real-time position of the center point of the glue gun nozzle is output by the robot controller in the robot base coordinate system, which transforms the glue gun nozzle coordinate information in the robot base coordinate system to the sensor coordinate system.

[0049] In some examples, the sensor coordinates and the robot base coordinate system are calibrated. The coordinate information in the robot coordinate system is transformed to obtain the coordinate information of the glue gun nozzle in the sensor coordinate system. For example, during the system installation phase, the rigid transformation relationship between the laser coordinate system Σ_s and the robot base coordinate system Σ_r is determined through offline calibration. The transformation matrix is ​​expressed as:

[0050] in, For rotation matrix, It is a translation vector.

[0051] The robot provides the real-time position of the glue gun nozzle center in the robot's base coordinate system Σ_r:

[0052] By using the pre-defined coordinate transformation relationship between the robot's base coordinate system and the sensor coordinate system, the system is transformed to the sensor coordinate system Σ_s:

[0053] in, That is, the coordinates obtained in the sensor coordinate system:

[0054] After the coordinate unification process described above, the surface modeling, normal vector calculation, and current gap height calculation between the glue gun nozzle and the workpiece surface are all performed in the sensor coordinate system Σ_s, thereby ensuring the consistency and accuracy of the geometric calculations.

[0055] In some embodiments of the present invention, the local geometric feature parameters of the workpiece surface are determined based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle, including: Sub-step 22: Determine the neighborhood of the workpiece surface based on the coordinate information of the projection point.

[0056] In practical applications, the projection point is on the surface of the workpiece. The set of points on the workpiece surface within a fixed radius of the projection point is selected as the neighborhood.

[0057] Sub-step 23: Based on the three-dimensional point cloud information, perform surface modeling on the neighborhood to obtain the local geometric feature parameters of the neighborhood.

[0058] In practical applications, we can first extract the 3D point cloud information of each point in the neighborhood, and then model and solve the local surface of the neighborhood based on the 3D point cloud data of each point in the neighborhood to obtain the feature parameters of the local geometry.

[0059] In some embodiments of the present invention, the three-dimensional point cloud information includes the three-dimensional coordinate information of multiple laser points on the surface of the workpiece. Based on the three-dimensional point cloud information, surface modeling is performed on the neighborhood to obtain the local geometric feature parameters of the neighborhood, including: Sub-step 231: Determine the target laser point corresponding to the neighborhood and the three-dimensional coordinate information of the target laser point from the three-dimensional point cloud information.

[0060] In some examples, the target laser point refers to the laser points within the neighborhood. Not all points in the neighborhood necessarily have 3D point cloud information; the key is to identify the target laser point within the neighborhood.

[0061] After determining the neighborhood of the projection point, we can first determine each point in the neighborhood, find the points that are repeated with the 3D point cloud information, determine the target laser point, and then obtain the 3D information of these target laser points on the workpiece surface from the 3D point cloud information.

[0062] Sub-step 232: Determine the weight information of the target laser point.

[0063] After obtaining the information of the target laser point, a weight is determined based on the distance between the target laser point and the projection point. Target laser points closer to the projection point have a higher weight, ensuring that the constructed surface conforms to the actual local state directly below the glue gun nozzle. In some examples, such as using the glue gun nozzle projection point... Centered on a point, a neighborhood of radius R is defined. Each sample point can be weighted using the following formula:

[0064] in, Bandwidth (typically 3~8mm); R: neighborhood radius; K(r): cubic kernel; :max(.,0) takes the largest number between the variable and 0. T is the transpose sign, which means converting the row vector into a column vector.

[0065] Sub-step 233: Based on the weight information and the three-dimensional coordinate information of the target laser point, perform surface modeling on the neighborhood to obtain the local geometric feature parameters of the neighborhood.

[0066] In practical applications, the equations of the local surface are constructed, a relevant design matrix is ​​built using 3D coordinate information, and the optimization equation is solved by combining weights to obtain the correlation coefficients. In some examples, weights and the least squares method are combined for solving, and the following formula can be used:

[0067] The correlation coefficient is represented as a vector. , The coefficient of the first-order term in x represents the slope of the surface in the x-direction. The coefficient of the first-order term y describes the slope of the surface in the y-direction. The coefficients of the xy cross term describe the degree of distortion of the surface. Describes the degree of curvature of the surface in the x-direction. Describes the degree of curvature of the surface in the y-direction. The constant term in the corresponding equation determines the reference elevation of the surface.

[0068] Construct the relevant design matrix Solve the optimization equation:

[0069]

[0070] Where argmin is the value of the independent variable that minimizes the function, and in this formula, it represents the surface coefficient vector that minimizes the sum of squared fitting errors. The value, The weights are for i laser points. This is the sum of squares of the fitting error. It is a weighted diagonal matrix. To design the transpose of the matrix.

[0071] Step 104: Determine the current gap height between the glue gun nozzle and the workpiece surface based on the local geometric feature parameters and the current nozzle coordinate information, and control the height of the glue gun nozzle based on the current gap height.

[0072] In some examples, the current clearance height refers to the actual normal spacing, which is the normal distance relative to the local curved surface of the workpiece.

[0073] After obtaining the local geometric feature parameters, the local surface equations can be obtained by solving them. Based on the coordinate information of the projection point, the three-dimensional coordinate information of the projection point on the local surface can be calculated. The current gap height can be determined based on the three-dimensional coordinate information of the projection and the nozzle coordinates. The movement height of the glue gun nozzle can be controlled by the current gap height.

[0074] In some embodiments of the present invention, determining the current gap height between the glue gun nozzle and the workpiece surface based on the local geometric feature parameters and the current nozzle coordinate information includes: Sub-step 31: Determine the elevation information corresponding to the projection point based on the local geometric feature parameters.

[0075] In some examples, elevation information refers to the numerical value of the z-coordinate.

[0076] After obtaining the local geometric feature parameters and determining the shape characteristics of the local surface, the elevation information corresponding to the projection point can be obtained by substituting the xy coordinates of the projection point into the equation. In some examples, the following formula can be used:

[0077] in, It refers to Elevation at the surface of the workpiece.

[0078] In some embodiments of the present invention, determining the current gap height between the glue gun nozzle and the workpiece surface based on the local geometric feature parameters and the current nozzle coordinate information includes: Sub-step 32: Based on the local geometric feature parameters, determine the gradient information corresponding to the projection point, and based on the gradient information, obtain the surface unit normal vector information corresponding to the projection point.

[0079] In some examples, the gradient is a vector that describes the trend of surface change.

[0080] After obtaining the local geometric feature parameters, the gradient at that point can be calculated based on the surface equation and the coordinates of points on the surface. After obtaining the gradient, the unit normal vector of the surface at that point can be calculated based on the gradient information. In some examples, the following formula can be used:

[0081] in, It is the gradient of the surface. It is the partial derivative of z with respect to x and y. It is the x-coordinate of that point on the workpiece. It is the y-coordinate of that point on the workpiece.

[0082] The unit normal n of the surface is:

[0083] Sub-step 33: Based on the elevation information, the surface unit normal vector information, and the current nozzle coordinate information, obtain the current gap height between the glue gun nozzle and the workpiece surface.

[0084] After calculating the current elevation and surface unit normal vector information, the height difference in the z-coordinate is obtained based on the current nozzle coordinates and elevation. Then, the distance between the glue gun nozzle and the workpiece surface in the normal direction is calculated using the surface unit normal vector and the height difference. For example, the following formula can be used:

[0085] in, This refers to the current nozzle coordinate information, where s is the coordinate information of the projection point and d is the current clearance height.

[0086] In some embodiments of the present invention, controlling the height of the glue gun nozzle based on the current gap height includes: Sub-step 41: Obtain the target gap height.

[0087] In some examples, the target gap height refers to the desired gap height between the glue gun nozzle and the workpiece surface, set before the glue gun nozzle begins operation.

[0088] After obtaining the current gap height, the target gap height that the glue gun nozzle needs to be between the local curved surface and the manually set distance during the operation is obtained.

[0089] Sub-step 42: Determine the height compensation amount based on the current gap height and the target gap height.

[0090] In some examples, the height compensation amount refers to the amount of displacement that the end of the glue gun nozzle needs to be adjusted.

[0091] After obtaining the real-time measured current gap height and the preset target gap height, the controller is used to predict the compensation amount that needs to be delivered to the glue gun nozzle when the end of the glue gun nozzle maintains the target gap height.

[0092] In some embodiments of the present invention, determining a height compensation amount based on the current gap height and the target gap height includes: Sub-step 421: Determine the height error between the current gap height and the target gap height.

[0093] After obtaining the current clearance height and the target clearance height, the height error is calculated based on the real-time measured current clearance height and the set target clearance height. In some examples, the following formula can be used:

[0094] Where e(t) is the height error (mm); d(t): the real-time measured gap between the glue gun nozzle and the board surface (mm). The process target clearance (mm), for example, 2.5–4mm, is primarily used to provide an error signal to the controller.

[0095] In some embodiments of the present invention, determining a height compensation amount based on the current gap height and the target gap height includes: Sub-step 422: Determine the height compensation amount based on the height error.

[0096] After determining the height error, a PID controller is introduced, and the height error is input into the PID controller to obtain the feedback compensation amount. Based on the local geometric feature parameters, the shape of the surface is determined, and the current local surface gap height change trend is calculated based on the surface shape. The feedforward compensation amount is then determined based on this change trend. The two compensation amounts are superimposed to determine the final height compensation amount. In some examples, the following formula can be used:

[0097] Where u[k] is the Z-axis compensation amount (mm) at the k-th sampling time; e[k] is the height error at the k-th time; For proportional gain; For integral gain; This is the differential gain; It is an integral term; The sampling period is in milliseconds (ms). This is the control quantity after saturation; This is the inverse integral saturation coefficient, used to prevent the integral term from becoming too large.

[0098] In some examples, the proportional gain Integral gain Differential gain It is not a fixed value, but rather obtained through experimental tuning based on the specific equipment and operating conditions. Specifically, this involves: while maintaining the glue gun nozzle along a predetermined trajectory, applying a step-like height disturbance or simulating a change in the plate height, collecting the dynamic response curve of the glue gun nozzle-plate gap, and gradually adjusting the PID parameters based on indicators such as the system's rise time, overshoot, and steady-state error, so that the system meets the requirements of stability and control accuracy while ensuring response speed.

[0099] In some examples, the compensation also includes feedforward compensation, such as using workpiece geometry priors to correct for changes in the nozzle-plate gap of the glue gun in advance, reducing PID lag and overshoot, thereby making the glue gun nozzle follow more smoothly and the glue application more continuous.

[0100]

[0101] in, is the feedforward compensation amount (mm) at time k; The clearance (mm) is estimated based on the workpiece's geometric model or theory. The target gap.

[0102]

[0103] in, This is the feedback compensation amount based on the real-time gap error; This is the feedforward compensation amount based on the workpiece's geometric priors.

[0104] In some embodiments of the present invention, controlling the height of the glue gun nozzle based on the current gap height includes: Sub-step 43: Control the height of the glue gun nozzle according to the height compensation amount.

[0105] After determining the height compensation amount, the compensation amount is sent to the glue gun nozzle, and the height of the glue gun nozzle is controlled according to the compensation amount.

[0106] The process of controlling the height of the glue gun nozzle also includes: A multi-level threshold is obtained, the current gap height is compared with the multi-level threshold, a comparison result is obtained, and anti-collision protection is provided for the glue gun nozzle based on the comparison result; and / or, the remaining time before the glue gun nozzle collides with the workpiece surface is determined, and anti-collision protection is provided for the glue gun nozzle based on the remaining time.

[0107] In some examples, multi-level thresholds include warning thresholds, safety lower limits, and extreme lower limits, while anti-collision protection prevents the glue gun nozzle from moving beyond the current gap height and colliding with the workpiece surface.

[0108] Because the glue gun nozzle may collide with the workpiece surface due to various reasons during the glue gun control process, various anti-collision protections are needed for the glue gun nozzle during height control.

[0109] During the control of the glue gun nozzle, the current gap height needs to be compared with a threshold in real time. An alarm is triggered when the current gap height is less than the warning threshold; the glue gun nozzle is immediately raised when the current gap height is less than the safety lower limit; and the glue gun nozzle is immediately stopped when the current gap height is less than the extreme lower limit. In some examples, the following formula can be used:

[0110] in, The interval value for triggering the warning; This is the lower safety limit at which the glue gun nozzle must be raised. This is the limit clearance; if the clearance is less than this value, an emergency stop must be initiated immediately. , These are the deviation thresholds (mm) for the warning and safety lower limit, respectively. The target gap (mm). This is the lower limit; if the value is less than this, an emergency stop must be initiated.

[0111] During the glue gun nozzle application process, it's necessary to predict in advance whether the nozzle will collide with the workpiece and adjust the height accordingly. In some examples, TTC (Time to Contact) prediction can be used, primarily to provide early warnings that the nozzle might be less than the robot system's minimum reaction time. At that time, raise the nozzle of the glue gun in advance.

[0112]

[0113] in, This refers to a pre-defined safety gap. The time rate of change of the normal gap. The maximum value is a very small positive value, where d is the current gap height, and max is the maximum value. and The maximum value function between.

[0114] After the collision protection is activated, once the gap returns to a safe range and remains stable for a certain period of time, gradually restore the movement speed of the glue gun nozzle and the glue dispensing flow rate to avoid uneven glue dispensing caused by sudden stops.

[0115] In this embodiment of the invention, during the glue spraying operation on the workpiece using a glue gun nozzle, a laser is controlled to project a laser line onto the workpiece surface, and an image of the laser line is captured by a camera. Based on the laser line image, the three-dimensional point cloud information of the laser line centerline is determined. Based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle, the local geometric feature parameters of the workpiece surface are determined. Based on the local geometric feature parameters and the current nozzle coordinate information, the current gap height between the glue gun nozzle and the workpiece surface is determined, and the height of the glue gun nozzle is controlled based on the current gap height. Through visual inspection combining laser and camera, the local curved surface of the workpiece is modeled, and the glue gun nozzle is controlled using height compensation. This achieves real-time measurement and adaptive control of the gap between the glue gun nozzle and the workpiece, improving the glue gun nozzle's following ability on complex curved surfaces and preventing glue gun collision damage.

[0116] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0117] Reference Figure 5 The diagram illustrates a structural block diagram of a glue gun control device according to some embodiments of the present invention. The glue gun is equipped with a glue gun nozzle, and the glue gun nozzle is equipped with a camera and a laser. Specifically, it may include the following modules: The laser line image acquisition module 501 is used to control the laser to project a laser line onto the surface of the workpiece during the glue gun nozzle spraying glue onto the workpiece, and to acquire the laser line image captured by the camera. The three-dimensional point cloud information determination module 502 is used to determine the three-dimensional point cloud information of the center line of the laser line based on the laser line image; The geometric feature parameter acquisition module 503 is used to determine the local geometric feature parameters of the workpiece surface based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle. The height control module 504 is used to determine the current gap height between the glue gun nozzle and the workpiece surface based on the local geometric feature parameters and the current nozzle coordinate information, and to control the height of the glue gun nozzle based on the current gap height.

[0118] In some embodiments of the present invention, the geometric feature parameter acquisition module 503 includes: The projection point coordinate information determination submodule is used to determine the projection point coordinate information of the glue gun nozzle projected onto the workpiece surface based on the current nozzle coordinate information; The neighborhood determination submodule is used to determine the neighborhood of the workpiece surface based on the projection point coordinate information; The local geometric feature parameter determination submodule is used to perform surface modeling on the neighborhood based on the three-dimensional point cloud information to obtain the local geometric feature parameters of the neighborhood.

[0119] In some embodiments of the present invention, the three-dimensional point cloud information includes the three-dimensional coordinate information of multiple laser points on the workpiece surface, and the local geometric feature parameter determination submodule includes: A three-dimensional coordinate information determination unit is used to determine the target laser point corresponding to the neighborhood and the three-dimensional coordinate information of the target laser point from the three-dimensional point cloud information; A weight determination unit is used to determine the weight information of the target laser point; The local geometric feature parameter determination unit is used to perform surface modeling on the neighborhood based on the weight information and the three-dimensional coordinate information of the target laser point to obtain the local geometric feature parameters of the neighborhood.

[0120] In some embodiments of the present invention, the height control module 504 includes: The elevation information determination submodule is used to determine the elevation information corresponding to the projection point based on the local geometric feature parameters. The surface unit normal vector information determination submodule is used to determine the gradient information corresponding to the projection point based on the local geometric feature parameters, and to obtain the surface unit normal vector information corresponding to the projection point based on the gradient information. The current gap height determination submodule is used to obtain the current gap height between the glue gun nozzle and the workpiece surface based on the elevation information, the surface unit normal vector information, and the current nozzle coordinate information.

[0121] In some embodiments of the present invention, the three-dimensional point cloud information determination module 502 includes: The pixel coordinate determination submodule is used to determine the pixel coordinates of the center line of the laser line on the workpiece surface based on the laser line image. The three-dimensional point cloud information determination submodule is used to convert the pixel coordinates into three-dimensional coordinates in the laser coordinate system based on the camera calibration data, so as to obtain the three-dimensional point cloud information of the laser line centerline.

[0122] In some embodiments of the present invention, the height control module 504 further includes: The target gap height acquisition submodule is used to acquire the target gap height. The height compensation amount determination submodule is used to determine the height compensation amount based on the current gap height and the target gap height; The height control submodule is used to control the height of the glue gun nozzle according to the height compensation amount.

[0123] In some embodiments of the present invention, the height compensation amount determination submodule includes: A height error determination unit is used to determine the height error between the current gap height and the target gap height; The height compensation amount determination unit is used to determine the height compensation amount based on the height error.

[0124] In some embodiments of the present invention, the apparatus further includes: The threshold protection module is used to acquire multi-level thresholds, compare the current gap height with the multi-level thresholds, obtain the comparison result, and provide anti-collision protection for the glue gun nozzle based on the comparison result.

[0125] The remaining time protection module is used to determine the remaining time before the glue gun nozzle collides with the workpiece surface, and to provide anti-collision protection for the glue gun nozzle based on the remaining time.

[0126] Some embodiments of the present invention also provide a glue-applying robot, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.

[0127] Some embodiments of the present invention also provide a computer-readable storage medium on which a computer program is stored, and which, when executed by a processor, implements the method described above.

[0128] Some embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0129] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0131] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0132] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0134] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0136] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the present invention.

[0137] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.

[0138] The above provides a detailed description of the method, device, and robot for controlling a glue gun. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method of controlling a glue gun, characterized by, The glue gun is equipped with a glue gun nozzle, and the glue gun nozzle is equipped with a camera and a laser. The method includes: During the process of applying glue to the workpiece using the glue gun nozzle, the laser is controlled to project a laser line onto the surface of the workpiece, and the laser line image is captured by the camera. Based on the laser line image, determine the three-dimensional point cloud information of the laser line centerline; Based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle, the local geometric feature parameters of the workpiece surface are determined; Based on the local geometric feature parameters and the current nozzle coordinate information, the current gap height between the glue gun nozzle and the workpiece surface is determined, and the height of the glue gun nozzle is controlled according to the current gap height.

2. The method of claim 1, wherein, Based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle, the local geometric feature parameters of the workpiece surface are determined, including: Based on the current nozzle coordinate information, determine the coordinate information of the projection point of the glue gun nozzle onto the workpiece surface; Based on the coordinate information of the projection points, the neighborhood of the workpiece surface is determined; Based on the 3D point cloud information, a surface model is performed on the neighborhood to obtain the local geometric feature parameters of the neighborhood.

3. The method of claim 2, wherein, The three-dimensional point cloud information includes the three-dimensional coordinate information of multiple laser points on the workpiece surface. Based on the three-dimensional point cloud information, surface modeling is performed on the neighborhood to obtain the local geometric feature parameters of the neighborhood, including: From the three-dimensional point cloud information, determine the target laser point corresponding to the neighborhood and the three-dimensional coordinate information of the target laser point; Determine the weight information of the target laser point; Based on the weight information and the three-dimensional coordinate information of the target laser point, a surface model is performed on the neighborhood to obtain the local geometric feature parameters of the neighborhood.

4. The method according to any one of claims 1 to 3, characterized in that, Based on the local geometric feature parameters and the current nozzle coordinate information, the current gap height between the glue gun nozzle and the workpiece surface is determined, including: Based on the local geometric feature parameters, determine the elevation information corresponding to the projection point; Based on the local geometric feature parameters, determine the gradient information corresponding to the projection point, and based on the gradient information, obtain the surface unit normal vector information corresponding to the projection point; Based on the elevation information, the surface unit normal vector information, and the current nozzle coordinate information, the current gap height between the glue gun nozzle and the workpiece surface is obtained.

5. The method according to any one of claims 1 to 3, characterized in that, Based on the laser line image, determine the three-dimensional point cloud information of the laser line centerline, including: Based on the laser line image, determine the pixel coordinates of the center line of the laser line on the workpiece surface; Based on the camera's calibration data, the pixel coordinates are converted into three-dimensional coordinates in the laser coordinate system to obtain the three-dimensional point cloud information of the laser line centerline.

6. The method according to any one of claims 1 to 3, characterized in that, Based on the current gap height, the height of the glue gun nozzle is controlled, including: Obtain the target gap height; The height compensation amount is determined based on the current gap height and the target gap height; The height of the glue gun nozzle is controlled according to the height compensation amount.

7. The method of claim 6, wherein, Based on the current clearance height and the target clearance height, the height compensation amount is determined, including: Determine the height error between the current gap height and the target gap height; Based on the height error, determine the height compensation amount.

8. The method according to any one of claims 1 to 3, characterized in that, In the process of controlling the height of the glue gun nozzle, the method further includes: A multi-level threshold is obtained, the current gap height is compared with the multi-level threshold, a comparison result is obtained, and anti-collision protection is provided for the glue gun nozzle based on the comparison result; And / or, determine the remaining time before the glue gun nozzle collides with the workpiece surface, and provide anti-collision protection for the glue gun nozzle based on the remaining time.

9. A glue gun control device characterized by, The glue gun is equipped with a glue gun nozzle, and the glue gun nozzle is equipped with a camera and a laser. The device includes: The laser line image acquisition module is used to control the laser to project a laser line onto the surface of the workpiece during the glue gun nozzle spraying glue onto the workpiece, and to acquire the laser line image captured by the camera. A three-dimensional point cloud information determination module is used to determine the three-dimensional point cloud information of the center line of the laser line based on the laser line image; The geometric feature parameter acquisition module is used to determine the local geometric feature parameters of the workpiece surface based on the three-dimensional point cloud information and the current nozzle coordinate information of the glue gun nozzle. The height control module is used to determine the current gap height between the glue gun nozzle and the workpiece surface based on the local geometric feature parameters and the current nozzle coordinate information, and to control the height of the glue gun nozzle based on the current gap height.

10. A gluing robot, characterized in that It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 8.

Images