Control system and method of full-automatic dispensing machine

Abstract

The invention relates to the technical field of dispensing machine control, and discloses a control system and method of a full-automatic dispensing machine. After a dispensing task is started, an industrial camera collects an image containing a specific reference feature area of a workpiece in real time; and the industrial personal computer detects and positions key reference feature points from the image to obtain pixel coordinates of the key reference feature points. And converting the feature points into actual positions under world coordinates based on a coordinate system conversion relation between a pre-calibrated camera and a machine motion system. The actual position is accurately compared with a preset ideal position, and comprehensive deviation transformation parameters of the workpiece are calculated. And finally, according to the deviation parameter, a preset dispensing path is dynamically adjusted, a dispensing driving instruction is generated, and a dispensing head is guided to accurately reach a target position, so that the influence of factors such as workpiece incoming material difference and positioning errors is effectively overcome, and the precision, stability and automation level of dispensing operation are remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of dispensing machine control, and more specifically, to a control system and method for a full-automatic dispensing machine. Background Art

[0002] With the rapid development of modern manufacturing towards high precision, high efficiency and high automation, the automatic dispensing technology has become an indispensable key process in many fields such as electronic packaging, automotive manufacturing, medical devices, and new energy. The dispensing quality, including the accuracy of the dispensing position, the uniformity of the glue volume, and the consistency of the glue line shape, directly affects the performance, reliability and service life of the product. Traditional dispensing operations, whether manual or relying on purely mechanical positioning automation equipment, all face significant challenges. Manual dispensing is inefficient, difficult to ensure consistency, and highly dependent on the skills and experience of operators; while early automation dispensing equipment, although improving production efficiency, its control methods often rely on preset fixed paths and assumptions about the ideal positions of workpieces, lacking effective adaptation and correction capabilities for factors such as incoming material differences of workpieces, positioning errors, fixture deviations, and possible minor deformations of workpieces themselves during the processing in actual production.

[0003] Specifically, in the prior art, when many full-automatic dispensing machines execute dispensing tasks, their motion trajectories are pre-programmed or set by teaching methods. The premise of this method is that each workpiece to be processed can be accurately and consistently placed at a predetermined position. However, in the actual production environment, the shape tolerance of the workpiece, the slight variation of the surface features, and the inevitable positioning errors during the transfer and clamping processes will all cause the actual position and posture of the workpiece to deviate from the ideal state. If the dispensing system cannot sense and compensate for these deviations and the dispensing head still moves according to the fixed program path, it is very easy to cause the dispensing position to deviate from the target area, resulting in quality defects such as glue overflow, dispensing omission or glue line deviation, and even leading to product scrapping in severe cases, increasing production costs and reducing the qualified rate. Summary of the Invention

[0004] In order to solve the problem that the existing automatic dispensing methods are insufficient in adaptability when dealing with the changes in the actual position and posture of workpieces, resulting in difficult-to-guarantee dispensing accuracy, the present application is proposed. Embodiments of the present application propose a control system and method for a full-automatic dispensing machine.

[0005] According to one aspect of the present application, a control method for a full-automatic dispensing machine is provided, including: after detecting a dispensing task start instruction, sending a camera trigger signal; an industrial camera collects original image data including a workpiece reference feature area according to the camera trigger signal, and transmits the original image data including the workpiece reference feature area to an industrial control computer through transmission; the industrial control computer detects and locates key reference feature points in the original image data including the workpiece reference feature area to obtain a set of positions of the key reference feature points in pixel coordinates; based on the coordinate calibration parameters of the camera and the machine motion system, the key reference feature points are converted from the pixel coordinate system to the world coordinate system to obtain a set of positions of the key reference feature points in the world coordinate system; the set of positions of the key reference feature points in the world coordinate system is compared with the ideal set of positions of the key reference feature points in the world coordinate system to obtain a deviation transformation parameter; based on the deviation transformation parameter, a dispensing drive instruction is generated.

[0006] In a possible implementation manner, the original image data including the workpiece reference feature area is transmitted to the industrial control computer through a high-speed interface.

[0007] In a possible implementation manner, the industrial control computer detects and locates key reference feature points in the original image data including the workpiece reference feature area to obtain a set of positions of the key reference feature points in pixel coordinates, including: performing distortion correction, image noise reduction and normalization processing on the original image data including the workpiece reference feature area to obtain a preprocessed image; inputting the preprocessed image into a trained deep learning model to obtain a set of positions of the key reference feature points in pixel coordinates.

[0008] In a possible implementation manner, the trained deep learning model is a key point detection model based on a convolutional neural network.

[0009] In a possible implementation manner, inputting the preprocessed image into a trained deep learning model to obtain a set of positions of the key reference feature points in pixel coordinates, including: using a convolutional neural network model to extract image features of the preprocessed image to obtain an image feature matrix; performing convergence optimization based on tensor feature correlation gradient and dynamic tensor field on the image feature matrix to obtain an optimized image feature matrix; inputting the optimized image feature matrix into a softmax function to determine the probability value of each pixel as a key point; based on the probability value of each pixel as a key point, determining a set of positions of the key reference feature points in pixel coordinates.

[0010] In a possible implementation, performing convergence optimization on the image feature matrix based on tensor feature correlation gradient and dynamic tensor field to obtain an optimized image feature matrix includes: performing deconvolution operation on the image feature matrix to obtain a prior feature matrix in the gradient direction; deriving the differential field quantity distribution based on gradient response based on the image feature matrix and the prior feature matrix to obtain a gradient inverse response matrix; calculating the hyperspace superposition field of the gradient based on the image feature matrix and the prior feature matrix to obtain a hyperspace anchor matrix; using the hyperspace anchor matrix as a hyperspace anchor to normalize and constrain the center convergence of the field strength to obtain a gauge field strength matrix; fusing the gradient inverse response matrix and the gauge field strength matrix with hyperparameters as weighting coefficients to obtain the optimized image feature matrix.

[0011] In a possible implementation, comparing the set of positions of the key reference feature points in the world coordinate system with the set of ideal positions of the key reference feature points in the world coordinate system to obtain deviation transformation parameters includes: calculating the optimal 2D rigid body transformation matrix between the set of positions of the key reference feature points in the world coordinate system and the set of ideal positions of the key reference feature points in the world coordinate system, where the optimal 2D rigid body transformation matrix includes a translation parameter in the X direction, a translation parameter in the Y direction, and a rotation parameter around the Z axis; extracting the translation parameter in the X direction, the translation parameter in the Y direction, and the rotation parameter around the Z axis from the optimal 2D rigid body transformation matrix as the deviation transformation parameters.

[0012] In a possible implementation, generating a dispensing drive instruction based on the deviation transformation parameters includes: obtaining preset dispensing path data; inputting the deviation transformation parameters and the preset dispensing path data into a motion controller, and the motion controller fine-tuning the preset dispensing path data based on the deviation transformation parameters to obtain adjusted dispensing path data; generating the dispensing drive instruction based on the adjusted dispensing path data.

[0013] According to another aspect of the present application, a control system for a full-automatic dispensing machine is provided for implementing the control method of the full-automatic dispensing machine as described above, including: a dispensing start trigger control module for issuing a camera trigger signal after detecting a dispensing task start instruction; an image acquisition and transmission module for an industrial camera to acquire original image data including a workpiece reference feature area according to the camera trigger signal and transmit the original image data including the workpiece reference feature area to an industrial control computer; an image feature point detection and positioning module for the industrial control computer to detect and locate key reference feature points in the original image data including the workpiece reference feature area to obtain a set of positions of the key reference feature points in pixel coordinates; a coordinate system conversion processing module for converting the key reference feature points from a pixel coordinate system to a world coordinate system based on the coordinate system calibration parameters of the camera and the machine motion system to obtain a set of positions of the key reference feature points in the world coordinate system; a feature point position comparison module for comparing the set of positions of the key reference feature points in the world coordinate system with the ideal set of positions of the key reference feature points in the world coordinate system to obtain a deviation transformation parameter; and a dispensing drive instruction generation module for generating a dispensing drive instruction based on the deviation transformation parameter.

[0014] Compared with the prior art, for the control system and method of the full-automatic dispensing machine provided by the present application, after the dispensing task is started, the industrial camera collects images including the workpiece specific reference feature area in real time; the industrial control computer then detects and locates the key reference feature points from the images and obtains their pixel coordinates. Based on the pre-calibrated coordinate system conversion relationship between the camera and the machine motion system, the feature points are converted into the actual positions in the world coordinate system. By accurately comparing this actual position with the preset ideal position, the comprehensive deviation transformation parameter of the workpiece is calculated. Finally, based on this deviation parameter, the preset dispensing path is dynamically adjusted to generate a dispensing drive instruction to guide the dispensing head to accurately reach the target position, thereby effectively overcoming the influence of factors such as workpiece incoming material differences and positioning errors, and significantly improving the accuracy, stability and automation level of the dispensing operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other objects, features and advantages of the present application will become more obvious. The accompanying drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation to the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 FIG. shows a schematic flow chart of the control method of the full-automatic dispensing machine according to an embodiment of the present application.

[0017] Figure 2Illustrates a schematic flowchart of step S3 in the control method of a fully automatic dispensing machine according to an embodiment of the present application.

[0018] Figure 3 Illustrates a schematic flowchart of step S5 in the control method of a fully automatic dispensing machine according to an embodiment of the present application.

[0019] Figure 4 Illustrates a schematic flowchart of step S6 in the control method of a fully automatic dispensing machine according to an embodiment of the present application.

[0020] Figure 5 Illustrates a schematic block diagram of the control system of a fully automatic dispensing machine according to an embodiment of the present application. Detailed implementation manners

[0021] Next, exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein.

[0022] Figure 1 Illustrates a schematic flowchart of the control method of a fully automatic dispensing machine according to an embodiment of the present application. As Figure 1 shown, the present application provides a control method for a fully automatic dispensing machine, including: S1: After detecting a start instruction for a dispensing task, sending a camera trigger signal; S2: An industrial camera collects original image data including a workpiece reference feature area according to the camera trigger signal, and transmits the original image data including the workpiece reference feature area to an industrial control computer; S3: The industrial control computer detects and locates key reference feature points in the original image data including the workpiece reference feature area to obtain a set of positions of the key reference feature points in pixel coordinates; S4: Based on the coordinate calibration parameters of the camera and the machine motion system, converting the key reference feature points from the pixel coordinate system to the world coordinate system to obtain a set of positions of the key reference feature points in the world coordinate system; S5: Comparing the set of positions of the key reference feature points in the world coordinate system with the ideal set of positions of the key reference feature points in the world coordinate system to obtain deviation transformation parameters; S6: Generating a dispensing drive instruction based on the deviation transformation parameters.

[0023] Specifically, in step S1, after detecting the start instruction of the dispensing task, a camera trigger signal is issued. It should be understood that in the automated production process, the start instruction of the dispensing task indicates that a specific workpiece is ready and about to enter the dispensing process. Triggering the camera at this time can ensure that the image captured by the camera is of the latest state of the current workpiece to be processed, including its actual position and attitude information on the workbench. If the camera is triggered too early, the workpiece may not be in place or stable, and the captured image will have no reference value; if it is triggered too late, the entire dispensing cycle may be delayed, affecting production efficiency, and even the dispensing action may start incorrectly before the camera has completed imaging. Therefore, the camera trigger is tightly coupled with the task start instruction, that is, after detecting the start instruction of the dispensing task, a camera trigger signal is issued.

[0024] In one embodiment, the start instruction of the dispensing task can be issued by the upper control system (such as a PLC or MES system) after confirming that the workpiece has passed the previous process and is accurately conveyed to the dispensing station. In other embodiments of the present application, the start instruction of the dispensing task can also be generated manually by an operator pressing a start button through a human-machine interface; or automatically triggered by a sensor (such as a photoelectric sensor or a limit switch) after detecting the workpiece in place. When the start instruction of the dispensing task is detected, the industrial control computer software will immediately execute a preset action, that is, issue a camera trigger signal. The triggering methods of the camera are mainly divided into hardware triggering and software triggering. In the hardware triggering mode, the core controller will send a level signal, such as a rising edge, a falling edge, or a pulse signal with a specific pulse width, to a dedicated trigger input pin of the industrial camera through its digital output port. This physical connection ensures the low latency and high reliability of the trigger signal. In the software triggering mode, the industrial control computer and the industrial camera are connected through a data interface. When the control software on the industrial control computer detects the start instruction of the dispensing task, it will send a specific software command to the camera through the software development kit or application programming interface provided by the camera, instructing the camera to perform image acquisition.

[0025] Specifically, in step S2, the industrial camera acquires the original image data including the reference feature area of the workpiece according to the camera trigger signal, and transmits the original image data including the reference feature area of the workpiece to the industrial control computer. It should be understood that the trigger instruction is issued when the dispensing task starts, that is, when the workpiece is basically in place. The camera only performs image acquisition after receiving this specific trigger, which means that the acquired image data can truly reflect the immediate state of the current workpiece to be processed on the workbench. This event-driven acquisition method avoids the massive amount of invalid data and unnecessary consumption of computing resources caused by continuous acquisition, ensures that each frame of the processed image is targeted, and thus maximizes the effectiveness and real-time performance of visual guidance.

[0026] Specifically, when the industrial camera receives the camera trigger signal from the industrial control computer, the internal image acquisition process is activated. The industrial camera immediately performs an acquisition of an image frame according to its preset exposure parameters and other settings, so as to acquire the original image data containing the workpiece reference feature area. Here, it should be understood that although the actual workpiece may have a complex structure, usually only a few key, easily recognizable and position-stable reference feature points on it, such as specific holes, edges, corner points or preset marking points, are needed to accurately calculate the position and orientation of the entire workpiece. By configuring the field of view and resolution of the camera to be exactly able to clearly resolve these specific reference feature areas, the complexity of subsequent image processing algorithms can be significantly simplified, the processing time can be shortened, and the accuracy of feature extraction and anti-interference ability can be improved. Then, the original image data containing the workpiece reference feature area is transmitted to the industrial control computer through a high-speed interface.

[0027] Specifically, in step S3, the industrial control computer detects and locates the key reference feature points in the original image data containing the workpiece reference feature area to obtain the position set of the key reference feature points in pixel coordinates. It should be understood that in the actual production environment, the incoming position and orientation of the workpiece may deviate due to small errors in links such as conveying and clamping, and the workpiece itself may also have certain deformations. If the dispensing machine completely relies on the preset fixed path for operation, these deviations will cause inaccurate dispensing positions, seriously affecting the product quality and even resulting in product scrapping. Therefore, it is necessary to have the ability to sense the actual position and orientation of the workpiece and adjust the dispensing path accordingly. Based on this, the industrial control computer detects and locates the key reference feature points in the original image data containing the workpiece reference feature area. By accurately identifying and locating the representative feature points on the workpiece, the deviation of the workpiece relative to the ideal position can be calculated.

[0028] In one embodiment, as Figure 2 shown, the industrial control computer detects and locates the key reference feature points in the original image data containing the workpiece reference feature area to obtain the position set of the key reference feature points in pixel coordinates, including: S31: performing distortion correction, image denoising and normalization processing on the original image data containing the workpiece reference feature area to obtain a preprocessed image; S32: inputting the preprocessed image into the trained deep learning model to obtain the position set of the key reference feature points in pixel coordinates. The trained deep learning model is a key point detection model based on a convolutional neural network.

[0029] First, a series of preprocessing operations are performed on the original image data containing the workpiece reference feature region to obtain a preprocessed image, thereby eliminating or reducing the noise and distortion introduced during the image acquisition process and converting the image data into a format more suitable for processing by a deep learning model. Specifically, the preprocessing operations include distortion correction, image denoising, and normalization. Among them, distortion correction takes into account that industrial camera lenses, especially wide-angle lenses or low-cost lenses, often introduce geometric distortions, mainly including radial distortion and tangential distortion. Radial distortion makes the straight lines in the image bend, and the bending is more obvious the farther away from the center of the image; tangential distortion is caused by the lens itself not being completely parallel to the image sensor plane. These distortions will cause a deviation between the true position of the object points in the image and their imaging positions in the image. If not corrected, it will directly affect the accuracy of key point localization. In a specific embodiment, the internal and external parameters and distortion coefficients obtained based on camera calibration can be used. For example, the Zhang-Zhengyou calibration method can be adopted. By taking images of the checkerboard calibration board in different poses, the internal parameter matrix K and the distortion coefficient vector D of the camera can be calculated. After obtaining these parameters, for any point in the original image, it can be mapped back to the undistorted ideal position through a mathematical model. By applying this inverse distortion model to each pixel of the original image, an image after distortion correction can be generated.

[0030] Next, considering that during the image acquisition and transmission process, the image will inevitably be interfered by various noise sources, such as sensor thermal noise, photon shot noise, readout circuit noise, etc. These noises will reduce the signal-to-noise ratio of the image, blur the image details, and interfere with the accurate recognition of subsequent feature points. Therefore, image denoising is performed on the image after distortion correction. In a specific embodiment, Gaussian filtering can be used. The Gaussian filter is a linear smoothing filter that uses a two-dimensional Gaussian function as the convolution kernel. The weight distribution of the Gaussian kernel is bell-shaped, with the largest weight at the center point and the smaller weights for the pixels farther away from the center. Its function is to perform weighted averaging on the pixel values in the neighborhood, thereby smoothing the image and suppressing high-frequency noise.

[0031] Then, normalization is performed on the image after image denoising. Normalization is to adjust the pixel values of the image to a predefined range or distribution, which is crucial for the stable training and efficient convergence of the deep learning model. Different images may have significant differences in overall brightness and contrast due to factors such as lighting conditions and exposure settings. Normalization can eliminate the influence brought by these differences. In a specific embodiment, min-max normalization can be used to linearly scale the pixel values to the interval [0, 1] or [-1, 1].

[0032] After the above-mentioned distortion correction, image denoising, and normalization processing, the obtained preprocessed image will be used as input and fed into a trained key-point detection model based on a convolutional neural network. Convolutional neural networks are very suitable for object localization tasks in images due to their powerful feature extraction capabilities and effective modeling of spatial hierarchical information. In a specific embodiment, the key-point detection model based on a convolutional neural network includes: an input layer, a convolutional neural network model, and a Softmax function. Specifically, the input layer is used to receive the preprocessed image, and the convolutional neural network model includes multiple convolutional layers, multiple pooling layers, and fully connected layers. In a specific embodiment, the convolutional neural network model includes: The first convolutional layer: uses 32 convolutional kernels of size 3×3, with a stride of 1, padding of 1, and the activation function is ReLU. The first pooling layer: max pooling, with a pooling window size of 2×2 and a stride of 2. The second convolutional layer: 64 convolutional kernels of size 3×3, with a stride of 1, padding of 1, and the activation function is ReLU. The second pooling layer: max pooling, with a pooling window size of 2×2 and a stride of 2. The third convolutional layer: 128 convolutional kernels of size 3×3, with a stride of 1, padding of 1, and the activation function is ReLU. The third pooling layer: max pooling, with a pooling window size of 2×2 and a stride of 2. After the convolution and pooling operations, the three-dimensional feature map is flattened into a one-dimensional vector, and then the dimension is adjusted through the fully connected layer, and the final output is an image feature matrix. The image feature matrix is input into the Softmax function to generate the probability value of each pixel as a key point, forming a probability heat map, where the value of each pixel represents the probability value of that pixel as a key point. In a specific embodiment, on this probability heat map, a threshold is applied to filter out those points that are considered noise or unimportant. For example, a threshold of 0.7 is set (of course, this is only an example here and can be adjusted according to the actual situation), which means that only when the probability value of a certain pixel position is greater than or equal to 0.7, this pixel is considered a potential key point. Next, traverse the entire probability heat map to find all pixel positions that meet the above conditions. For each such pixel position, further check its surrounding area to ensure that it is indeed a local maximum. This is achieved by comparing the probability value of this pixel with that of its adjacent pixels (for example, 8-neighborhood, that is, neighbors in the up, down, left, right, and four diagonal directions). If the probability value of the current pixel is higher than that of all its adjacent pixels, it is confirmed as a key reference feature point. Record the coordinates (x, y) of these key points, and the position set of the key reference feature points in pixel coordinates is obtained.

[0033] Specifically, since the key point detection model based on the convolutional neural network extracts the image features of the preprocessed image through convolutional kernels to determine the probability of pixel features as key points for key point detection, considering the width and height dimensions of the images processed by the convolutional neural network and the channel dimension of the convolutional neural network itself, in essence, the convolutional kernels of the convolutional neural network perform dynamic modeling of the dynamic tensor field of tensor features. Moreover, since the subsequent key reference feature points also need to be converted from the pixel coordinate system to the world coordinate system and calculate the coordinate deviation transformation parameters, therefore, the convolutional neural network needs to have strong constraints on the associated gradient direction and the field strength convergence center of the dynamic tensor field in the non-linear tensor space, so as to enhance the expression effect of the key point space linear-nonlinear characteristics in the image semantic feature scenario.

[0034] Based on this, in another embodiment of the present application, inputting the preprocessed image into the trained deep learning model to obtain the position set of the key reference feature points in pixel coordinates, including: using a convolutional neural network model to extract the image features of the preprocessed image to obtain an image feature matrix; performing convergence optimization based on tensor feature association gradient and dynamic tensor field on the image feature matrix to obtain an optimized image feature matrix; inputting the optimized image feature matrix into the softmax function to determine the probability value of each pixel as a key point; and determining the position set of the key reference feature points in pixel coordinates based on the probability value of each pixel as a key point.

[0035] Specifically, performing convergence optimization based on tensor feature association gradient and dynamic tensor field on the image feature matrix to obtain an optimized image feature matrix, including: performing deconvolution operation on the image feature matrix output by the convolutional neural network model to obtain a prior feature matrix in the gradient direction, that is .

[0036] Then, based on the image feature matrix and the prior feature matrix, perform derivation of the differential field quantity distribution based on gradient response to obtain a gradient inverse response matrix, that is: ; where represents the image feature matrix, represents the prior feature matrix, represents element-wise subtraction, represents the inverse matrix of the image feature matrix , represents matrix multiplication, represents the gradient inverse response matrix.

[0037] That is, approximate the differential field in the non-linear tensor space through the gradient inverse response of the image feature matrix.

[0038] Then, further based on the image feature matrix and the prior feature matrix, calculate the hyperspace superposition field of the gradient to obtain the hyperspace anchor matrix, that is; ; where represents the transpose symbol, represents the hyperspace anchor matrix.

[0039] Next, use the hyperspace anchor matrix as the hyperspace anchor to normalize and constrain the convergence of the field strength center to obtain the gauge field strength matrix, that is ; where represents the gauge field strength matrix.

[0040] Then, perform the superposition of the gradient field and the gauge field by using the hyperparameter as the weighting coefficient, that is, fuse the gradient inverse response matrix and the gauge field strength matrix by using the hyperparameter as the weighting coefficient to obtain the optimized image feature matrix, that is , where and represent learnable hyperparameters, represents element-wise multiplication, represents element-wise addition, represents the optimized image feature matrix. By superimposing the geometric transformation representation, the derivation accuracy of the spatial linear-nonlinear characteristics based on the tensor feature correlation gradient and the dynamic composite manifold field strength convergence mapping of the nonlinear microdifferential manifold can be improved. That is, the transfer learning effect of the coordinate system space field quantity of the convolutional neural network is enhanced, thereby improving the detection accuracy of the key reference feature points in the coordinate system transformation and coordinate deviation transformation scenarios.

[0041] Specifically, in step S4, based on the coordinate system calibration parameters of the camera and the machine motion system, convert the key reference feature points from the pixel coordinate system to the world coordinate system to obtain the position set of the key reference feature points in the world coordinate system. It should be understood that the image captured by the industrial camera is essentially a two-dimensional pixel array, where the positions of the key reference feature points are described in pixels. This pixel coordinate is relative to the camera image sensor and does not directly carry any absolute information about the size or position of the workpiece in the real physical space. Whether a feature that occupies 10 pixels in the image corresponds to 1 millimeter or 1 centimeter in the physical world depends entirely on the optical settings of the camera, the distance from the workpiece, and the lens parameters, etc. On the other hand, the machine motion system operates in a well-defined physical space, that is, the world coordinate system (usually a three-dimensional or two-dimensional Cartesian coordinate system fixed to the workbench). The dispensing drive instructions, such as moving the dispensing head to a certain position, must be specified in physical units in the world coordinate system.

[0042] Therefore, if we only stay at the pixel coordinate level, it will be impossible to understand the actual position of the workpiece in the physical world, let alone calculate the true deviation between its position and the ideal physical position. Without this deviation information expressed in physical units, it is impossible to generate correct and effective compensation instructions to adjust the movement path of the dispensing head. Based on this, in the present application, further based on the coordinate calibration parameters of the camera and the machine motion system, the key reference feature points are transformed from the pixel coordinate system to the world coordinate system to obtain the position set of the key reference feature points in the world coordinate system. Those of ordinary skill in the art should know that the calibration between the coordinate systems of the camera and the machine motion system is a conventional process. Among them, the purpose of the calibration between the camera and the machine motion system is to determine the coordinate calibration parameters of the camera and the machine motion system.

[0043] In one embodiment, the key reference feature points can be transformed from the pixel coordinate system to the world coordinate system through planar homography transformation. Specifically, there is a mapping relationship of a direct 3x3 homography matrix H between the pixel coordinates (u, v) and the world coordinates (Xw, Yw) on this plane: ; where s is a scale factor, represents transpose, represents the homography matrix, and the homography matrix can be obtained by collecting several points with known world coordinates on the calibration board and their corresponding image pixel coordinates , and then solved by an optimization algorithm such as the least squares method. Of course, this is only an example here, and other geometric transformation methods can also be used to transform the key reference feature points from the pixel coordinate system to the world coordinate system.

[0044] Specifically, in step S5, the position set of the key reference feature points in the world coordinate system is compared with the ideal position set of the key reference feature points in the world coordinate system to obtain the deviation transformation parameters. It should be understood that by comparing the actual position set of the key reference feature points measured in the world coordinate system with the world coordinate position set where the key reference feature points should be when the workpiece is in an absolutely ideal state, the actual offset difference can be accurately perceived.

[0045] In one embodiment, as Figure 3As shown, comparing the set of positions of the key reference feature points in the world coordinate system with the set of ideal positions of the key reference feature points in the world coordinate system to obtain deviation transformation parameters includes: S51: calculating the optimal 2D rigid body transformation matrix between the set of positions of the key reference feature points in the world coordinate system and the set of ideal positions of the key reference feature points in the world coordinate system, where the optimal 2D rigid body transformation matrix includes the translation parameter in the X direction, the translation parameter in the Y direction, and the rotation parameter about the Z axis; S52: extracting the translation parameter in the X direction, the translation parameter in the Y direction, and the rotation parameter about the Z axis from the optimal 2D rigid body transformation matrix as the deviation transformation parameters.

[0046] Specifically, when the industrial control computer obtains the set of actual positions of the key reference feature points in the world coordinate system, it also requires a reference, that is, the set of ideal positions of these key reference feature points in the world coordinate system when the workpiece is in the ideal position. This set of ideal positions is predefined and stored in the system, usually determined during workpiece design or the first process programming. For example, it can be directly extracted from the CAD model, or during teaching programming, the workpiece is placed in the absolute standard position, and then the world coordinates of these reference points are recorded by manually guiding the robot or the vision system.

[0047] Next, the set of positions of the key reference feature points in the world coordinate system is compared with the set of ideal positions of the key reference feature points in the world coordinate system. The process of comparing these two sets of positions is, of course, not simply the point-by-point coordinate difference, because the deviation of the workpiece is usually a combination of translation and rotation, rather than independent random movement of each point. Therefore, in this application, by finding an optimal 2D rigid body transformation matrix, this transformation can transform the set of ideal positions as a whole to the state closest to the set of actual positions. The so-called 2D rigid body transformation means that the relative distances and angles between points inside the object remain unchanged during the transformation, only involving translation and rotation in the plane. This conforms to the actual situation of workpiece deviation in most planar dispensing applications. Once this transformation matrix is obtained, it precisely describes the overall geometric transformation that has occurred from the ideal state to the current actual state, and the translation parameters and rotation parameters it contains are the required deviation transformation parameters.

[0048] In one embodiment, a rigid body transformation solution method for two-dimensional point sets is used to calculate the optimal 2D rigid body transformation matrix between the set of positions of the key reference feature points in the world coordinate system and the set of ideal positions of the key reference feature points in the world coordinate system. Those of ordinary skill in the art should know that the rigid body transformation solution method for two-dimensional point sets belongs to a conventional process, so it will not be elaborated further.

[0049] Specifically, in step S6, a dispensing drive instruction is generated based on the deviation transformation parameter. It should be understood that a fully automatic dispensing machine performs tasks according to a series of pre-programmed or taught path points (including position and pose information, and dispensing action instructions). This preset path is designed based on the workpiece being in an ideal position. If this preset path is directly used while the actual position of the workpiece has a deviation, then the dispensing will deviate from the target. Therefore, based on the deviation transformation parameter, this preset standard dispensing path is dynamically adjusted in real time to generate a new actual dispensing path that can accurately match the actual position of the current workpiece. Based on this corrected actual dispensing path, a truly effective dispensing drive instruction is generated to guide the dispensing head to complete dispensing actions (such as dispensing, scribing, filling, etc.) at the correct position of the workpiece, and at the same time accurately control the start and stop of the dispensing valve.

[0050] In one embodiment, as Figure 4 shown, generating a dispensing drive instruction based on the deviation transformation parameter includes: S61: Obtain preset dispensing path data; S62: Input the deviation transformation parameter and the preset dispensing path data into a motion controller, and the motion controller performs path fine-tuning on the preset dispensing path data based on the deviation transformation parameter to obtain dispensing path adjustment data; S63: Generate the dispensing drive instruction based on the dispensing path adjustment data.

[0051] Specifically, first, obtain preset dispensing path data. The specific obtaining methods include offline programming, teach programming, and vision-assisted teaching. Among them, offline programming uses CAD / CAM software to directly plan the dispensing trajectory on the digital three-dimensional model of the workpiece. The software can generate a series of path points including world coordinates (X, Y, Z, and possibly poses A, B, C), and dispensing instructions (such as valve opening, valve closing, glue volume control, etc.) executed at these points or between these points. These data are then downloaded to the controller of the fully automatic dispensing machine. Teach programming means that the operator manually guides the robot dispensing head to move to each key dispensing position on the workpiece through a handheld teach pendant, and records the world coordinates of these points and the corresponding dispensing actions. This method is intuitive, but the accuracy and efficiency may be affected by human factors. Vision-assisted teaching means that in combination with a vision system, the operator can click on the target dispensing position on the camera image, and the system automatically converts the pixel coordinates into world coordinates and records them.

[0052] Then, the standard dispensing path data is transformed according to the deviation transformation parameters to obtain the actual dispensing path data. Specifically, the industrial control computer calculates the deviation transformation parameters through the previous vision processing process, and these parameters need to be applied to each path point in the standard dispensing path to generate a new, compensated actual dispensing path. This transformation process is based on the mathematical principle of rigid body transformation. For any ideal path point in the standard dispensing path, its corresponding point under the actual position of the current workpiece needs to be calculated. The specific transformation process is as follows. First, if the rotation is around the centroid of the ideal point set, then the ideal points need to be first translated to a coordinate system with the centroid as the origin, rotated, and then translated back, and finally the overall translation deviation is applied. But a more direct method is that if the deviation parameters (ΔX, ΔY, Δθ) are defined as the parameters for directly transforming the ideal coordinate system to the actual coordinate system (i.e., rotating by Δθ around the origin first, and then translating by ΔX, ΔY to obtain the actual coordinates). The industrial control computer needs to traverse all the path points in the standard dispensing path that define spatial positions, apply the above transformation formula to the coordinates of each point to obtain new coordinates. If the path also contains attitude information (for example, the orientation of the dispensing needle), and the deviation transformation also includes attitude deviations (for example, if the workpiece not only translates and rotates but also tilts), then the attitude information also needs to be transformed accordingly. For 2D planar dispensing, usually only XY translation and rotation around the Z-axis are considered, and the attitude of the dispensing head (perpendicular to the plane) remains unchanged. By performing such transformations on all relevant points on the standard path, a set of actual dispensing path data is generated.

[0053] Finally, based on the actual dispensing path data, dispensing drive instructions for controlling the movement of the dispensing head and the opening and closing of the dispensing valve are generated. That is, the industrial control computer converts the actual dispensing path data (including world coordinates, movement type, dispensing actions, etc.) into dispensing drive instructions that the controller can directly understand and execute. In a specific embodiment, based on the dispensing path adjustment data, the dispensing drive instructions are generated, including: First, path interpolation. That is, the controller needs to generate a series of dense intermediate target position points through a specific interpolation algorithm (such as linear interpolation, circular interpolation, spline interpolation) for the discrete path points on the actual dispensing path (such as the transformed starting point and ending point), ensuring that the dispensing head can move smoothly and continuously along the corrected trajectory. Then, speed and acceleration planning. To ensure the smoothness and efficiency of the movement, the controller will plan the corresponding speed and acceleration curves (such as trapezoidal or S-shaped curves) for the interpolated path points according to parameters such as the set maximum speed and acceleration. Next, I / O control instruction generation. That is, according to the dispensing action information (such as when to open the valve, when to close the valve, glue volume control parameters, etc.) included in the actual dispensing path data, corresponding digital output (DO) or analog output (AO) instructions are generated. For example, when the dispensing head moves to a certain point on the actual path, the controller will send a DO signal to the dispensing valve controller to open the valve and start dispensing glue; when it reaches another point, another signal is sent to close the valve. If glue volume control is required, it may also involve analog output to control the dispensing pressure or time. Finally, instruction serialization and sending. That is, all the calculated joint target values, speeds, accelerations, and I / O control signals are organized into an instruction sequence executable by the robot controller in chronological order. These instructions will ultimately drive the robot servo motor and the dispensing valve actuator to act.

[0054] In summary, for the control method of the full-automatic dispensing machine provided in this application, after the dispensing task is started, the industrial camera real-time collects images containing the specific reference feature area of the workpiece; the industrial control computer then detects and locates the key reference feature points from the images and obtains their pixel coordinates. Based on the pre-calibrated coordinate transformation relationship between the camera and the machine motion system, the feature points are converted into the actual positions in the world coordinates. By precisely comparing this actual position with the preset ideal position, the comprehensive deviation transformation parameters of the workpiece are calculated. Finally, based on this deviation parameter, the preset dispensing path is dynamically adjusted, and dispensing drive instructions are generated to guide the dispensing head to accurately reach the target position, thereby effectively overcoming the influence of factors such as workpiece incoming material differences and positioning errors, and significantly improving the accuracy, stability, and automation level of the dispensing operation.

[0055] This application also provides a control system for a full-automatic dispensing machine for implementing the above control method of the full-automatic dispensing machine, such as Figure 5As shown in the figure, the control system 500 of the fully automatic dispensing machine includes: a dispensing start trigger control module 501, which is used to issue a camera trigger signal after detecting a dispensing task start instruction; an image acquisition and transmission module 502, which is used for an industrial camera to acquire original image data including the workpiece reference feature area according to the camera trigger signal, and transmit the original image data including the workpiece reference feature area to an industrial control computer; an image feature point detection and positioning module 503, which is used for the industrial control computer to detect and locate key reference feature points in the original image data including the workpiece reference feature area to obtain a set of positions of the key reference feature points in pixel coordinates; a coordinate system conversion processing module 504, which is used to convert the key reference feature points from the pixel coordinate system to the world coordinate system based on the coordinate system calibration parameters of the camera and the machine motion system to obtain a set of positions of the key reference feature points in the world coordinate system; a feature point position comparison module 505, which is used to compare the set of positions of the key reference feature points in the world coordinate system with the ideal set of positions of the key reference feature points in the world coordinate system to obtain deviation transformation parameters; and a dispensing drive instruction generation module 506, which is used to generate a dispensing drive instruction based on the deviation transformation parameters.

[0056] The basic principles of the present application have been described above in combination with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present application are only examples and not limitations. It cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present application. In addition, the above-disclosed specific details are only for the purposes of illustration and facilitating understanding, and not for limitation. The above details do not limit the present application to necessarily adopt the above specific details for implementation.

[0057] The flowcharts of the methods involved in the present application are only illustrative examples and do not intend to require or imply that the connections, arrangements, and configurations must be made in the manner shown in the flowcharts. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any way. Words such as "including", "comprising", "having", etc. are open-ended words, meaning "including but not limited to", and can be used interchangeably with each other. The words "or" and "and" used herein refer to the word "and / or", and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with each other.

[0058] It should also be noted that in the methods of the present application, each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present application.

[0059] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Thus, the present application is not intended to be limited to the aspects shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0060] The above description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present application to the form disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some variations, modifications, alterations, additions, and subcombinations thereof.

Claims

1. A control method for a fully automatic dispensing machine, characterized in that, Including: After detecting the start instruction of the dispensing task, a camera trigger signal is issued; the industrial camera acquires the original image data including the workpiece reference feature area according to the camera trigger signal, and transmits the original image data including the workpiece reference feature area to the industrial control computer; the industrial control computer detects and locates the key reference feature points in the original image data including the workpiece reference feature area to obtain the position set of the key reference feature points in pixel coordinates; based on the coordinate calibration parameters of the camera and the machine motion system, the key reference feature points are converted from the pixel coordinate system to the world coordinate system to obtain the position set of the key reference feature points in the world coordinate system; The position set of the key reference feature points in the world coordinate system is compared with the ideal position set of the key reference feature points in the world coordinate system to obtain the deviation transformation parameter; based on the deviation transformation parameter, a dispensing drive instruction is generated.

2. The control method of the full-automatic dispensing machine according to claim 1, characterized in that, The original image data including the workpiece reference feature area is transmitted to the industrial control computer through a high-speed interface.

3. The control method of the full-automatic dispensing machine according to claim 1, characterized in that, The industrial control computer detects and locates the key reference feature points in the original image data including the workpiece reference feature area to obtain the position set of the key reference feature points in pixel coordinates, including: performing distortion correction, image denoising and normalization processing on the original image data including the workpiece reference feature area to obtain the preprocessed image; inputting the preprocessed image into the trained deep learning model to obtain the position set of the key reference feature points in pixel coordinates.

4. The control method of the fully automatic dispensing machine according to claim 3, characterized in that, The trained deep learning model is a key point detection model based on a convolutional neural network.

5. The control method of the full-automatic dispensing machine according to claim 4, characterized in that, Inputting the preprocessed image into the trained deep learning model to obtain the position set of the key reference feature points in pixel coordinates, including: using a convolutional neural network model to extract the image features of the preprocessed image to obtain an image feature matrix; performing convergence optimization based on tensor feature correlation gradient and dynamic tensor field on the image feature matrix to obtain an optimized image feature matrix; inputting the optimized image feature matrix into the softmax function to determine the probability value of each pixel as a key point; based on the probability value of each pixel as a key point, determining the position set of the key reference feature points in pixel coordinates.

6. The control method of the full-automatic dispensing machine according to claim 5, characterized in that, Performing convergence optimization based on tensor feature correlation gradient and dynamic tensor field on the image feature matrix to obtain an optimized image feature matrix, including: performing deconvolution operation on the image feature matrix to obtain a prior feature matrix in the gradient direction; performing derivation of the differential field quantity distribution based on gradient response based on the image feature matrix and the prior feature matrix to obtain a gradient inverse response matrix; calculating the hyperspace superposition field of the gradient based on the image feature matrix and the prior feature matrix to obtain a hyperspace anchor matrix; using the hyperspace anchor matrix as a hyperspace anchor to normalize the convergence of the field strength center to obtain a gauge field strength matrix; using hyperparameters as weighting coefficients to fuse the gradient inverse response matrix and the gauge field strength matrix to obtain the optimized image feature matrix.

7. The control method of the full-automatic dispensing machine according to claim 1, wherein Compare the set of positions of the key reference feature points in the world coordinate system with the set of ideal positions of the key reference feature points in the world coordinate system to obtain deviation transformation parameters, including: calculating the optimal 2D rigid body transformation matrix between the set of positions of the key reference feature points in the world coordinate system and the set of ideal positions of the key reference feature points in the world coordinate system, where the optimal 2D rigid body transformation matrix includes a translation parameter in the X direction, a translation parameter in the Y direction, and a rotation parameter around the Z axis; extracting the translation parameter in the X direction, the translation parameter in the Y direction, and the rotation parameter around the Z axis from the optimal 2D rigid body transformation matrix as the deviation transformation parameters.

8. The control method of the full-automatic dispensing machine according to claim 7, characterized in that Generate a dispensing drive instruction based on the deviation transformation parameters, including: obtaining preset dispensing path data; inputting the deviation transformation parameters and the preset dispensing path data into a motion controller, and the motion controller fine-tuning the preset dispensing path data based on the deviation transformation parameters to obtain adjusted dispensing path data; generating the dispensing drive instruction based on the adjusted dispensing path data.

9. A control system of a fully automatic dispensing machine, characterized in that, Including: A dispensing start trigger control module, configured to send a camera trigger signal after detecting a dispensing task start instruction; An image acquisition and transmission module, configured for an industrial camera to acquire original image data including a workpiece reference feature area according to the camera trigger signal, and transmit the original image data including the workpiece reference feature area to an industrial control computer; an image feature point detection and positioning module, configured for the industrial control computer to detect and locate key reference feature points in the original image data including the workpiece reference feature area to obtain a set of positions of the key reference feature points in pixel coordinates; A coordinate system conversion processing module, configured to convert the key reference feature points from the pixel coordinate system to the world coordinate system based on the coordinate system calibration parameters of the camera and the machine motion system to obtain a set of positions of the key reference feature points in the world coordinate system; A feature point position comparison module, configured to compare the set of positions of the key reference feature points in the world coordinate system with the set of ideal positions of the key reference feature points in the world coordinate system to obtain deviation transformation parameters; a dispensing drive instruction generation module, configured to generate a dispensing drive instruction based on the deviation transformation parameters.

Images