Gluing robot gluing control method and system based on motion capture

Abstract

The invention discloses a gluing robot gluing control method and system based on motion capture, and the method comprises the steps: building a gluing gun rigid body model based on the structure parameters of a gluing gun, collecting the track data of a to-be-glued part through a motion capture system, building a dynamic motion primitive model based on the obtained glue gun track data, and carrying out the manual gluing of the to-be-glued part, when the gluing robot is adopted for gluing, parameter setting is carried out on the gluing track, wherein the parameter setting comprises the position, the posture and the gluing speed of a starting point and the position, the posture and the gluing speed of a terminal point; on the basis of the set parameters, a position coordinate data sequence, a posture data sequence and a speed data sequence from the starting point position to the final point position are generated through the built dynamic motion primitive model, and a gluing motion trail is generated. And the labor intensity of workers is reduced, the operation precision and stability are improved, and remarkable advantages can be better shown in a complex space.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of optical motion capture technology, in particular to a glue applying robot glue applying control method and system based on motion capture. BACKGROUND

[0002] Gluing is widely used in aviation manufacturing to seal the wing tank. Gluing work is highly repetitive, with many straight lines and circular curves, and the glue line is a sealing curve. The amount of glue applied needs to be uniform. Artificial glue application can result in uneven glue output, glue overflow, and incomplete glue sealing due to the fluid nature of glue.

[0003] With increasing demands for product quality in the industrial sector, the gluing process also needs to continuously improve precision and quality. For example, by using advanced sensors and control systems, precise control of the amount of glue applied can be achieved, thereby improving product quality and consistency.

[0004] With the development of emerging industries and the advancement of intelligent manufacturing, the market demand for gluing processes is growing. In particular, in the fields of semiconductor manufacturing, new energy vehicles, smart homes, and other fields, the application of gluing processes is becoming more and more widespread. Although robotic automated gluing has many advantages, there is still a technical gap in achieving human-like gluing control for robots.

[0005] Traditional task-based programming methods still have modeling errors and uncertain disturbances when achieving high precision operations. When a robot mimics human movements, due to limitations in control methods, it often cannot achieve truly human-like movements. SUMMARY

[0006] The purpose of the present application is to provide a glue applying robot glue applying control method and system based on motion capture to overcome the shortcomings of the prior art.

[0007] To achieve the above-mentioned purpose, the technical solution adopted by the present application is as follows: A glue applying robot glue applying control method based on motion capture, comprising the following steps: S1, based on the structure parameters of the glue gun, a rigid body model of the glue gun is constructed, trajectory data during manual gluing of the part to be glued is collected based on the rigid body model of the glue gun, and a dynamic motion primitive model is constructed based on the acquired trajectory data; S2, when using a glue applying robot for gluing, the parameters of the gluing trajectory are set, including the position, attitude, gluing speed of the starting point, and the position, attitude, gluing speed of the ending point, the position coordinate data sequence, attitude data sequence and speed data sequence from the starting point position to the ending point position are generated based on the above set parameters using the constructed dynamic motion primitive model, and the gluing motion trajectory is generated.

[0008] Preferably, reflective markers are set on the glue gun, and a visual motion capture system is used to identify the reflective markers to collect the movement information of the glue gun.

[0009] Preferably, a rigid body model of the glue gun is established, which specifically includes the following steps: The markers are evenly distributed on the glue gun, including key positions such as the glue gun nozzle and the circumference of the axis. At least 7 reflective markers are required. The position data of the reflective markers are set by the circumcircle of the glue gun axis to fit the initial posture of the glue gun axis.

[0010] Preferably, the motion information of the glue gun tip is collected by a visual motion capture system, including position, posture and speed information. Based on the glue gun rigid body model, the position, posture and speed information of the part of the glue gun held by the operator are converted.

[0011] Preferably, after collecting the adhesive application trajectory data, for any adhesive application trajectory data, a set is used to represent its time series data:

[0012] in: The position at time t; The velocity at time t; Let be the attitude angle at time t; This represents the number of data points on the i-th adhesive application trajectory. Representing the set of each glue application trajectory as elements of a set forms a high-level set:

[0013] in: This refers to the number of times the adhesive is applied. For the first The set of trajectories for each application of adhesive.

[0014] Preferably, the collected adhesive application trajectory data is preprocessed. First, the data is aligned. Second, a Gaussian mixture model is trained using multiple trajectory data. Finally, the Gaussian mixture model is used to predict the trajectory data to obtain an effective adhesive application trajectory.

[0015] Preferably, a Gaussian mixture model is trained using multiple trajectory data points: The Gaussian mixture model used is shown below:

[0016] in: K is the number of Gaussian distributions; The weights of the k-th Gaussian distribution are non-negative and satisfy the following conditions: ; It is the mean vector (center) of the k-th Gaussian distribution; It is the covariance matrix of the k-th Gaussian distribution, which determines the shape and direction of the distribution.

[0017] Preferably, based on the trained Gaussian mixture model, with a given position as input, Gaussian mixture regression is used to predict the corresponding velocity and attitude, thus obtaining an effective adhesive coating trajectory. The specific steps are as follows: Given an input x, the conditional mean of each Gaussian component can be calculated. Conditional covariance matrix :

[0018]

[0019] in: It is the mean of velocity, acceleration, and attitude in the Gaussian component k; It is the covariance matrix between position and velocity, acceleration, and attitude; It is the covariance matrix of the positions; It is the difference between the input position and the mean value of the Gaussian component position.

[0020] The conditional mean and conditional covariance of the entire Gaussian mixture model are obtained by weighted summation of each Gaussian component.

[0021] Weighted conditional mean of Gaussian mixture regression The calculation formula is:

[0022] in: It is the conditional mean of the k-th Gaussian component.

[0023] Weighted conditional covariance of Gaussian mixture regression The calculation formula is:

[0024] in: It is the conditional covariance of the k-th Gaussian component; It is the difference between the conditional mean of the k-th Gaussian component and the conditional mean of the entire Gaussian mixture regression.

[0025] Sampling from the conditional distribution p(y|x): , generated This corresponds to the velocity and attitude at a given input position x.

[0026] A motion capture-based glue-applying robot glue-applying control system, including a data acquisition system and a data processing control system: The data acquisition system constructs a rigid body model of the glue gun based on the glue gun parameters, collects trajectory data of manual glue application on the glue-to-be-applied area based on the rigid body model of the glue gun, and constructs a dynamic motion element model based on the acquired trajectory data. The data processing and control system sets parameters for the area to be coated with adhesive, including the starting position, attitude, and coating speed, as well as the ending position, attitude, and coating speed. Based on the parameters set above, it uses a constructed dynamic motion primitive model to generate a sequence of position coordinate data, attitude data, and velocity data from the starting position to the ending position, thus generating the coating motion trajectory.

[0027] Preferably, reflective markers are set on the glue gun, and a visual motion capture system is used to identify the reflective markers to collect the movement information of the glue gun.

[0028] Compared with the prior art, the present invention has the following beneficial technical effects: This invention discloses a glue-applying robot control method based on motion capture. It constructs a rigid body model of the glue gun based on its parameters, collects trajectory data of manual glue application on the area to be glued based on this model, and constructs a dynamic motion primitive model based on the acquired trajectory data. Parameters for the area to be glued are set, including starting position, posture, and glue application speed, as well as ending position, posture, and glue application speed. Based on these parameters, the constructed dynamic motion primitive model generates a sequence of position coordinate data, posture data, and velocity data from the starting position to the ending position, thus generating the glue application trajectory. This invention improves glue application efficiency and ensures glue application quality. By achieving human-like glue application through a robot, it not only reduces manual labor intensity but also improves operational accuracy and stability, exhibiting significant advantages, especially in complex spaces. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the glue application control method for a glue application robot based on motion capture in an embodiment of the present invention. Detailed Implementation

[0030] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0032] like Figure 1 As shown, this invention provides a glue-applying robot control method based on motion capture, specifically including the following steps: S1. Construct a rigid body model of the glue gun based on the structural parameters of the glue gun. Collect trajectory data of the glue gun when manually applying glue to the part to be glued based on the rigid body model of the glue gun. Construct a dynamic motion element model based on the acquired trajectory data. S2. When using a glue-applying robot for glue application, parameters are set for the glue application trajectory, including the starting position, attitude, glue application speed, and the ending position, attitude, and glue application speed. Based on the parameters set above, the constructed dynamic motion primitive model is used to generate a sequence of position coordinate data, attitude data, and speed data from the starting position to the ending position, thus generating the glue application motion trajectory.

[0033] In a specific embodiment of this application, reflective markers are set on the glue gun. The reflective markers are all set on the glue gun, including the glue gun nozzle and the circumference of the axis. At least 7 reflective markers need to be set. The set reflective markers are so that they can be monitored and captured in real time by the motion capture system. At the same time, the arrangement of the markers should not hinder the normal use and operation of the glue gun.

[0034] Specifically, a visual motion capture system is used to identify the reflective markings and collect information about the glue gun's movement. This system includes visual sensors and an image acquisition system. Visual sensors are deployed along the glue application path, with at least six sensors arranged to ensure no blind spots during movement tracking. This allows for multi-angle capture of the glue gun's movement, preventing target loss due to obstruction. Next, the visual sensors are connected to the image acquisition system, transmitting the image information captured by the visual sensors to the image acquisition system.

[0035] In a specific embodiment of this application, a visual motion capture system is used to capture the motion of a glue gun with reflective markings. This system can collect information on the position, posture, and speed of the markings on the glue gun and convert this information into information on the operator's hand movements while applying glue.

[0036] The rigid body model of the glue gun is created, which includes the following steps: The initial posture of the glue gun axis is fitted by the position data of the reflective marker points set by the circumcircle of the glue gun axis; every three in space can determine a circle, and the expression of the spatial circle is as follows.

[0037]

[0038] Substituting the coordinates of the three data points into the equations, the coordinates of the center and the radius of the circle can be obtained simultaneously. Then, based on the obtained coordinates of the three center points, the equation of the spatial straight line along the glue gun axis is fitted using the least squares method.

[0039] Choose the circumcircle closest to the tip of the glue gun and construct a rigid body along the Y-axis. Calculate the rotational transformation relationship between the axis space vector p and the rigid body space vector q. The calculation process is as follows.

[0040]

[0041]

[0042] Where vector n is the normal vector of the plane containing vectors p and q, and the rotation angle is... It is the angle between vectors p and q.

[0043]

[0044] In the formula, matrix N is the antisymmetric matrix form of vector n, that is:

[0045] The final rigid body transformation matrix T is as follows:

[0046] After obtaining the rotational transformation relationship between the two vectors, the rigid body is established by rotation, so that the axis of the rigid body coincides with the axis of the glue gun, ensuring that the initial posture of the rigid body and the initial posture of the glue gun are consistent.

[0047] The visual motion capture system collects motion information of the glue gun tip, including position, posture, and speed information. Based on the glue gun rigid body model, the position, posture, and speed information of the part of the glue gun held by the operator are converted.

[0048] In a specific embodiment of this application, the glue application is demonstrated multiple times along the same glue application trajectory, and the movement information of the glue gun is collected.

[0049] For any adhesive application trajectory data, a set is used to represent its time series data:

[0050] in: The position at time t; The velocity at time t; Let be the attitude angle at time t; denoted as the number of data points on the i-th adhesive application trajectory.

[0051] Representing the set of each glue application trajectory as elements of a set forms a high-level set:

[0052] in: This refers to the number of times the adhesive is applied. For the first The set of trajectories for each application of adhesive.

[0053] The collected adhesive application trajectory data were preprocessed. First, the data was aligned. Then, a Gaussian mixture model (GMM) was used to predict the trajectory data to obtain an effective adhesive application trajectory.

[0054] All trajectory data is saved to a single sample set:

[0055] Where N is the number of trajectories, It is the first The number of points on the trajectory.

[0056] In the specific implementation of this application, firstly, the collected data is aligned. If the lengths of the collected trajectory data are different, the trajectory data needs to be time aligned or interpolated to ensure that the length of each trajectory is consistent.

[0057] Secondly, a Gaussian Mixture Model (GMM) is trained using multiple trajectory data sets (including position, pose, and velocity). The Gaussian Mixture Model used is shown below:

[0058] in: K is the number of Gaussian distributions; The weights of the k-th Gaussian distribution are non-negative and satisfy the following conditions: ; It is the mean vector (center) of the k-th Gaussian distribution; It is the covariance matrix of the k-th Gaussian distribution, which determines the shape and direction of the distribution.

[0059] Finally, based on the trained Gaussian mixture model, with the given position as input, Gaussian mixture regression is used to predict the corresponding velocity and attitude.

[0060] The specific steps are as follows: Given an input x, the conditional mean of each Gaussian component can be calculated. Conditional covariance matrix :

[0061]

[0062] in: It is the mean of velocity, acceleration, and attitude in the Gaussian component k; It is the covariance matrix between position and velocity, acceleration, and attitude; It is the covariance matrix of the positions; It is the difference between the input position and the mean value of the Gaussian component position.

[0063] The conditional mean and conditional covariance of the entire Gaussian mixture model are obtained by weighted summation of each Gaussian component.

[0064] Weighted conditional mean of GMM The calculation formula is:

[0065] in: It is the conditional mean of the k-th Gaussian component.

[0066] Weighted Conditional Covariance of GMM The calculation formula is:

[0067] in: It is the conditional covariance of the k-th Gaussian component; It is the difference between the conditional mean of the k-th Gaussian component and the conditional mean of the entire GMM.

[0068] Sampling from the conditional distribution p(y|x): , generated This corresponds to the velocity and attitude at a given input position x.

[0069] The basic formula of the dynamic motion primitive model is shown below, which is obtained by changing the target state. and nonlinear terms Adjust the trajectory endpoint and trajectory shape. When it is necessary to change the trajectory speed, do so via the speed curve. Add a scaling item above To achieve this.

[0070]

[0071] In this dynamic control process, the system will eventually converge to a certain state. And nonlinear terms The participation will directly affect This directly affects the convergence process. By providing different... and It can obtain trajectories with different target states and different speeds based on the teaching curve. Redefined as:

[0072] in, Indicates the initial state ( ), The item can guarantee Will follow It converges and disappears.

[0073] basis functions The definition of is a radial basis function:

[0074] To construct a dynamic motion primitive model and describe the function approximation problem, the following formula is used:

[0075] The specific construction method is as follows: Input the valid trajectory data, taking the position coordinate y as an example, into the equation. On the left, we get:

[0076] For the weight w in the nonlinear term f, the loss function is constructed using the locally weighted regression (LWR) method:

[0077]

[0078] The solution is:

[0079] The expression obtained after construction is:

[0080] That is, to construct a dynamic motion primitive model with position coordinates y.

[0081] By inputting the three position data, three attitude angle data, and velocity data into the model using the above method, seven dynamic motion primitive models can be constructed.

[0082] The constructed dynamic motion primitive model is used for adhesive coating trajectory generation, and the specific method is as follows: By inputting the starting position, attitude, and glue application speed, as well as the ending position, attitude, and glue application speed, into the trained model, position coordinate data sequence, attitude data sequence, and velocity data sequence from the starting point to the ending point can be generated respectively, where the initial values ​​of the starting position, attitude, and velocity are all 0.

[0083] Taking the position coordinate y as an example, the specific application steps are explained as follows: Input the starting point y0 and the ending point g, and calculate the nonlinear forced term:

[0084] Acceleration can be calculated using dynamic equations:

[0085] in, , The first on the trajectory The point and the first points The position coordinates are obtained by numerical integration.

[0086]

[0087]

[0088] By inputting the three position data, three attitude angle data, and the initial and final values ​​of velocity respectively, the adhesive application trajectory data, including position, attitude, and velocity, can be generated using the above method.

[0089] Based on the obtained trajectory and speed information, the data is input into the robot controller to achieve robot glue application control.

[0090] Let the pose (position and orientation) of the end effector (glue gun) be: ,in: The robot's adhesive application task is based on inverse kinematics, and the controller receives the desired pose trajectory. and speed Calculate joint angular velocity Using the robot's kinematic equations:

[0091] Its first derivative is:

[0092] in, For the robot's Jacobian matrix; This refers to the terminal velocity.

[0093] Solving using inverse kinematics yields:

[0094] in, It is the pseudo-inverse of the Jacobian matrix.

[0095] In another specific embodiment of the present invention, a glue-applying robot control system based on motion capture, a data acquisition system, and a data processing control system are provided: The data acquisition system constructs a rigid body model of the glue gun based on the glue gun parameters, collects trajectory data of the glue gun during manual glue application on the area to be glued based on the rigid body model of the glue gun, and constructs a dynamic motion element model based on the acquired trajectory data. The data processing and control system sets parameters for the area to be coated with adhesive, including the starting position, attitude, and coating speed, as well as the ending position, attitude, and coating speed. Based on the parameters set above, it uses a constructed dynamic motion primitive model to generate a sequence of position coordinate data, attitude data, and velocity data from the starting position to the ending position, thus generating the coating motion trajectory.

[0096] Preferably, reflective markers are set on the glue gun, and a visual motion capture system is used to identify the reflective markers to collect the movement information of the glue gun.

[0097] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between components; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. Obviously, the embodiments described above are only some embodiments of this invention, not all embodiments. The accompanying drawings show preferred embodiments of the invention, but do not limit the patent scope of this invention. This invention can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this invention.

[0098] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this specification and drawings, whether directly or indirectly applied to other related technical fields, are similarly within the scope of protection of this patent.

Claims

1. A glue-applying robot control method based on motion capture, characterized in that, Includes the following steps: S1. Construct a rigid body model of the glue gun based on the structural parameters of the glue gun. Collect trajectory data of the glue gun when manually applying glue to the part to be glued based on the rigid body model of the glue gun. Construct a dynamic motion element model based on the obtained glue gun trajectory data. S2. When using a glue-applying robot for glue application, parameters are set for the glue application trajectory, including the starting position, attitude, glue application speed, and the ending position, attitude, and glue application speed. Based on the parameters set above, the constructed dynamic motion primitive model is used to generate a sequence of position coordinate data, attitude data, and speed data from the starting position to the ending position, thus generating the glue application motion trajectory.

2. The glue application control method for a glue application robot based on motion capture according to claim 1, characterized in that, Reflective markers are set on the glue gun, and a visual motion capture system is used to identify the reflective markers in order to collect the movement information of the glue gun.

3. The glue application control method for a glue application robot based on motion capture according to claim 1, characterized in that, The rigid body model of the glue gun is created, which includes the following steps: The markers are evenly distributed on the glue gun, including key positions such as the glue gun nozzle and the circumference of the axis. The position data of the reflective markers are set by the circumcircle of the glue gun axis to fit the initial posture of the glue gun axis.

4. The glue application control method for a glue application robot based on motion capture according to claim 2, characterized in that, The visual motion capture system collects motion information of the glue gun tip, including position, posture, and speed information. Based on the glue gun rigid body model, the position, posture, and speed information of the part of the glue gun held by the operator are converted.

5. The glue application control method for a glue application robot based on motion capture according to claim 1, characterized in that, After collecting the adhesive application trajectory data, for any adhesive application trajectory data, a set is used to represent its time series data: in: For time The position at that moment; For time The speed of time; For time Attitude angle at any moment; For the first Number of data points for the second adhesive application trajectory; Representing the set of each glue application trajectory as elements of a set forms a high-level set: in: This refers to the number of times the adhesive is applied; For the first The set of trajectories for each application of adhesive.

6. The glue application control method for a glue application robot based on motion capture according to claim 5, characterized in that, The collected adhesive application trajectory data is preprocessed. First, the data is aligned. Second, a Gaussian mixture model is trained using multiple trajectory data. Finally, the Gaussian mixture model is used to predict the trajectory data to obtain an effective adhesive application trajectory.

7. The glue application control method for a glue application robot based on motion capture according to claim 6, characterized in that, A Gaussian mixture model was trained using multiple trajectory data points. The Gaussian mixture model used is shown below: in: It is the number of Gaussian distributions; It is the first The weights of the Gaussian distribution are non-negative and satisfy the following conditions: ; It is the first The mean vector (center) of a Gaussian distribution; It is the first The covariance matrix of a Gaussian distribution determines the shape and direction of the distribution.

8. The glue application control method for a glue application robot based on motion capture according to claim 1, characterized in that, Based on the trained Gaussian mixture model, given a position as input, Gaussian mixture regression is used to predict the corresponding velocity and attitude, thus obtaining an effective adhesive application trajectory. The specific steps are as follows: Given input The conditional mean of each Gaussian component can be calculated. Conditional covariance matrix : in: It is a Gaussian component The mean values ​​of velocity, acceleration, and attitude in the middle; It is the covariance matrix between position and velocity, acceleration, and attitude; It is the covariance matrix of the positions; It is the difference between the input position and the mean value of the Gaussian component position.

9. A glue-applying robot control system based on motion capture, characterized in that, Data acquisition system and data processing and control system: The data acquisition system constructs a rigid body model of the glue gun based on the glue gun parameters, collects trajectory data of manual glue application on the glue-to-be-applied area based on the rigid body model of the glue gun, and constructs a dynamic motion element model based on the acquired trajectory data. The data processing and control system sets parameters for the area to be coated with adhesive, including the starting position, attitude, and coating speed, as well as the ending position, attitude, and coating speed. Based on the parameters set above, it uses a constructed dynamic motion primitive model to generate a sequence of position coordinate data, attitude data, and velocity data from the starting position to the ending position, thus generating the coating motion trajectory.

10. A glue-applying robot control system based on motion capture according to claim 9, characterized in that, Reflective markers are set on the glue gun, and a visual motion capture system is used to identify the reflective markers in order to collect the movement information of the glue gun.

Images