Information acquisition system and reproduction method for reproducing manual laying of two-dimensional fabrics

Abstract

The present invention relates to an information collection system and reproduction method for reproducing the manual laying of two-dimensional fabrics. The system includes a pressure roller that compacts the fabric on the mold surface, a clamping jaw that stretches and deforms the fabric, six-dimensional force / torque sensors installed above the pressure roller and the clamping jaw for force / torque detection of compaction and stretching, a motion capture system for detecting the motion trajectory of the pressure roller and the clamping jaw, and a depth camera for detecting the state of the fabric on the surface of the part mold. The reproduction method divides the laying process of the part mold into tasks and actions based on the information obtained by the reproduction system, obtains basic action data, and establishes a mathematical model using data encoding and data generalization methods. Based on the established data model, an admittance controller is used to design the normal force controller of the pressure roller and the six-dimensional force / torque controller of the clamping jaw, respectively, to achieve control of the robot's reproduction of manual operation skills. The present invention can be used for automated robot laying, thereby achieving high-quality and efficient laying.

Description

Technical Field

[0001] The present invention belongs to the technical field of two-dimensional fabric placement, and in particular relates to an information collection system and a method for reproducing manual placement of two-dimensional fabrics. Background Art

[0002] Two-dimensional carbon fiber fabrics are composed of interlaced warp and weft yarns. During the process of attaching a part to the mold surface, the change in the angle between the warp and weft yarns can adapt to the geometric shape of the part mold surface. Currently, for large-scale two-dimensional carbon fiber fabric parts with complex geometric features (such as planes, developable surfaces, non-developable surfaces, inclined surfaces, etc.), such as car seats and radar antenna covers with non-developable surfaces, honeycomb sandwiches with steep slopes, and ship hulls with multiple geometric features, they are generally laid by hand. The laying process is complicated, but the quality uniformity of manual laying is poor and the efficiency is low. Therefore, it is necessary to collect signals from the manual laying process and establish a skill expression model, and design a corresponding controller to facilitate the automated laying of robots based on the manual laying skill model. Summary of the Invention

[0003] The purpose of the present invention is to address the shortcomings of the existing technology and provide an information collection system for reproducing the manual placement of two-dimensional fabrics. The system can collect a series of complex manual placement operation skills and corresponding fabric states and reproduce them through a robot.

[0004] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0005] An information collection system for reproducing manual placement of two-dimensional fabrics, comprising:

[0006] A part mold for placing two-dimensional fabrics;

[0007] Pressing roller: A manual handheld pressing roller compacts the two-dimensional fabric laid on the part mold;

[0008] A first six-dimensional force / torque sensor is provided on the pressing roller and is used to detect the compaction force / torque during the compaction process;

[0009] Gripper, manually holding the gripper to stretch the two-dimensional fabric;

[0010] A second six-dimensional force / torque sensor is provided on the clamping jaw and is used to detect the stretching force / torque during the stretching process;

[0011] A plurality of motion capture tracking positioning balls, which are respectively arranged on the first six-dimensional force / torque sensor and the second six-dimensional force / torque sensor;

[0012] Multiple motion capture cameras, which are used to record the motion trajectory of the motion capture tracking positioning ball during the manual placement process;

[0013] A depth camera, which is positioned toward the part mold and is used to collect depth data of the part mold surface during manual placement; and

[0014] The data processing device is electrically connected to the first six-dimensional force / torque sensor, the second six-dimensional force / torque sensor, the plurality of motion capture cameras, and the depth camera.

[0015] Another object of the present invention is to provide an information collection system and method for reproducing the above-mentioned manual placement of two-dimensional fabrics, comprising the following steps:

[0016] The two-dimensional fabric is manually laid multiple times on the same part mold. During each laying process, a first six-dimensional force / torque sensor is used to collect the compaction force / torque during the roller compaction process, a second six-dimensional force / torque sensor is used to collect the tensile force / torque during the gripper stretching process, multiple motion capture cameras are used to record the motion trajectory and position of the motion capture tracking ball during manual laying, and a depth camera is used to collect point cloud information on the part mold surface;

[0017] The data processing device constructs a coordinate system for each component based on the collected information, and obtains motion trajectory data of multiple motion capture tracking spheres and force / torque information of the first six-dimensional force / torque sensor and the second six-dimensional force / torque sensor in the relevant coordinate system through a transformation matrix between the coordinate systems constructed between different components;

[0018] In a single placement process, the roller operation task is taken as the main task, and the gripper operation task is taken as the auxiliary task. The data processing device uses a method based on the compaction force threshold to segment the subtask data of different areas according to the collected compaction force / torque. The placement process of each track with a compaction force greater than the threshold is taken as each sub-action. Based on the start and end time information of each sub-task, the corresponding position and force / torque information of the roller and gripper operations contained in the sub-action time is obtained;

[0019] The data processing device determines whether the different placement areas pre-divided based on experience have been completed manually based on the changes in the depth information of the point cloud on the part mold surface, and obtains information on the completion process of different subtasks based on the different time information when the placement of each area is completed;

[0020] The data processing device constructs a contact model between the clamping jaw and the fabric during the teaching process based on the collected tensile force / torque during the stretching process, and obtains the contact stiffness between the clamping jaw and the fabric during the teaching process through the second six-dimensional force / torque data;

[0021] The data processing device encodes the sub-action data of the same task of repeated manual placement, extracts the essential features of each action, and reconstructs the extracted essential features of each action to obtain the generalized output of each action model. Then, according to the action and task sequence, the roller position and force, as well as the gripper position, force and contact stiffness of the sub-action in each sub-task are obtained;

[0022] The robot's admittance controller is designed based on the position and force of the pressure roller and the position, force and contact stiffness of the gripper in each sub-action of each sub-task, so as to design and control the normal force controller of the pressure roller and the six-dimensional force / torque controller of the gripper during robot operation, and finally realize the control of the robot's reproduction of human operation skills.

[0023] Furthermore, constructing the coordinate system of each element includes:

[0024] Constructing a motion capture system composed of multiple motion capture cameras: a coordinate system {O}; a coordinate system {A} of a first six-dimensional force / torque sensor mounted above the pressure roller; a coordinate system {B} of the end of the pressure roller; a coordinate system {C} of a second six-dimensional force / torque sensor mounted above the gripper; a coordinate system {D} of the end of the gripper; a rigid coordinate system {E} constructed by a motion capture tracking sphere on the first six-dimensional force / torque sensor; and a rigid coordinate system {F} constructed by a motion capture tracking sphere on the second six-dimensional force / torque sensor.

[0025] According to the constructed coordinate system, the homogeneous transformation matrix of the values ​​in coordinate system {A} to coordinate system {E}, the homogeneous transformation matrix of the values ​​in coordinate system {E} to coordinate system {B}, the homogeneous transformation matrix of the values ​​in coordinate system {C} to coordinate system {F}, the homogeneous transformation matrix of the values ​​in coordinate system {D} to coordinate system {F}, and the homogeneous transformation matrices of the values ​​in coordinate system {A}, the values ​​in coordinate system {B}, the values ​​in coordinate system {C}, the values ​​in coordinate system {D}, the values ​​in coordinate system {E}, and the values ​​in coordinate system {F} to coordinate system {O} are obtained.

[0026] Furthermore, during the laying process, the motion capture camera tracks the position of the rigid body coordinate system {E} of the motion capture tracking positioning ball on the first six-dimensional force / torque sensor in the coordinate system {O} E =(p E ,θ E ) T The position ξ of the coordinate system {F} formed by the motion capture tracking positioning ball on the second six-dimensional force / torque sensor in the coordinate system {O} F =(p F ,θ F ) T Perform motion capture, where p E and p F are position information, θE and θ F are respectively the postures, and according to the homogeneous transformation matrix, the positions of the roller coordinate system {B} and the gripper coordinate system {D} in the coordinate system {O} are ξ B =(p B ,θ B ) T ,ξ D =(p D ,θ D ) T , p B and p D are position information, θ B and θ D In addition, the value F of the first six-dimensional force / torque sensor in the coordinate system {A} is obtained. A =(f A ,τ A ) T , the value F of the second six-dimensional force / torque sensor in the coordinate system {C} C =(f C ,τ C ) T , where f A and f C are force, τ A and τ C are torque information respectively.

[0027] Furthermore, before building the model, it is necessary to filter the values ​​obtained by the first six-dimensional force / torque sensor and the second six-dimensional force / torque sensor. The method is as follows:

[0028] The first six-dimensional force / torque sensor information F of the collected single trajectory A and the second six-dimensional force / torque sensor information F C Perform sliding window filtering, the sliding window length is n, calculate the average value of the data contained in each window as the final teaching force / torque information; for each dimension value, take the sliding window length as n, and the adjacent values ​​are x i , x i- 1...x i-n+1 , calculate the average value of each window y i =(x i +x i-1 +...+x i-n+1 ) / n, and then obtain the filtered first six-dimensional force / torque sensor value F A '=(f A ',τ A ') T The second six-dimensional force / torque sensor value F C '=(fC ',τ C ') T , where f A ' and f C 'are force data, τ A ' and τ C ' are torque data respectively.

[0029] Furthermore, the method for obtaining the contact stiffness between the clamping jaw and the fabric is:

[0030] In the coordinate system, the gripping process of the gripper is equivalent to a second-order mass-spring model: in, x=ξ FO' They correspond to the gripper's posture acceleration, posture velocity, and posture respectively. In this model, M is equivalent to a unit diagonal matrix, and the damping coefficient C is selected based on experience, and the gripper tension / torque value F after filtering is used. C '、Gripper corresponding position ξ D The stiffness coefficient K corresponding to each moment is calculated using the time T.

[0031] Furthermore, the data collected from a single placement is segmented as follows:

[0032] The two-dimensional fabric placement task is divided into two layers. The upper layer is to manually divide the order of the part mold placement area into m1 placement areas based on experience, and each placement area is used as i different subtasks {C1,...,C i}, C i is the i-th subtask in the upper layer, and the bottom layer is the sub-actions {c1,...,c j}, c j is the jth subtask at the bottom layer; during the teaching process, each task is taught according to the order of the laying areas, and during the teaching process of a single task, the information of the first six-dimensional force / torque sensor, the second six-dimensional force / torque sensor and the position of each coordinate system in the coordinate system {O} are continuously collected;

[0033] The compaction force during manual teaching is used as the basis for further segmentation and judgment of each subtask teaching data. z When the current single trajectory is taught, the teaching data for each subtask is obtained.

[0034] Furthermore, the method of modeling and reconstructing to obtain the roller and gripper information contained in the sub-actions of each sub-task is as follows:

[0035] The number of sub-actions m2 in each area is kept consistent during each teaching process. After obtaining multiple teaching data of each sub-action corresponding to each area, the final sub-action including the trajectory corresponding to the pressure roller is obtained through processing. E 'Heli f z , the trajectory of the gripper ξ D , stiffness K, force / torque F C ;

[0036] Resample the multiple teaching data of each sub-action obtained above, place each dimension of the data in the range of independent variables 0 to 1, and fit a new curve. Then, re-discretize the data at the same interval and finally multiply it by the expected working time t exp , obtain data with the same number of discrete points;

[0037] According to the above-mentioned finally obtained several teaching data of a single sub-action with the same number of discrete points, the Gaussian mixture model GMM is used for clustering processing to extract the essential features of each action; and the Gaussian mixture regression model GMR is used to reconstruct the extracted essential features of each action to obtain the roller posture data ξ contained in the sub-action of each subtask BO’ '=(p BO’ ',θ BO’ ') T , the desired force f along the surface normal z 'And the gripper's position data ξ DO’ '=(p DO’ ',θ DO’ ') T , force / torque F DO’ '=(f DO’ ',τ DO’ ') T and stiffness coefficient K', where p BO’ ' and θ BO’ ' are the position and posture of the pressure roller, p DO’ ' and θ DO’ ' is the position and posture of the gripper, f DO’ ' is the gripper force, τ DO ” is the moment of the gripper;

[0038] The obtained roller posture data is converted into the roller posture in the surface normal coordinate system when the robot is working as ξ BO’ ”=(p BO' ”,θ BO’ ”) T , where p BO' ” and θ BO' ” are the final roller position and posture respectively.

[0039] Furthermore, the method for designing the normal force controller of the pressure roller is as follows:

[0040] Construct the base coordinate system {O'} when the robot reproduces the manual teaching operation, and convert the roller posture of the sub-action in each subtask into the roller posture in the base coordinate system {O'};

[0041] The unidirectional admittance controller is designed at the roller coordinate system {B} as follows: The expected force f d =f z , z-direction external force f ext , e is the position error, is the speed error, is the acceleration error, m is the mass; b is the damping, k is the stiffness;

[0042] The acceleration adjustment value in the z direction of the roller end is obtained by the unidirectional admittance controller The position adjustment amount is further obtained as Where Δt' is the time of the control cycle, is the current speed of the roller in the z direction; the z-position adjustment value x in the coordinate system {B} is obtained e , transform it into the adjustment value in the robot base coordinate system {O'}, specifically And add the robot's reference pose ξ at the next moment BO' ”=(p BO' ”,θ BO' ”) T , the actual control posture of the robot is ξ EO' c =(p BO' ”+ΔX,θ BO' ”) T .

[0043] Furthermore, the method for designing the six-dimensional force / torque controller of the gripper is as follows:

[0044] For the robot of gripping operation, a base coordinate system {O'} is constructed when the robot reproduces the manual teaching operation, and the gripper posture of the sub-action in each subtask is converted into the gripper posture under the base coordinate system {O'}; a six-degree-of-freedom admittance controller is used Where E = ξ F c -ξ ext ,ξ ext is the position of {D} relative to {O'} during the actual robot execution process, ξ F c The posture control posture of {D} relative to {O'}, combined with the control period Δt, can further obtain the speed and acceleration M1 is a six-dimensional diagonal unit mass matrix, B1 is a six-dimensional diagonal matrix, K1 = K'; F ext =(f ext ,τ ext ) T This is the actual data of the second six-dimensional force / torque sensor at the end of the gripper;

[0045] From the above, the acceleration adjustment of the coordinate system {D} relative to the coordinate system {O'} is The posture adjustment value is further obtained as where ξ e The position and attitude adjustment amounts are Δx DO' and Δθ DO' ,in and are the current posture velocity and acceleration adjustment of {D} relative to {O'} respectively; when the posture adjustment ξ relative to the gripper coordinate system {D} is obtained e ={Δx DO' ,Δθ DO'}, the position control value of the gripper at the next moment is obtained as ξ F c =(Δx DO' +p DO’ ',Δθ DO' +θ DO’ ') T .

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] The present invention realizes the collection of skill information of manual laying of two-dimensional fabrics, obtains the skill data during the manual laying process, and segments the laying tasks and actions of complex molds. Based on the segmentation results, a manual laying skill expression model is established. Finally, a position control robot based on an admittance force controller that can be used for commercialization is designed, thereby realizing the robot's automatic laying according to the manual laying skill model to replace the manual laying skill model, greatly improving production efficiency and uniformity of laying quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of the structure of an information collection system for reproducing manual placement of two-dimensional fabrics according to an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of the coordinate system of each component constructed for an embodiment of the present invention;

[0050] Figure 3 The flowchart of the reproduction method of the information collection system for manually laying two-dimensional fabrics according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0052] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0053] The present invention will be further described below with reference to specific examples, but they are not intended to limit the present invention.

[0054] The embodiment of the present invention discloses an information collection system for reproducing manual laying of two-dimensional fabrics. Figure 1 As shown, it includes a pressure roller 2 for compacting the two-dimensional carbon fiber fabric onto a part mold 1, a first six-dimensional force / torque sensor 3 mounted on the pressure roller 2, a clamping jaw 4 for pulling the two-dimensional carbon fiber fabric during the placement process, a second six-dimensional force / torque sensor 5 mounted on the clamping jaw 4, a motion capture tracking sphere 6 mounted on the surface of the first six-dimensional force / torque sensor 3, a motion capture tracking sphere 7 attached to the second six-dimensional force / torque sensor 5, multiple motion capture cameras 9 spaced around the part mold via a support frame 8, a depth camera 11 positioned toward the part mold via a fixed bracket 10, and a data processing device 12 electrically connected to the first six-dimensional force / torque sensor 3, the second six-dimensional force / torque sensor 5, the multiple motion capture cameras 9, and the depth camera 11. In this embodiment, the data processing device 12 is a computer. For ease of holding, handles are provided on both the pressure roller 2 and the clamping jaw 4.

[0055] like Figure 2 As shown, the coordinate systems of each component are constructed, specifically: the coordinate system of the motion capture system composed of multiple motion capture cameras {O}, the coordinate system of the first six-dimensional force / torque sensor 3 installed above the pressure roller 2 {A}, the coordinate system of the end of the pressure roller 2 {B}, the coordinate system of the second six-dimensional force / torque sensor 5 installed above the clamp 4 {C}, the coordinate system of the end of the clamp 4 {D}, the rigid body coordinate system {E} constructed by the motion capture tracking positioning ball 6 on the first six-dimensional force / torque sensor 3 on the pressure roller 2, and the rigid body coordinate system {F} constructed by the motion capture tracking positioning ball 7 on the second six-dimensional force / torque sensor 5 on the clamp 4.

[0056] As shown in equation (1): The position vector of the origin of coordinate system {A} relative to coordinate system {E} is E P A , the orientation of coordinate system {A} relative to coordinate system {E} is The homogeneous transformation matrix from the value in coordinate system {A} to coordinate system {E} is The position vector of the origin of coordinate system {B} relative to coordinate system {E} is E P B , the orientation of coordinate system {B} relative to coordinate system {E} is The homogeneous transformation matrix from the value in coordinate system {B} to coordinate system {E} is The position vector of the origin of coordinate system {E} relative to coordinate system {O} is O P E , the orientation of coordinate system {E} relative to coordinate system {O} is The homogeneous transformation matrix from the value in coordinate system {E} to coordinate system {O} is

[0057]

[0058] As shown in formula (2): The position vector of the origin of coordinate system {C} relative to coordinate system {F} is F P C , the orientation of coordinate system {C} relative to coordinate system {F} is The homogeneous transformation matrix from the value in coordinate system {C} to coordinate system {F} is The position vector of the origin of coordinate system {D} relative to coordinate system {F} is F P D , the orientation of coordinate system {D} relative to coordinate system {F} is The homogeneous transformation matrix from the value in coordinate system {D} to coordinate system {F} is The position vector of the origin of coordinate system {F} relative to coordinate system {O} is O P F , the coordinate system relative to the coordinate system {O} is The homogeneous transformation matrix from the value in coordinate system {F} to coordinate system {O} is

[0059]

[0060] As shown in formula (3), the transformation matrix from the value in coordinate system {A} to coordinate system {O} is The transformation matrix from the value in coordinate system {B} to coordinate system {O} is

[0061]

[0062] As shown in formula (4), the transformation matrix from the value in coordinate system {C} to coordinate system {O} is The transformation matrix from the value in coordinate system {D} to coordinate system {O} is

[0063]

[0064] like Figure 3 As shown, first, the data of the manual laying process is collected. During the manual teaching process, the same person or different people hold the corresponding handle on the pressure roller 2 and the corresponding handle on the clamp 4 respectively, and the motion capture camera 9 captures the position and posture of the rigid coordinate system {E} and {F}. The human hand applies pressure to the pressure roller 2 through the handle to press the fabric on the surface of the part mold 1. The clamp 4 prevents the fabric from contacting the part mold 1 in advance during the laying process, and applies a certain tension to the fabric so that the fabric adapts to the curved surface fitting requirements. The motion capture camera 9 captures the position and posture of the rigid coordinate system {E} formed by the motion capture tracking positioning balls 6 and 7 on the first six-dimensional force / torque sensor 3 on the pressure roller 2 and the second six-dimensional force / torque sensor 5 corresponding to the clamp 4 in the coordinate system {O} E =(p E ,θ E ) T and the pose ξ of the coordinate system {F} F =(p F ,θ F ) T Perform motion capture, where p E and p F are position information, θ E and θ F Through equations (3) and (4), we can obtain the poses of coordinate system {B} and coordinate system {D} in coordinate system {O} as ξ B =(p B ,θ B ) T ,ξ D =(p D ,θ D ) T , where p B and p D are position information, θ B and θ D At the same time, the value of the first six-dimensional force / torque sensor 3 on the pressure roller 2 in the coordinate system {A} is F A =(f A ,τ A ) T The value of the second six-dimensional force / torque sensor 5 on the gripper 4 in the coordinate system {C} is F C =(f C ,τ C ) T , where f A and f C are force, τ A and τ Care torque information respectively.

[0065] The data collected from the continuous single placement of the mold is segmented by the computer 12. The computer divides the fabric placement task into two layers. The upper layer is the manual division of the order of the placement of the part mold 1 into m1 placement areas based on experience, and each placement area is used as i different subtasks {C1,...,C i}, C i is the i-th subtask in the upper layer, and the bottom layer is the sub-actions {c1,...,c j}, c j During the teaching process, each task is taught in the order of the placement area, and during the teaching process of a single task, the information of two six-dimensional force / torque sensors and the position of each coordinate system in the coordinate system {O} are continuously collected.

[0066] A depth camera 11 is used to collect real-time surface information of the mold 1 during manual placement, ultimately obtaining time-series-related mold surface depth information d. After the fabric is placed, the mold surface depth information changes to d'. By comparing the change in depth information over time during the placement process (Δd = d' - d), the computer determines the depth change of the placement area over time. This information then determines whether the manual placement area division is complete and determines the time index t1 for each area's completion, thus segmenting the collected data for the corresponding areas during the entire mold placement process.

[0067] The compaction force during manual teaching is used as the basis for further segmentation and judgment of the teaching data of each area (subtask). thod =5N, the teaching of the current single trajectory is completed, and then the teaching data for completing the single trajectory (sub-action) is obtained.

[0068] Since the data obtained by the six-dimensional force / torque sensor contains noise, it brings certain difficulties to mathematical modeling. Therefore, alignment and filtering are first performed. Specifically, the first six-dimensional force / torque information F of the single trajectory collected above is A and the second six-dimensional force / torque information F C Perform sliding window filtering, the sliding window length is set to n, and calculate the average value of the data contained in each window as the final taught force / torque information. For each dimension value, take the sliding window length to n, and the adjacent values ​​are x i , x i-1 ...x i-n+1 , calculate the average value of each window y i =(x i +x i-1 +...+x i-n+1) / n, and then obtain the filtered roller force / torque value F A '=(f A ',τ A ') T and the gripper force / torque value F C '=(f C ',τ C ') T , where f A ' and f C 'are force data, τ A ' and τ C ' are torque data respectively.

[0069] Since the robot is working in the contact task, the general pressure is along the surface normal. The computer searches for the position p of the pressure roller in the coordinate system {O} through the information of the mold surface STL (STereoLithography) in the coordinate system {O}. E Search and get the normal vector of the triangle center at the corresponding position on the corresponding surface As the z direction, the forward direction is the x direction, and the new y direction is obtained as Then the new x direction is And through the transformation relationship between the rotation matrix and the quaternion, the rotation matrix {x, y, n} is transformed to obtain the quaternion q E ', and the final trajectory along the mold normal is ξ E '=(p E ,θ E ') T , where p E and θ E ' are position and posture respectively. The coordinate axes of {A} and {B} are parallel and the force values ​​along the z-axis are the same. Therefore, the z-direction force f of the force data collected by the six-dimensional force / torque sensor above the pressure roller can be z =f A '·n, where n = [0, 0, 1], and is used as the desired force of the roller during robot operation. During the compaction process of roller 2, gripper 4 needs to assist in completing the task synchronously. The gripper's actions in the time periods corresponding to the roller's subtasks and sub-actions are divided into gripper subtasks and sub-actions.

[0070] The position and posture data obtained above are in the coordinate system {O}. When the robot reproduces the manual teaching operation, the control of the robot is equivalent to the robot's base coordinate system {O'}. The homogeneous transformation matrix from the coordinate system {O} to the coordinate system {O'} is in is the rotation matrix, O 'P Ois the position vector of the origin of the coordinate system {O} relative to the coordinate system {O'}, and then the above-mentioned pose data ξ B and ξ D Transform to the robot's base coordinate system {O'}. The position of the roller in the coordinate system {O'} is ξ BO' =(p BO' ,θ BO' ) T , where p BO' is the position; θ BO' is the posture, and the posture of the gripper is ξ DO' =(p DO' ,θ DO' ) T , where p DO' is the position, θ DO' For posture.

[0071] Since the deformation process of carbon fiber fabric is nonlinear, the clamping force during the placement process will facilitate placement on the one hand, and cause it to deform to adapt to the placement on the other hand. During the deformation process, the human arm is working in a process of variable stiffness. To ensure the robot's reproduction effect, the robot is made to reproduce this variable stiffness clamping process. In the coordinate system {O'}, the clamping process of the gripper is equivalent to a second-order mass spring model: in, x=ξ FO' They correspond to the position acceleration, position velocity, and position of the gripper, which are the teaching data corresponding to each sub-action. F is the force / torque. In this model, M is equivalent to the unit diagonal matrix. The damping coefficient B is selected based on experience, and the stiffness coefficient K can be calculated. Force / torque in coordinate system {O'} Where D is the vector O ' P D The antisymmetric matrix, O'P D is the position vector of the origin of coordinate system {D} relative to coordinate system {O'}; is the rotation transformation matrix from {D} to {O'}; D1 is a vector D P C The antisymmetric matrix of D P C is the position vector of the origin of coordinate system {C} relative to coordinate system {D}, is the rotation transformation matrix from {C} to {D}.

[0072] The above operations can complete the collection and processing of teaching information. Usually, in order to avoid accidental teaching, it is necessary to teach the same mold placement process several times. In each teaching process, the number of sub-actions m2 in each area is kept consistent, which is convenient for modeling multiple teaching data. After obtaining multiple teaching data of each sub-action corresponding to each area, and through the above processing process, the final sub-action includes the trajectory corresponding to the pressure roller ξ E 'Heli f z , the trajectory of the gripper ξ DO' , stiffness K, force / torque F DO' Resample the multiple teaching data of each sub-action obtained above, place each dimension of the data in the range of 0 to 1 of the independent variable, and fit a new curve. Then, re-discretize the data at the same interval and finally multiply it by the expected working time t exp , and obtain data with the same number of discrete points.

[0073] The teaching data of the above-mentioned single sub-action with the same number of discrete points are clustered using K Gaussian mixture models (GMM) to obtain K Gaussian cluster distributions of the teaching data, where K is a positive integer. The parameters of the K Gaussian mixture models are α k represents the weight of the kth Gaussian distribution, μ k ,Σ k are the mean and variance of the kth sub-model respectively. The number of Gaussian components K is determined by the Bayesian Information Criterion (BIC) criterion. The expression of the BIC criterion is S BIC = -2ln(L) + Kln(n), where L is the maximum value of the likelihood function of the Gaussian mixture model and n is the number of teaching samples. BIC When S BIC The K value corresponding to the minimum value is the desired K value.

[0074] For a GMM with K Gaussian distributions linearly added, the weight of the kth Gaussian distribution is α k , and satisfies and 0≤α k ≤1, jth observation data ξ j =(ξ t ,ξ s ),i=1,2,3,…,N,ξ s is the trajectory or force / torque variable in the teaching data, ξ t is a time variable. The probability distribution of a single data point in the overall data is Among them, the distribution of the k-th Gaussian model is μ k ,Σ k are the mean and variance of the kth sub-model respectively; α k Is the prior probability. Use the Expectation Maximization (EM) algorithm to iteratively calculate the GMM model parameters Furthermore, μ k and Σ k The input time and output components can be expressed as μ k ={μ t,k ,μ s,k}and where μ t,k is the mean of input time data, μ s,k is the mean of the output data, Σ tt,k is the variance of time, Σ ts,k and ∑ st,k is the covariance of the time data and the output data. In the process of reproducing the teaching trajectory, at each iterative time step t, the time variable value ξ is given. t , the expected trajectory ξ s The conditional distribution of The probability of the kth Gaussian model is N(ξ t ;μ t,i ,Σ tt,i ) is μ t,i and Σ tt,i are Gaussian distributions with mean and variance respectively; given input ξ t , conditional distribution P(ξ s |ξ t ) is the predicted mean of Prediction variance According to the linear characteristics of Gaussian distribution, the total expectation and variance are used to convert P(ξ s |ξ t ) is approximately Gaussian, and P(ξ s |ξ t ) distribution is defined as The mean is The variance is Will It is called Gaussian Mixture Regression (GMR) with index K. When the robot reproduces the manual operation, the trajectory reproduced by the robot is P(ξ s |ξ t )’s expected value E(ξ s |ξ t ).

[0075] The multiple collected data of the sub-actions included in each sub-task are processed by GMM / GMR, and the position data of the roller included in the sub-actions in each sub-task are output by GMR. BO ”=(p BO ”,θ BO ”) T , the desired force f along the surface normal z '、Gripper's position data ξ DO ”=(p DO ”,θ DO ”) T , force / torque F of the gripper DO ”=(f DO ”,τ DO ”) T And the stiffness coefficient K' during the gripper's task execution, where p BO ” and θ BO " are the position and posture of the pressure roller, p DO ” and θ DO ” is divided into the position and posture of the gripper, f DO ” is the force of the gripper, τ DO " is the moment of the gripper. But at this time p BO The location points contained in the " may not be on the mold surface, θ BO The posture contained in it is also not perpendicular to the mold surface. In order to obtain the normal direction along the mold surface as much as possible, first search for the distance p on the mold. BO The nearest triangle center point is used as the new roller position point p BO ”', and obtain the new roller posture θ along the normal movement of the mold surface according to the above method BO ”', the final posture of the pressure roller is ξ BO ”'=(p BO ”',θ BO ”') T , where p BO ”' and θ BO ” and ' are the final roller position and posture respectively.

[0076] Since position-controlled robots are more widely used in commerce than torque-controlled robots, a position-controlled robot is used when migrating the manual fabric laying operation to a robot, and an admittance controller is used for the operation at the end of the robot to ensure the force / torque reproduction effect. When the robot reproduces the task, a unidirectional admittance controller is used to track the normal force, and the end of the robot is allowed to move along the surface normal in the z direction. The joint posture of the end of the robot is the same as the posture of the pressure roller, so the z-direction force of the end of the robot is the same as the z-direction force of the pressure roller. The six-dimensional force / torque sensor at the end of the robot is used to detect the z-direction force, and the admittance force control is used for force feedback adjustment. The z-direction unidirectional admittance controller is designed at the pressure roller coordinate system {B} as follows: The expected force f d =f z ; Actual external force in z direction f ext ; e = x dz -x ez is the position error, is the speed error, is the acceleration error, where p BO ”” is the actual position of {B} in {O'} during the robot's task execution; m is the mass, b is the damping, k f is the stiffness. The homogeneous transformation matrix between the robot base coordinate system and the roller end coordinate system {B} in is the rotation matrix from {O'} to {B}; B P O' is the position vector of the origin of coordinate system {O'} relative to coordinate system {B}.

[0077] The acceleration adjustment value in the z direction of the roller end is obtained by the unidirectional admittance controller The position adjustment amount is further obtained as Where Δt is the time of the control cycle, The current speed of the roller in the z direction. The z-direction position adjustment x in the coordinate system {B} is obtained. e , and further transform it to the base coordinate system {O'} of the robot arm, and the position adjustment amount in the base coordinate system of the robot is obtained as And add the robot's reference pose ξ at the next moment BO ”'=(p BO ”',θ BO ”') T , the actual control posture of the robot is ξ EO' c =(p BO ”'+ΔX,θ BO ”') T .

[0078] For the robot of gripping operation, a six-degree-of-freedom admittance controller is used in the coordinate system {O'} Where E = ξ F c -ξ ext ,ξ ext is the position of {D} relative to {O'} during the actual robot execution process, ξ F c The posture control posture of {D} relative to {O'}, combined with the control period Δt, can further obtain the speed and acceleration M1 is a six-dimensional diagonal unit mass matrix; B1 is a six-dimensional diagonal matrix; the value of B1 needs to be adjusted according to the actual force tracking effect; K1 = K'; F ext =(f ext ,τ ext ) T The actual data of the second six-dimensional force / torque sensor at the end of the gripper, the force / torque in the coordinate system {O'} D2 is the position vector O'P of the origin of coordinate system {C} relative to coordinate system {O'} C The antisymmetric matrix of the coordinate system {D}. The acceleration adjustment of the coordinate system {O'} is The posture adjustment value is further obtained as where ξ e The position and attitude adjustment amounts are Δx DO' and Δθ DO' ,in and They are the current posture velocity and acceleration adjustment of {D} relative to {O'}. After obtaining the posture adjustment ξ relative to the gripper coordinate system {D} e ={Δx DO' ,Δθ DO'}, the position control value of the gripper at the next moment is obtained as ξ F c =(Δx DO' +p DO ”,Δθ DO' +θ DO ”) T A six-degree-of-freedom admittance controller is used to adjust the position and posture of the robot during gripping operations, achieving the reproduction of the taught posture and force / torque.

[0079] The above are only preferred embodiments of the present invention and do not limit the implementation mode and protection scope of the present invention. For those skilled in the art, it should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the contents of the present invention specification should be included in the protection scope of the present invention.

Claims

1. An information collection system for reproducing manually laid two-dimensional fabrics, characterized in that: include: A part mold for placing two-dimensional fabrics; Pressing roller: A manual handheld pressing roller compacts the two-dimensional fabric laid on the part mold; A first six-dimensional force / torque sensor is provided on the pressing roller and is used to detect the compaction force / torque during the compaction process; Gripper, manually holding the gripper to stretch the two-dimensional fabric; A second six-dimensional force / torque sensor is provided on the clamping jaw and is used to detect the stretching force / torque during the stretching process; A plurality of motion capture tracking positioning balls, which are respectively arranged on the first six-dimensional force / torque sensor and the second six-dimensional force / torque sensor; Multiple motion capture cameras, which are used to record the motion trajectory of the motion capture tracking positioning ball during the manual placement process; A depth camera is positioned toward the part mold to collect depth data of the part mold surface during manual placement; as well as The data processing device is electrically connected to the first six-dimensional force / torque sensor, the second six-dimensional force / torque sensor, the plurality of motion capture cameras, and the depth camera.

2. A method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 1, characterized in that: The steps include: The two-dimensional fabric is manually laid multiple times on the same part mold. During each laying process, a first six-dimensional force / torque sensor is used to collect the compaction force / torque during the roller compaction process, a second six-dimensional force / torque sensor is used to collect the tensile force / torque during the gripper stretching process, multiple motion capture cameras are used to record the motion trajectory and position of the motion capture tracking ball during manual laying, and a depth camera is used to collect point cloud information on the part mold surface; The data processing device constructs a coordinate system for each component based on the collected information, and obtains motion trajectory data of multiple motion capture tracking spheres and force / torque information of the first six-dimensional force / torque sensor and the second six-dimensional force / torque sensor in the relevant coordinate system through a transformation matrix between the coordinate systems constructed between different components; In a single placement process, the roller operation task is taken as the main task, and the gripper operation task is taken as the auxiliary task. The data processing device uses a method based on the compaction force threshold to segment the subtask data of different areas according to the collected compaction force / torque. The placement process of each track with a compaction force greater than the threshold is taken as each sub-action. Based on the start and end time information of each sub-task, the corresponding position and force / torque information of the roller and gripper operations contained in the sub-action time is obtained; The data processing device determines whether the different placement areas pre-divided based on experience have been completed manually based on the changes in the depth information of the point cloud on the part mold surface, and obtains information on the completion process of different subtasks based on the different time information when the placement of each area is completed; The data processing device constructs a contact model between the clamping jaw and the fabric during the teaching process based on the collected tensile force / torque during the stretching process, and obtains the contact stiffness between the clamping jaw and the fabric during the teaching process through the second six-dimensional force / torque data; The data processing device encodes the sub-action data of the same task of repeated manual placement, extracts the essential features of each action, and reconstructs the extracted essential features of each action to obtain the generalized output of each action model. Then, according to the action and task sequence, the roller position and force, as well as the gripper position, force and contact stiffness of the sub-action in each sub-task are obtained; The robot's admittance controller is designed based on the position and force of the pressure roller and the position, force and contact stiffness of the gripper in each sub-action of each sub-task, so as to design and control the normal force controller of the pressure roller and the six-dimensional force / torque controller of the gripper during robot operation, and finally realize the control of the robot's reproduction of human operation skills.

3. The method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 2, characterized in that: The coordinate systems for constructing each component include: Constructing a motion capture system composed of multiple motion capture cameras: a coordinate system {O}; a coordinate system {A} of a first six-dimensional force / torque sensor mounted above the pressure roller; a coordinate system {B} of the end of the pressure roller; a coordinate system {C} of a second six-dimensional force / torque sensor mounted above the gripper; a coordinate system {D} of the end of the gripper; a rigid coordinate system {E} constructed by a motion capture tracking sphere on the first six-dimensional force / torque sensor; and a rigid coordinate system {F} constructed by a motion capture tracking sphere on the second six-dimensional force / torque sensor. According to the constructed coordinate system, the homogeneous transformation matrix of the values ​​in coordinate system {A} to coordinate system {E}, the homogeneous transformation matrix of the values ​​in coordinate system {E} to coordinate system {B}, the homogeneous transformation matrix of the values ​​in coordinate system {C} to coordinate system {F}, the homogeneous transformation matrix of the values ​​in coordinate system {D} to coordinate system {F}, and the homogeneous transformation matrices of the values ​​in coordinate system {A}, the values ​​in coordinate system {B}, the values ​​in coordinate system {C}, the values ​​in coordinate system {D}, the values ​​in coordinate system {E}, and the values ​​in coordinate system {F} to coordinate system {O} are obtained.

4. The method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 3, characterized in that: During the placement process, the motion capture camera's position in the coordinate system {O} of the rigid body coordinate system {E} formed by the motion capture tracking positioning ball on the first six-dimensional force / torque sensor The position ξ of the coordinate system {F} formed by the motion capture tracking positioning ball on the second six-dimensional force / torque sensor in the coordinate system {O} F =(p F ,θ F ) T Perform motion capture, where p E and p F are position information, θ E and θ F are respectively the postures, and according to the homogeneous transformation matrix, the positions of the roller coordinate system {B} and the gripper coordinate system {D} in the coordinate system {O} are ξ B =(p B ,θ B ) T ,ξ D =(p D ,θ D ) T , p B and p D are position information, θ B and θ D They are posture; In addition, the value F of the first six-dimensional force / torque sensor in the coordinate system {A} is obtained A =(f A ,τ A ) T , the value F of the second six-dimensional force / torque sensor in the coordinate system {C} C =(f C ,τ C ) T , where f A and f C are force, τ A and τ C are torque information respectively.

5. The method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 4, characterized in that: Before building the model, it is necessary to filter the values ​​obtained by the first six-dimensional force / torque sensor and the second six-dimensional force / torque sensor. The method is as follows: The first six-dimensional force / torque sensor information F of the collected single trajectory A and the second six-dimensional force / torque sensor information F C Perform sliding window filtering, the sliding window length is n, calculate the average value of the data contained in each window as the final teaching force / torque information; for each dimension value, take the sliding window length as n, and the adjacent values ​​are x i , x i-1 ...x i-n+1 , calculate the average value of each window y i =(x i +x i-1 +...+x i-n+1 ) / n, and then obtain the filtered first six-dimensional force / torque sensor value F A '=(f A ',τ A ') T The second six-dimensional force / torque sensor value F C '=(f C ',τ C ') T , where f A ' and f C 'are force data, τ A ' and τ C ' are torque data respectively.

6. The method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 5, characterized in that: The method to obtain the contact stiffness between the gripper and the fabric is: In the robot coordinate system, the gripping process of the gripper is equivalent to a second-order mass-spring model: in, x corresponds to the position acceleration, position velocity, and position of the gripper, respectively. It is the teaching data corresponding to each sub-action. F is the force / torque. In this model, M is equivalent to the unit diagonal matrix, B is the damping coefficient, and the stiffness coefficient K is obtained by calculation.

7. The method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 3, characterized in that: The method for segmenting the data collected for a single placement is: The two-dimensional fabric placement task is divided into two layers. The upper layer is to manually divide the order of the part mold placement area into m1 placement areas based on experience, and each placement area is used as i different subtasks {C1,...,C i }, C i is the i-th subtask in the upper layer, and the bottom layer is the sub-actions {c1,...,c j }, c j is the jth subtask at the bottom layer; during the teaching process, each task is taught according to the order of the laying areas, and during the teaching process of a single task, the information of the first six-dimensional force / torque sensor, the second six-dimensional force / torque sensor and the position of each coordinate system in the coordinate system {O} are continuously collected; The compaction force during manual teaching is used as the basis for further segmentation and judgment of each subtask teaching data. z When the current single trajectory is taught, the teaching data for each subtask is obtained.

8. The method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 7, characterized in that: The method of modeling and reconstructing to obtain the roller and gripper information contained in the sub-actions of each sub-task is as follows: The number of sub-actions m2 in each area is kept consistent during each teaching process. After obtaining multiple teaching data of each sub-action corresponding to each area, the final sub-action including the trajectory corresponding to the pressure roller is obtained through processing. E 'Heli f z , the trajectory of the gripper ξ D , stiffness k, force / torque F C ; Resample the multiple teaching data of each sub-action obtained above, place each dimension of the data in the range of independent variables 0 to 1, and fit a new curve. Then, re-discretize the data at the same interval and finally multiply it by the expected working time t exp , obtain data with the same number of discrete points; According to the above-mentioned finally obtained several teaching data of a single sub-action with the same number of discrete points, the Gaussian mixture model GMM is used for clustering processing to extract the essential features of each action; and the Gaussian mixture regression model GMR is used to reconstruct the extracted essential features of each action to obtain the roller posture data ξ contained in the sub-action of each subtask BO' '=(p BO' ',θ BO' ') T , the desired force f along the surface normal z 'And the gripper's position data ξ DO' '=(p DO' ',θ DO' ') T , force / torque F DO' '=(f DO' ',τ DO' ') T and stiffness coefficient K', where p BO' ' and θ BO' ' are the position and posture of the pressure roller, p DO' ' and θ DO' ' is the position and posture of the gripper, f DO' ' is the gripper force, τ DO' ' is the moment of the gripper; The obtained roller posture data is converted into the roller posture in the surface normal coordinate system when the robot is working as ξ BO' ”=(p BO' ”,θ BO' ”) T , where p BO' ” and θ BO' ” are the final roller position and posture respectively.

9. The method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 8, characterized in that: The method for designing the normal force controller of the pressure roller is: Construct the base coordinate system {O'} when the robot reproduces the manual teaching operation, and convert the roller posture of the sub-action in each subtask into the roller posture in the base coordinate system {O'}; The unidirectional admittance controller is designed at the roller coordinate system {B} as follows: The expected force f d =f z , z-direction external force f ext , e is the position error, is the speed error, is the acceleration error, m is the mass; b is the damping, k is the stiffness; The acceleration adjustment value in the z direction of the roller end is obtained by the unidirectional admittance controller The position adjustment amount is further obtained as Where Δt is the time of the control cycle, is the current speed of the roller in the z direction; the z-position adjustment value x in the coordinate system {B} is obtained e , transform it into the adjustment value in the robot base coordinate system {O'}, specifically And add the robot's reference pose ξ at the next moment BO' ”=(p BO' ”,θ BO' ”) T , the actual control posture of the robot is ξ EO' c =(p BO' ”+ΔX,θ BO' ”) T .

10. The method for reproducing an information collection system for manually laying two-dimensional fabrics according to claim 8, characterized in that: The method for designing the six-dimensional force / torque controller of the gripper is: For the robot of gripping operation, a base coordinate system {O'} is constructed when the robot reproduces the manual teaching operation, and the gripper posture of the sub-action in each subtask is converted into the gripper posture under the base coordinate system {O'}; a six-degree-of-freedom admittance controller is used Where E = ξ F c -ξ ext ,ξ ext is the position of {D} relative to {O'} during the actual robot execution process, ξ F c The posture control posture of {D} relative to {O'} is further obtained by combining the control period Δt to obtain the velocity and acceleration M1 is a six-dimensional diagonal unit mass matrix, B1 is a six-dimensional diagonal matrix, K1 = K'; F ext =(f ext ,τ ext ) T This is the actual data of the second six-dimensional force / torque sensor at the end of the gripper; From the above, the acceleration adjustment of the coordinate system {D} relative to the coordinate system {O'} is The posture adjustment value is further obtained as where ξ e The position and attitude adjustment amounts are Δx DO' and Δθ DO' ,in and are the current posture velocity and acceleration adjustment of {D} relative to {O'} respectively; when the posture adjustment ξ relative to the gripper coordinate system {D} is obtained e ={Δx DO' ,Δθ DO' }, the position control value of the gripper at the next moment is obtained as ξ F c =(Δx DO' +p DO' ',Δθ DO' +θ DO' ') T .

Images